audio.cpp

audio.cpp is a high-performance C++ audio inference framework built on top of ggml, designed to make modern local audio models practical, portable, and fast.

Tired of juggling a dozen Conda environments, hundreds of Python packages, and dependency conflicts just to try a few audio models? audio.cpp gives those paths a shared native runtime instead. Runs on Windows, Linux, and macOS, with support for NVIDIA, AMD, Apple Silicon, and CPU-only machines.

Lost in a sea of audio model names, papers, and codebases? Explore 100+ audio models in the Audio Model Architecture Atlas to see how they're built, what they share, and where they differ, whether you're learning or building your own.

[!TIP]

v0.9.3 highlights: LoRA support across 11 models, with adapter loading, strength controls, Studio integration, and a shared tool for merging adapters into GGUF packages. See the LoRA guide for supported models and formats.

Performance optimizations and performance modes: Faster execution across speech and music models, with perf_mode presets to select model-specific optimizations. Tested CUDA presets reduce RTF by 27% for Qwen3-TTS streaming, 25–26% for BreezeTTS2, 16% for VoxCPM2, 15–47% for YuE2, 59–62% for ACE-Step, and 33% for MiniMax Music3 versus the paths before these optimizations. Some models also benefit from further reductions in peak VRAM usage.

Many performance optimizations are disabled by default for broader compatibility. Not happy with performance? Enable the cuda_perf preset or create your own preset for your hardware!

New TTS app in the WebUI: Turn scripts into narration in a dedicated workspace. Import text, split it into passages, and assign each passage a reusable speaker with its own model, voice, and generation settings. Generate or refine passages individually, compare takes, edit waveforms and pauses, then combine, enhance, and export your finished audio. Projects and speaker presets are saved for reuse.

AMD support and HIP test releases: Thanks to AMD for providing audio.cpp with access to AMD DevLab! We now offer HIP release builds for testing on AMD GPUs.

[!IMPORTANT]

• 130+ audio model families and 200+ variants, running locally. TTS, ASR, music generation, voice conversion, separation, and more through a shared CLI and server API.

• Built-in WebUI: Generate and clone voices in Studio, compare models side by side in Arena, manage models and voice/speaker libraries, and create narration with passage editing, multiple speakers, and audio assembly in the TTS app.

• Performance: Selected CUDA TTS paths run 1.8–8× faster than Python; tested Q8 packages offer up to 1.53× speedup and 37% lower peak VRAM versus 16-bit weights (report). VibeVoice 1.5B reaches 5.15× real time, and Supertonic 3 generates 10 hours of audio in 3 minutes on RTX 5090 (demo). Nemotron 3.5 ASR matches WER at roughly ¼ the wall time in the TranscrIA benchmark.

• Integrations and deployment: audio.cpp powers speech, music, and speech-to-text in Unsloth Studio. Deploy Fun-ASR-Nano and SenseVoice through the FunASR platform!

It is built for real end-to-end execution rather than one-off model demos: the same runtime powers TTS, voice cloning, voice conversion, ASR, diarization, VAD, source separation, alignment, codec-style models, and higher-level workflows through a common framework surface.

Highlights:

• Parity. Strong parity tooling against Python reference paths.
• Performance. Performance-focused execution, reusable sessions, and batch-style offline inference. Optimized for CUDA.
• Portability. A portable native stack centered on ggml, with CUDA, HIP/ROCm, Vulkan, Metal, and CPU backends behind shared CLI and server entry points instead of Python-only deployment paths.
• Pipelines. Experimental JSON pipeline support for higher-level multi-step workflows.
• Audio Utilities. Built-in denoise, enhancement, resampling, and STFT/ISTFT utilities for real production-style task paths. Model-based utilities need separately downloaded weights; see downloads and usage.

The goal of the framework is to provide highly optimized, reusable building blocks for audio-related models, so new model integrations can be brought up faster, shared components can be improved once and benefit many families, and real end-to-end inference paths can stay efficient, maintainable, and portable.

audio.cpp would not be moving this quickly without generous contributors bringing in real fixes, new capabilities, and careful polish. See CONTRIBUTING.md for how to contribute and for a shout-out to the people already helping shape the project.

[!TIP] Contribution focus: the most helpful contributions right now are improvements to the UI, API server, and pipeline/workflow subsystems. These areas make the existing model surface easier to use, serve, compose, and validate. See CONTRIBUTING.md for more details.

New model PRs: before starting a new model port, please check the supported model table because several families are already implemented or under testing and read the New Model PRs section in CONTRIBUTING.md. New ports should start under the community model source tree, where review is lighter than core models but still needs reproducible validation.

News

[!IMPORTANT] Release v0.9.3: New models include Fish Audio Drama3 and Drama3-small, Ming 3.0 TTS, Ming 3.0 Audio Tiny, Coqui XTTS v2, Plapre 3, Gradium Phonon, and Earshot VAD, plus experimental LatentSync 1.6 lip synchronization. Existing families gain Irodori-TTS MeanFlow and multilingual Granite TurboCTC support.

New features include LoRA support across 11 models, Studio adapter controls and GGUF merging, a TTS workspace with reusable speakers, Ming3-TTS streaming, Kokoro voice blending, and built-in recipes for voice cloning and cross-language dubbing. Performance optimizations and perf_mode presets improve speech and music generation, with further memory reductions for selected models. Hardware support expands with HIP test releases, more ARM NEON acceleration, and additional Rockchip NPU paths and Docker images. Thanks to everyone contributing code, testing, feedback, and hardware access!

Release v0.9.2: New models include Voxtral TTS, TF-GridNet, MossFormer2, Hviske v6, FireRed VAD, AST, CED, and wake-word detection models. This release expands TTS streaming, adds multi-turn LFM2.5-Audio conversations, and improves model performance and compatibility. The WebUI gains a shared Voice Library and layout improvements. Downloadable C API libraries, Windows Vulkan portable builds, and packaged server configuration examples make integration easier. Thanks to everyone contributing code, testing, and feedback!

2026-09-30 - Release v0.9.0: 100+ audio models, an enhancement and denoising WebUI tab, speaker-tagged ASR, and performance and memory improvements. Explore the Architecture Atlas. Thanks to our community!

2026-06-25 to 2026-09-23 (releases 0.1 to 0.8): audio.cpp grew to 80+ model families and 120+ variants across TTS, ASR, music generation, voice conversion, separation, and speech analysis. Major additions included the native WebUI and Arena, GGUF packaging, LoRA adapters, streaming, and CUDA/Metal performance improvements, alongside Vulkan and experimental HIP support. The C ABI, Colab integration, MP3/HTTPS frontends, and models such as YuE2 and SheetSage2 expanded integration and audio-to-score workflows.

Supported Models

Task tags: TTS text to speech, Clone voice cloning, VC voice conversion, S2S speech-to-speech, ASR speech recognition, Classify audio classification, Wake wake-word detection, Align forced alignment, VAD voice activity detection, Diar speaker diarization, Codec audio codec, Sep source separation, MIDI audio-to-symbolic MIDI/events, Music music/song generation, SFX sound effects, Video video generation, Edit audio/music editing, Design voice design, Dialogue multi-speaker dialogue TTS,