Speech Swift

AI speech models for Apple Silicon, powered by MLX Swift and CoreML.

📖 Read in: English · 中文 · 日本語 · 한국어 · Español · Deutsch · Français · हिन्दी · Português · Русский · العربية · Tiếng Việt · Türkçe · ไทย

On-device speech recognition, synthesis, and understanding for Mac and iOS. Runs locally on Apple Silicon — no cloud, no API keys, no data leaves your device.

📚 Full Documentation → · 🤗 HuggingFace Models · 📝 Blog · 💬 Discord

Local Speech AI on a MacBook — watch the 4-minute open-source library tour on YouTube

Use cases: Voice Agents · Transcription · Speech Generation

Built with Speech Swift

16 public repositories with verifiable Speech Swift package references.

AnythingLLM · Palmier Pro · Anarlog · ClawdHome · Jabber · Ora · VoxFlow · LokalBot · Voicey · HushType · DexDictate macOS · Watchtower · Wishper App · FriSpeak · Scribe · VoicePen

Capability groups: STT / ASR · Alignment · TTS · LLMs & translation · Speech-to-speech · Enhancement/restoration · Source separation · Music/audio generation · Wake word, VAD, diarization & speaker identity

STT / ASR

• Qwen3-ASR — Speech-to-text (automatic speech recognition, 52 languages, MLX + CoreML)
• WhisperASR — Whisper Large-v3 Turbo speech-to-text via native CoreML runtime (ANE, multilingual)
• MOSS Transcribe Diarize — Native CoreML/MLX offline transcription with model-generated speaker labels and timestamps (128K MLX context; INT5/INT8)
• Parakeet TDT — Speech-to-text via CoreML (Neural Engine, NVIDIA FastConformer + TDT decoder, 25 languages)
• Omnilingual ASR — Speech-to-text (Meta wav2vec2 + CTC, 1,672 languages across 32 scripts, CoreML 300M + MLX 300M/1B/3B/7B) — 0.28 RTF on iPhone 16 Pro
• Cohere Transcribe 2B — Native MLX speech-to-text (14 languages, FP16/INT5/INT8)
• Kyutai STT — Native MLX streaming speech recognition (1B English/French; 2.6B English configuration; 24 kHz Mimi; FP16/INT4/INT8 exports) · INT8 (default) · INT4 · FP16
• Voxtral Mini 3B — Native MLX speech-to-text (8 languages, FP16/INT5/INT8) — 0.074 RTF on M5 Pro
• Streaming Dictation — Real-time dictation with partials and end-of-utterance detection (Parakeet-EOU-120M) — 0.04 RTF on iPhone 16 Pro
• Nemotron Streaming (Multilingual) — Low-latency streaming ASR with native punctuation and capitalization (NVIDIA Nemotron-3.5-ASR-Streaming-0.6B, CoreML + MLX, 40 language-locales)
• Nemotron Streaming (English) — Low-latency streaming ASR with native punctuation and capitalization (NVIDIA Nemotron-Speech-Streaming-0.6B, CoreML, English-only, smaller and faster than the multilingual variant)

Alignment

• Qwen3-ForcedAligner — Word-level timestamp alignment (audio + text → timestamps)

TTS / Speech Generation

• Qwen3-TTS — Text-to-speech (highest quality, streaming, custom speakers, 10 languages)
• CosyVoice TTS — Streaming TTS with voice cloning, multi-speaker dialogue, emotion tags (9 languages)
• VoxCPM2 — 48 kHz studio-quality TTS with voice cloning + instruction-driven voice design (2B, MLX bf16/int8, 30 languages)
• IndexTTS2 — Native MLX voice cloning from a reference voice (IndexTeam IndexTTS-2, 1.5B-class fp16 bundle, speaker/emotion/pause controls)
• F5-TTS — Zero-shot voice cloning from a short reference clip + transcript (SWivid F5-TTS v1 Base, DiT flow matching + Vocos, MLX fp16, 24 kHz, English + Mandarin; non-commercial license)
• Higgs TTS 3 — Conversational TTS with zero-shot voice cloning and inline emotion/style/SFX/prosody tags (Boson Higgs TTS 3, Qwen3-4B backbone, MLX bf16, 24 kHz, 100+ languages; research/non-commercial license)
• Breeze TTS 2 — English/Chinese TTS with voice design, reference cloning and direction, and streamed audio (MLX BF16/8-bit, 24 kHz; research/non-commercial license)
• Kokoro TTS — On-device TTS (82M, CoreML/Neural Engine, 54 voices, iOS-ready, 10 languages + two German voice packs) — 0.08 RTF on iPhone 16 Pro
• VibeVoice TTS — Long-form / multi-speaker TTS (Microsoft VibeVoice Realtime-0.5B + 1.5B, MLX, up to 90-min podcast/audiobook synthesis, EN/ZH)
• Magpie TTS — Multilingual TTS (NVIDIA Magpie-TTS Multilingual 357M, MLX INT8 411 MB or CoreML INT8 342 MB, 9 languages, 5 baked speakers, streaming on MLX)
• MOSS-TTS-Nano — Streaming TTS and voice cloning without a transcript (CoreML, 48 kHz stereo, 80 ms chunks, 367.6 MB compiled graphs; macOS 15+ / iOS 18+)
• Supertonic TTS — On-device flow-matching TTS (Supertone Supertonic-3 99M, CoreML/Neural Engine, 31 languages, 10 voices, G2P-free, 44.1 kHz) — 0.15 RTF on iPhone 16 Pro
• Chatterbox TTS — Multilingual TTS with zero-shot voice cloning (Resemble AI Chatterbox Multilingual, MLX fp16 ~1.3 GB, 23 runtime languages; Hebrew requires niqqud, MIT) plus Chatterbox Flash Core ML text-to-waveform from precomputed reference conditioning.
• OmniVoice TTS — Non-autoregressive diffusion TTS with zero-shot voice cloning (k2-fsa OmniVoice, Qwen3 backbone, MLX fp16 default / int8 available, 600+ languages, Apache-2.0)
• Indic-Mio — Hindi/Indic TTS with inline emotion markers and optional reference-voice cloning (MLX, 24 kHz)
• CSM (Conversational Speech Model) — Conversational text-to-speech with zero-shot voice cloning from a reference clip + transcript (Sesame CSM-1B, Llama-1B backbone + Mimi codec, MLX int8/fp16, 24 kHz, English; Apache-2.0)

LLMs & Translation

• Qwen3Chat — On-device LLM chat (Qwen3.5-0.8B MLX/CoreML plus dense Qwen3 4B and Gemma 4 E2B/E4B MLX backends, streaming tokens)
• FunctionGemma — On-device LLM for structured function / tool calls (Gemma 3 270M, CoreML 8-bit palettized, Neural Engine, ~252 tok/s on M5 Pro · 128 tok/s on iPhone 16 Pro)
• MADLAD-400 — Many-to-many translation across 400+ languages (3B, MLX INT4 + INT8, T5 v1.1, Apache 2.0)
• GLiNER2.5-Decide - Local schema-conditioned text classification and entity spans (DeBERTa-v3-large encoder, MLX, INT8/FP16/FP32; development preview, Apache 2.0)
• Clef-flash — Local decisions from text and allowed answers (9B, MLX 4-bit; devel