English | 中文
This demo runs the MiniCPM-V family of multimodal models fully on-device on iOS, Android, and HarmonyOS NEXT. Currently supported:
This repository contains three on-device demos for MiniCPM-V (multimodal LLM) running fully locally via llama.cpp:
MiniCPM-V-demo/ — iOS demo (Xcode project)MiniCPM-V-demo-Android/ — Android demo (Gradle / Kotlin)MiniCPM-V-demo-HarmonyOS/ — HarmonyOS NEXT demo (DevEco Studio / ArkTS)All three demos share the same llama.cpp-omni submodule (branch ios-omni) at the repo root.
NOTE: This project bundles
llama.cppas a git submodule. The upstream forktc-mb/llama.cpp-omnicarries a dozen unrelated branches and a full clone weighs ~350 MB, soshallow = trueis set in.gitmodulesby default. The recommended shallow + single-branch clone is:```bash
one-shot (parent repo + submodules, all shallow)
git clone --recurse-submodules --shallow-submodules \ https://github.com/OpenBMB/MiniCPM-V-Apps.git cd MiniCPM-V-Apps ```
Or, if you've already cloned the parent repo and want to init the submodule afterwards:
bash git submodule update --init --recursive --depth 1 --single-branchThis only pulls a single commit of the
ios-omnibranch (~tens of MB) instead of the full llama.cpp-omni fork history. Developers who need to push totc-mb/llama.cpp-omni:ios-omnican rungit fetch --unshallowinside the submodule to lift the shallow restriction.
The README is organised in two parts:
Just want to install the app? Pre-built packages are below; step-by-step instructions (requirements, signing notes, MD5) are in DOWNLOAD.md. The rest of this README is only needed if you want to build from source.
| Platform | Install |
|---|---|
| 🍎 iOS / iPadOS | TestFlight — public link |
| 🤖 Android | APK — latest release |
| 📱 HarmonyOS | HAP — latest release |
NOTE: To deploy and test the app on an iOS device, you may need an Apple Developer account.
bash
xcode-select --install
* Agree to the software license agreement:
bash
sudo xcodebuild -license
* CMake ≥ 3.28 (needed by the xcframework build in the next step):
bash
brew install cmake
This repo does not track any compiled artefacts, so the prebuilt llama.xcframework (~189 MB) needs to be produced locally from the llama.cpp-omni submodule and dropped into MiniCPM-V-demo/thirdparty/ for Xcode to link against. A one-shot script is provided — by default it only builds the two slices the demo actually links (real device + simulator), which takes ~2-3 min on a modern M-series Mac:
./scripts/build_xcframework.sh
The output is installed at MiniCPM-V-demo/thirdparty/llama.xcframework/.
If you need a different build scope, the script forwards MINIMAL_MODE:
MINIMAL_MODE=ios-sim ./scripts/build_xcframework.sh # simulator only (~3 min)
MINIMAL_MODE=ios-device ./scripts/build_xcframework.sh # device only (~3 min)
MINIMAL_MODE=ios ./scripts/build_xcframework.sh # device + simulator (default, ~3 min)
MINIMAL_MODE=macos ./scripts/build_xcframework.sh # macOS slice only (not used by this demo)
This demo only links the iOS slices (device + simulator), so the default is all you need.
macosis a separate mode for the Mac-side CLI;allcurrently behaves identically toios(same branch in the script), it is not a multi-platform build.
The script drives cmake directly against the llama.cpp-omni submodule (it modifies no file inside
it), then copies the resulting llama.cpp-omni/build-apple/llama.xcframework into
MiniCPM-V-demo/thirdparty/. There is no pre-existing build script inside the submodule — if you
need custom compile flags, read scripts/build_xcframework.sh.
During the build you will see warnings like ignoring duplicate libraries and skipping debug map object with duplicate name and timestamp — these come from llama.cpp's mtmd module having identically-named .o files across different model architectures. They are harmless and the resulting framework works correctly.
When do I need to rebuild? - The parent repo bumped the llama.cpp-omni submodule pointer (
git submodule statusshows a commit different from your last local build). - You edited any source underllama.cpp-omni/that affects the framework.
OCR layout detection links Microsoft ONNX Runtime. The repo does not track onnxruntime.xcframework (~44 MB zip / ~194 MB unpacked). Fetch the public onnxruntime-c 1.21.0 CocoaPods archive (iOS device + simulator + macOS slices) with:
./scripts/download_onnxruntime.sh
The script writes MiniCPM-V-demo/thirdparty/onnxruntime.xcframework/ and skips the download if 1.21.0 is already present. Re-fetch with FORCE=1 ./scripts/download_onnxruntime.sh.
The archive URL is Microsoft's public host: https://download.onnxruntime.ai/pod-archive-onnxruntime-c-1.21.0.zip (SHA-256 is pinned in the script). This Xcode project always links that framework, so the download is required even if you are not using OCR.
Open MiniCPM-V-demo.xcodeproj at the repository root (it is not under the MiniCPM-V-demo/ subdirectory). It may take a moment for Xcode to automatically download the required dependencies.
MiniCPM-V-demo target → Signing & Capabilities
and change Team to your own account. The repo ships with the author's Team ID, so building without
changing it fails to sign.Settings → Privacy & Security → Developer Mode.If Xcode fails with There is no XCFramework found at '.../llama.xcframework', you skipped §1.1.2 — go back and run the build. If the missing path is onnxruntime.xcframework, you skipped §1.1.3.
Requirements:
27.0.12077973 and CMake 3.22.1)arm64-v8a)Build & run:
cd MiniCPM-V-demo-Android
./gradlew assembleDebug
Or open MiniCPM-V-demo-Android/ directly in Android Studio and click Run.
The first launch will download the GGUF model files into the app's external storage. You can also sideload model files manually via adb push — see in-app Model Manager for the expected directory layout.
Requirements:
arm64-v8a)Build & run:
MiniCPM-V-demo-HarmonyOS/ in DevEco Studio.File → Project Structure → Signing Configs and tick Automatically generate signature (requires a Huawei developer account; this only needs to be done once).After the first launch, open the in-app Model Manager and tap Download. You can also sideload model files via hdc file send; see MiniCPM-V-demo-HarmonyOS/README_zh.md for the expected directory layout.
The HarmonyOS port shares the exact same
llama.cpp-omnisubmodule, model catalogue, HuggingFace / ModelScope download URLs and MD5 hashes with the iOS / Android demos.
The on-device memory needed to run a model is roughly (model file size) + KV cache + a few hundred MB of working memory for the vision encoder and llama.cpp internals. The recommended values below leave enough headroom for the OS and the demo app itself.
| Model | LLM params | Recommended quant | LLM file | mmproj (f16) | Total download | Recommended device RAM |
|---|---|---|---|---|---|---|
| MiniCPM-V 2.6 | 8B | Q4_K_M | ~4.4 GB | ~1.0 GB | ~5.4 GB | ≥ 8 GB |
| MiniCPM-V 4.0 | 4.1B | Q4_K_M | ~2.0 GB | ~0.9 GB | ~2.9 GB | ≥ 6 GB |
| MiniCPM-V 4.6 | 1.3B | Q4_K_M | ~0.5 GB | ~1.1 GB | ~1.6 GB | ≥ 6 GB |
| MiniCPM-V 4.7 35B-A3B | 35B (3B active) | q3gud-im (Q3_K experts) | ~1.7 GB | ~1.1 GB | ~16.6 GB | ≥ 12 GB |
| MiniCPM-V 4.7 1B | 1B | Q4_K_M | ~0.7 GB | ~1.1 GB | ~1.8 GB | ≥ 4 GB |
| MiniCPM5-1B | 1B | Q4_K_M | ~0.5 GB | — | ~0.5 GB | ≥ 4 GB |
| MiniCPM5-2B | 2B | Q4_K_M | ~1.5 GB | — | ~1.5 GB | ≥ 6 GB |
| VoxCPM2 (TTS) | ~2B | Q4_K_M + F16 | ~1.0 GB | — | ~2.8 GB | ≥ 6 GB |
| MiniCPM-OCR 1.0 (iOS) | 752M | Q4_K_M | ~0.5 GB | ~0.9 GB | ~1.6 GB | ≥ 6 GB |
Notes:
mmproj is the vision projector + ViT weights; it is shipped in f16 because quantising the visual tower hurts perception quality noticeably more than quantising the LLM.mmproj file.mmproj either — it is a pure TTS model.mmproj is a 4x projector and is not interchangeable with the chat models' mmproj.mix_q3gud_im) rather than Q4, and its routed experts are split out of the main .gguf into a separate experts/ directory. The ~16.6 GB above is the main file + all 40 expert shards + mmproj; plan for ~17 GB of free storage. Because the experts are streamed from SSD on demand rather than held in RAM, this variant also needs fast internal storage — see §2.4.Download the language model file (e.g., ggml-model-Q4_0.gguf) and the vision model file (mmproj-model-f16.gguf) from the repository.
Download the language model file (e.g., ggml-model-Q4_K_M.gguf) and the vision model file (mmproj-model-f16.gguf) from the repository.
Download the language model file (e.g., MiniCPM-V-4_6-Q4_K_M.gguf) and the vision model file (mmproj-model-f16.gguf) from the repository.
MiniCPM-V 4.7 ships in two sizes:
.gguf into a separate experts/ directory and streamed from SSD on
demand, which is what makes a 35B-class model runnable on a phone.There are two sets, for two different purposes:
Standard quants — desktop / server. The official repos carry the usual whole-model quant range (F16, Q4_K_M, Q5_K_M, Q6_K, Q8_0, …). Pick whichever fits your machine's memory.
HuggingFace: openbmb/MiniCPM-V-4.7-35B-A3B-gguf · openbmb/MiniCPM-V-4.7-1B-gguf
ModelScope: openbmb/MiniCPM-V-4.7-35B-A3B-gguf · openbmb/MiniCPM-V-4.7-1B-gguf
On-device split — for this demo, on a phone. The -les main file plus its experts/ directory
(and the 1B revision this demo is actually built and tested against).
HuggingFace: tc-mb/MiniCPM-V-Apps-gguf
Why a separate split exists. The official repos do not carry a
-lesvariant, and this demo cannot load their 35Bmix_q3gud_im.gguf: that file keeps the routed experts inline, so a phone would need the whole ~15.5 GB as a single Metal buffer (mmap) or as resident memory (no-mmap) — either way the OS kills it. Stripping the experts intoexperts/lets them be paged in from SSD on demand, which is what makes 35B-class inference possible on a phone at all. The split is therefore an artifact made for this demo, not a general-purpose release. 1B needs no split; it is hosted above only because that is the revision this demo was validated against.
Files are grouped into per-model sub-directories:
| Variant | Files | On-device location |
|---|---|---|
| 35B-A3B | MiniCPM-V-4.7-35B-A3B/MiniCPM-V-4_7-35B-A3B-mix_q3gud_im-les.ggufMiniCPM-V-4.7-35B-A3B/mmproj-model-f16.ggufMiniCPM-V-4.7-35B-A3B/experts/ (40 shards + manifest.json) |
Documents/MiniCPMV47/ |
| 1B | MiniCPM-V-4.7-1B/MiniCPM-V-4_7-1B-Q4_K_M.ggufMiniCPM-V-4.7-1B/mmproj-model-f16.gguf |
Documents/MiniCPMV47_1B/ |
The
experts/directory must stay together with its main file. The-lesvariant carries no routed-expert weights of its own, so a mismatched pair fails the on-disk consistency check and the model refuses to load. The reliable way to get this right is to download inside the app (Settings → tap the model → Download), which fetches every file and verifies the shard sizes for you.
Download the language model file (e.g., MiniCPM5-1B-Q4_K_M.gguf) from the repository. MiniCPM5 is text-only, so no mmproj is required.
Download the language model file (e.g., MiniCPM5-2B-Q4_K_M.gguf) from the repository. MiniCPM5 is text-only, so no mmproj is required.
VoxCPM2 is a ~2B-parameter text-to-speech model. It consists of two GGUF files:
VoxCPM2-BaseLM-Q4_K_M.gguf, ~956 MB) — the MiniCPM-4 based language modelVoxCPM2-Acoustic-F16.gguf, ~1.74 GB) — the acoustic decoder (FSQ + LocEnc + LocDiT + AudioVAE)VoxCPM2 supports 30+ languages and 9 Chinese dialects. It can generate speech from text directly (voice-design mode) or clone a voice from a reference audio clip.
After downloading, place both .gguf files in the app's Documents directory (iOS) or in the per-model folder under {ExternalFilesDir}/models/ (Android / HarmonyOS). You can also download them directly inside the app via Model Manager. Once models are downloaded, select VoxCPM2 and the app switches to the TTS interface automatically.
MiniCPM-OCR 1.0 turns a photo, screenshot or PDF page into Markdown: it detects layout blocks first, then runs the vision-language model on each crop and stitches the results back into reading order (headings, paragraphs, tables, formulas and seals).
The pipeline needs three files:
MiniCPM-OCR-1_0-Q4_K_M.gguf, ~529 MB) — 752M-parameter text towerMiniCPM-OCR-1_0-mmproj-F16-4x.gguf, ~901 MB) — 4x vision projector (ViT + final merger)PPDocLayoutV3_fix4.onnx, ~142 MB) — PP-DocLayoutV3 ONNX model, run through ONNX RuntimeRoughly 1.6 GB in total. Plan for ≥ 6 GB RAM; OCR keeps the LLM context plus the vision encoder resident.
The
mmprojhere is a 4x projector (scale_factor=2) and is specific to OCR. Do not reuse the chat models'mmproj-model-f16.gguf, and do not feed this one to a chat model.
MiniCPM-OCR-1.0/)MiniCPM-OCR-1.0/)The upstream weights live in openbmb/MiniCPM-OCR-1.0-gguf. The Q4_K_M file above is quantised from the official MiniCPM-OCR-1_0-F16.gguf with llama-quantize; the mmproj is byte-identical to the official one.
Easiest path: open Settings → Model Management → MiniCPM-OCR 1.0 and tap Download All. The app races HuggingFace and ModelScope in parallel and keeps whichever responds first, so it works both inside and outside mainland China.
To place the files by hand, put all three in the app's Documents directory and keep these exact names:
Documents/
├── MiniCPM-OCR-1_0-Q4_K_M.gguf
├── MiniCPM-OCR-1_0-mmproj-F16-4x.gguf
└── PPDocLayoutV3_fix4.onnx
Then select MiniCPM-OCR 1.0 in Settings; the app switches to the OCR interface. Downloads are MD5-checked before use, so a truncated transfer is rejected instead of silently failing at load time.
Building from source additionally requires
onnxruntime.xcframework— see §1.1.3.
| 日期 | 榜单 | 排名 | 周期新增 | 总 STAR |
|---|