China Hot Tech @ChinaHotTech
← All posts
Models · On-Device · Open Source

MiniCPM-V 4.7: Open On-Device MoE Multimodal with LES

MiniCPM-V 4.7 from 面壁智能 ModelBest / OpenBMB is the first open-source MoE multimodal in the MiniCPM-V line that the team says can run smoothly on-device without capability sacrifice* — via Lookahead Expert Scheduling (LES), fine-grained layer-wise quantization, and DSpark speculative sampling. The headline SKU is 35B-A3B (~35.2B total / ~3.5B activated per token; 256 experts, 8/token; backbone qwen3_5_moe_text), paired with a dense 1B (~1.55B) sibling. Both share a 256K context and a SigLIP canvas M-RoPE vision tower. What you can download today is open weights + demo apps — not a phone you buy in a store. Vendor on-device and Pareto claims below are flagged with asterisks.

· ~7 min read · China Hot Tech

Soft lilac, jade, and warm paper-cream editorial: phone screen filled with MoE expert tiles while expert shards stream from an SSD experts/ block — MiniCPM-V 4.7 LES on-device MoE

Illustration: lilac / jade / cream — phone + expert shards streaming from SSD. Distinct from recent paper/coral, fog-blue/amber, mint/cream, and sand/copper art. Asterisked labels are vendor / OpenBMB figures.

MoE models already win the “more total parameters than you activate” bet on servers. Phones are a different constraint: total expert weight still has to live somewhere, device RAM cannot hold every expert, and naïve load-from-SSD stalls kill interactive decode. Restricting the expert pool per prompt can keep latency down — and quietly drop experts the router would have picked. MiniCPM-V 4.7’s pitch is that you do not have to choose: keep token-level routing intact, prefetch the next layer’s likely experts, and stream the rest from storage fast enough that a 35B-class multimodal feels interactive. For overseas engineers the useful checks are which files to pull, what LES actually guarantees, and which speed/score numbers are OpenBMB’s own benches.

What shipped

Per the OpenBMB docs and Hugging Face cards:

Can you use it today?

As weights + inference stacks — yes. Hugging Face: openbmb/MiniCPM-V-4.7-35B-A3B and MiniCPM-V-4.7-1B (plus GGUF / BNB / AWQ / GPTQ / FP8 variants). Native Transformers path needs transformers >= 5.18.0 with AutoModelForImageTextToText — no trust_remote_code. Serving: SGLang (fork branch noted in docs), vLLM, llama.cpp, Ollama. Fine-tune: SWIFT, LLaMA-Factory; OpenBMB also published an SRFT (Self-Replay Fine-Tuning) recipe. On-device demos for iOS, Android, and HarmonyOS live in MiniCPM-V-Apps (TestFlight / APK / HAP). Day-0 chip paths listed: Rockchip, Houmo, Intel, AMD, Arm; OpenVINO notebook for Intel. As a retail phone SKU or App Store product — no. This is open weights and developer demo apps.

Three-panel editorial: LES lookahead expert scheduling, MoE-SSD expert shards on storage, DSpark speculative sampling — plus platforms iOS Android HarmonyOS and SGLang vLLM llama.cpp Ollama

LES · MoE-SSD · DSpark — the three pieces OpenBMB says make on-device MoE interactive without freezing the router. Asterisks mark vendor / OpenBMB on-device and bench figures.

LES, MoE-SSD, and what “lossless” means*

OpenBMB’s write-up is unusually concrete about the failure modes they are trying to avoid:

The phone-oriented GGUF split is called out in MiniCPM-V-Apps as MoE-SSD: routed experts are stripped from the main .gguf into an experts/ directory (40 shards + manifest for the demo pack) and streamed from SSD on demand. That is why a 35B-class multimodal can run with a small resident working set — OpenBMB’s headline number is ~3 GB memory and 13+ tokens/s decode on iPhone 17 Pro Max* with LES + layer-wise quant + DSpark. Plan storage separately: the demo’s on-device 35B pack is roughly ~16.6 GB download (main -les file + experts + mmproj) and the apps README recommends ≥ 12 GB device RAM for that variant; devices with less RAM fall back to a smaller expert cache and slow down. Prefer that OpenBMB “13+” figure over older media paraphrases that floated “20+” or “iPhone 18” wording.

Benches* (vendor / OpenBMB)

Prefer the HF card and OpenBMB docs over secondary blogs:

These are OpenBMB’s own comparisons, not independent re-runs. Treat them as directional signals for capability density, then verify on your tasks (OCR, multi-image, video) with the cookbook prompts.

What changes for an overseas stack

If you already ship llama.cpp / Ollama edge demos, or serve Qwen3.5/3.6 VLMs behind vLLM, MiniCPM-V 4.7 changes a few practical checks:

Caveats

Bottom line: MiniCPM-V 4.7 is OpenBMB’s bid that a 35B-class multimodal MoE can be interactive on a phone without freezing the router — LES for prefetch, MoE-SSD for storage, DSpark for decode. Overseas teams can pull HF weights and the Apps demos today; keep asterisks on every speed and leaderboard claim, and budget disk as carefully as RAM.

Sources