r/LocalLLaMA • u/Skyline34rGt • 13h ago
New Model openbmb/MiniCPM-V-4.7-35B-A3B
https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3Bgguf's: https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf
"MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 35B-A3B variant, with 35B-A3B being the first MoE model in the MiniCPM-V series. It builds on the family's single-image, multi-image, and video understanding capabilities, combining stronger foundation capabilities with efficient visual encoding and on-device inference. Notable features of MiniCPM-V 4.7 include:
- š± The First Open-Source MoE Model with Lossless, Smooth On-Device Inference. With our Lookahead Expert Scheduling (LES) technique introduced alongside MiniCPM-V 4.7, combined with fine-grained layer-wise quantization and DSpark speculative sampling integration, MiniCPM-V 4.7 35B-A3B becomes the first open-source MoE model to run smoothly on-device without compromising performance, achieving a decoding speed of 13+ tokens/s on iPhone 17 Pro Max.
- š„ A New Pareto Frontier in Performance and Efficiency. As model capabilities continue to advance, inference efficiency has become an important measure of a model's value. On resource-constrained devices, delivering the strongest possible capabilities within a limited resource budget calls for higher capability density. The 35B-A3B variant of MiniCPM-V 4.7 advances the performanceāefficiency Pareto frontier for MLLMs. In Instruct mode, 35B-A3B achieves a higher overall score in less inference time than Qwen3.6 35B-A3B.
- š Efficient Architecture. Based on the intra-ViT early compression technique from LLaVA-UHD v4, MiniCPM-V 4.7 compresses visual tokens early within the vision encoder, reducing computation in subsequent encoder layers. It also supports mixed 4x/16x visual token compression, allowing developers to balance visual detail and inference cost across tasks.
- š» Broad Platform and Chip Support. MiniCPM-V 4.7 supports iOS, Android, and HarmonyOS, together with open-source on-device deployment support. It also provides day-0 support for chip platforms from Rockchip, Houmo, Intel, AMD, and Arm, extending deployment options across mobile and local computing devices.
- š ļø Developer Friendly. MiniCPM-V 4.7 supports inference frameworks including SGLang, vLLM, llama.cpp, and Ollama, along with fine-tuning frameworks including SWIFT and LLaMA-Factory. It provides GGUF, BNB, AWQ, and GPTQ quantized variants and adds FP8 quantization support. We also release Self-Replay Fine-Tuning, a fine-tuning approach that helps developers extend model capabilities at low cost while reducing forgetting of existing capabilities, making further customization easier."
22
u/Skyline34rGt 13h ago
5
u/jojotdfb 12h ago
So, use it for tool calling and not coding.
16
u/EvolvingDior 11h ago
LiveCodeBench says you should prefer Gemma4-31b to Q38-27b for coding. ĀÆ\(ć)/ĀÆ
1
u/DigiDecode_ 2h ago
the footnote says "instruct mode" not thinking mode, I might be wrong, but I believe these are non-thinking scores
0
u/Suspicious_Moment_87 9h ago
So the point is, qwen 3.6 is much better than 3.8 ?
1
u/Disastrous_Food_2428 48m ago
It's way worse than Qwen3.8-27B when it comes to coding capabilities. Honestly, other than being fast, it's pretty mediocre overall.
40
28
25
u/danigoncalves llama.cpp 13h ago
Comparing with 3.8 27B is really bold.
21
9
u/No-Name-Person111 12h ago
Iām going to do some A/B testing on this today and see what it looks like in practice. Iām very tired of existing benchmarks and have come to value TTFT and TTFA over many of these other metrics.
1
u/danigoncalves llama.cpp 12h ago edited 12h ago
Actually what I do is battle test on my repos and give it clear tasks. At the end I evaluate the code or if one agent was able to fullfill a specifiv assignment and infer from that. Tomorrow I will give it a all day spin to see how it behaves
3
u/Several-Tax31 11h ago
Let us know. If they compare with 27B with reasoning off, not a meaningful results. On the other hand, I'm still looking for things better than 35B, so that might be it...Ā
1
u/Disastrous_Food_2428 49m ago
It's way worse than Qwen3.8-27B when it comes to coding capabilities. Honestly, other than being fast, it's pretty mediocre overall.
1
u/No-Name-Person111 10h ago
Maybe itās a chat template issue, llama.cpp, or something else, but this model justā¦isnāt good. Itās fast. But it jumps to conclusions and seems more eager to deliver an answer than deliver a good answer.
No thinking enabled is straight up hallucination territory.
Thinking enabled gets decent results, but thereās no depth or substance.
1
u/fatihmtlm 8h ago
Have you built llamacpp from the source; https://github.com/ggml-org/llama.cpp/pull/29416
2
u/Educational-Region98 12h ago
It is what everyone wants though. I know that would be my first questjon
1
8
9
u/Skyline34rGt 12h ago
They release today also new tiny Minicpm v4.7 1B model (1.3B) - post here https://www.reddit.com/r/LocalLLaMA/comments/1x492f9/openbmbminicpmv471b/
7
u/BreakfastFriendly728 12h ago
On iPad? SSD?
3
u/banana_slurp_jug 11h ago
Well the M4 iPad Pro's disk read write speeds range from 1500MB/s for the 256GB version to 3000MB/s for the 1TB version which are well within SSD speeds (I'm pretty sure it is one soldered in there anyways)
4
u/MerePotato 13h ago
No code benchmarks other than livecodebench?
14
u/DrBattletoad 13h ago
Their V-models focus on image and video understanding. It says so in the 2nd sentence of theĀ model card: "It builds on the family's single-image, multi-image, and video understanding capabilities, combining stronger foundation capabilities with efficient visual encoding and on-device inference"Ā
The benchmarks are for "Vision-Language" and "Language", not for coding.Ā
6
u/Outdatedm3m3s 13h ago
Wait is this actually on par with qwen 27b???
19
u/AlternateWitness 11h ago
Qwen 3.8 27b without reasoning*. So, you know. If you turned off the underlying feature that made the model what we know it as.
5
2
u/OcelotOk8071 10h ago edited 6h ago
we need llama.cpp support (edit: i mean for expert lookahead)
1
u/Chance-Research-9302 10h ago
general.architecture qwen35moe
Should work on any recent llamacpp build
1
4
u/gardenvarietyzombie 10h ago
From the acknowledgements section of the model card:
We would like to thank the following projects: Qwen3.6 for providing the language backbone
So does that mean it is a finetune?
3
2
u/killerstreak976 transformers 12h ago
This is the the only qwen finetune I've been eagerly waiting for
1
1
u/quadra-lab 13h ago
Seeing the 13tks on iPhone 17 Pro is impressive. Though I hugely doubt that any HarmonyOS devices will be able to run this thing in tk/s
1
u/geldonyetich 12h ago edited 12h ago
I had to make do with battery power for a month when the fusebox was damaged by a water leak and insurance didn't want to pay for it. At that point, my Android Phone became my desktop computer.
It worked better than expected. Models like this open the door to the possibility that even local LLM inference might be an option in that scenario.
I've been tempted to pull the trigger on an iPhone 18 because those benchmarks are fire. I've so far managed not to have my smart phone pulled into the Apple cult, but Qualcomm better get a move on.
0
u/CiobanuXashi 11h ago
13+ tokens/s on a phone for a 35B MoE that also does video, and local vision finally feels practical instead of theoretical.
1
1
u/Ecstatic-Wash-7667 10h ago
Wait wait wait so itās not just another finetune of qwen???????????? Did not expect this if true
1
u/zanar97862 6h ago
Of course it is, all 35B A3B models you see are Qwen fine tunes. That's not a bad thing its a good base.
1
1
u/Equivalent_Bit_461 8h ago
I wonder how good it would be at image sorting and cataloguing but also rating images based on given parameters, could be a good image sorterĀ
1
1
1
u/pseudonerv 13h ago
No way itās better than qwen 3.8 27b. Anybody has tried it?
5
0
u/nunodonato 13h ago
only in very specific benchmarks, most of the common ones were left one. compare at https://tinyleague.nunodonato.com
-1
-1
-11
u/MythOfDarkness 13h ago edited 13h ago
So it's not a Qwen finetune? They still only compared it to 3.6 :/
NVM, it's a Qwen 3.5 finetune. FUCK!
8
2
u/jacek2023 llama.cpp 13h ago
what's the source of information that it is a qwen finetune?
4
u/CoolConfusion434 12h ago
From their HF:
We would like to thank the following projects:
- Qwen3.6 for providing the language backbone
- SigLIP2 for providing the vision understanding module
- Transformers
2
u/MythOfDarkness 13h ago
It actually is.
https://huggingface.co/openbmb/MiniCPM-V-4.7-35B-A3B-gguf?show_file_info=MiniCPM-V-4_7-35B-A3B-Q4_K_S.gguf
It's in the GGUF metadata.1
0
28
u/jacek2023 llama.cpp 13h ago
It's nice to see it resurrected!!! (It was posted here on r/LocalLLaMA, then changed to 404) In the meantime, support for it was merged into llama.cpp, so we can finally run it!
https://www.reddit.com/r/LocalLLaMA/comments/1x2qsh3/model_support_minicpmv_47_by_tcmb_pull_request/