r/LocalLLaMA • u/d77chong • 14h ago
New Model Boole-1.7B: A 1-bit Qwen3-1.7B model that outperforms Bonsai-1.7B
We're launching Precision Numerics by releasing Boole-1.7B, a 1-bit version of Qwen3-1.7B.
- 247 MB packed weights: 13.9× smaller than BF16.
- 85.9% of the BF16 model’s average score across our 10-task evaluation - outperforms 1-bit Bonsai-1.7B and nearly matches 1.58-bit Bonsai-1.7B.
Try it locally in your browser (it's for low-latency tasks using English and non-thinking mode), or use it with vLLM or llama.cpp.
We'll ship more capable models for the open-source community. Let us know what you what models/capabilities you want to see next.
- Blog: precisionnumerics.ai/blog/boole-1p7b
- Weights: huggingface.co/precision-numerics/Boole-1.7B (GGUF)
- Demo code (for vLLM, llama.cpp): github.com/PrecisionNumerics/boole-demo
- Local chat in browser: precisionnumerics.ai/chat/
- X: https://x.com/precision_num/status/2109309446379950447
35
u/StupidScaredSquirrel 14h ago
This field needs to learn about error bars ffs
2
u/d77chong 13h ago
We didn't notice severe fluctuations across runs, and some evals are averages across multiple runs.
We'll make sure to include it in future releases, thanks for the feedback!
7
u/Fedor_Doc 13h ago edited 17m ago
Overfitted on one-turn conversation, incoherent in simple chat. Contradicts itself in one parapgraph easily.
But it outputs something readable at this compression ratio, this is a success in itself
10
u/valarauca14 13h ago
Contradicts itself in one parapgraph easily
Granted; Opus-5.5 does that all that time
Yes! [10 sentences of slop]. One load bearing assumption you need to consider; this won't actually fucking work but I literally cannot disagree with you.
5
1
u/d77chong 13h ago
Thanks for the feedback, we'll fix this for our next model. Let us know what you'd like to see next!
4
u/x_MASE_x 13h ago
What can we do with this model realistically? Like is it just proving it's smaller or is it actually useful for something?
3
u/d77chong 13h ago
It works okay for simple tasks - I'd say it is a mix of both, we wanted a low-latency model and see how far it could go.
What would make a small model useful for you? We'll try to ship it in future models!
2
5
u/geldonyetich 13h ago edited 13h ago
Not bad at all!
While we're greedy for the next big intelligence boost on local inference, we often miss that the niche of punching well above the weight with smaller parameter models is still there. Hence we end up with Qwen 3.8 on under 6GB of VRAM (Ternary Bonsai 2) before we end up with Qwen 4.0.
2
u/d77chong 13h ago
Like in previous papers, we've also noticed smaller models are harder to quantize - our next models should be better!
2
2
u/abajinn 14h ago
What is it good at.
2
u/d77chong 13h ago
It's pretty good for simple writing and math tasks - you can try chatting with it on your phone/pc in your browser (supports both cpu and gpu, and recommend using English).
1
1
u/tiffanytrashcan 12h ago
I want to see this type of extreme quantization done to an MoE model!
Ling3-Tiny is probably the perfect size I would ask for, 8B-1A.
1
1
u/Ardalok 12h ago
It seems to me that tests like these on small models are useless, because even without quantization they're quite bad. You should quantize the models that can at least hold some kind of conversation at Q4. I know this may sound rude, but I sincerely think there's little benefit to this.
2
1
u/celsowm 14h ago
please uses liquid-ai model next time, their 2.6b model is incredible and I think 1-bit of it gonna rocks
6
u/pmttyji 14h ago
LFM models are small already. Also during Version 2.5, LFM introduced QAD(Q4 quant) which's giving 96% of BF16. They optimized models even for CPU. Additionally they provided DSpark for ~3x performance. So it already rocks.
1
u/d77chong 13h ago edited 13h ago
Yes we compared with LiquidAI's QAD models as well - their models are really good!
1
u/Snoo_28140 13h ago
Cool, but gemma4 e2b q4 qat which takes LESS vram to use (~1.6 vram + around 2gb ram).
I haven't seen a model with a better balance of resources and quality then this.
1
1
u/AltruisticList6000 14h ago
I tried it for the lols and for some reason the reasoning is not working, as in it keeps skipping the reasoning block, leaving it empty.
1
u/d77chong 13h ago
Yes we focused on low-latency, so non-thinking was our focus - this won't be the case for future models.
2
u/AltruisticList6000 13h ago
Oh alright, I was confused because the jinja it ships with has the reasoning still in it and since reasoning was enabled on my textgen webui (which detects from the jinja), it just kept adding empty blocks. Thought it would come with non-reasoning jinja if it was not a reasoning model.
2
u/d77chong 13h ago
Yeah we'll make sure this won't happen in future models - thanks for the feedback!
2
58
u/seamonn 14h ago
How much brain damage?