See what models you can run locally with its rank and the command to run it
Compare them with API models on the same score
Building it made me realise a 32GB Mac can now run a model that scores on-par with GPT-5, completely locally and for free (e.g. Qwen3.8 27B at 4-bit).
Each model gets one score fitted across 19,731 published benchmark results, and every result links to its source. The method is at whatmodel.sh/method, and the data, scoring code and bench script are open source: github.com/jakemanger/whatmodel
It works from the terminal too: curl whatmodel.sh
If you'd like to contribute, there's a bench script to collect real-world tok/s and you can also submit any new models and benchmark scores along with their source on the github.
these tools *always* undervalue MoEs (35b-a3b works shockingly good even on 8gb vram + 32gb system ram) and over estimate what fits on my system (27b absolutely does not run on 8gb vram unless i want 0.1 tok/s)
Agreed 35b-a3b absolutely rocks on the snapdragon X lenovo mini pc (32gb LPDDR5) I just got for £300 - I'm getting 50+tok/s read and 20tok/s output on CPU inference 👍
I agree with the other reply, qwen 3.6 35b-a3b or one of the finetunes (ornith, kat coder, you just have to try them and see how they feel for you) has been the best sweetspot for me, you have to look into offloading them properly though 'auto' settings on llamacpp these days is pretty solid in my experience, you can push it even further with one of the APEX quantizations to get more context and maybe a little more speed out of it too
The biggest difference would be that whatmodel is for llms, image, video, tts and stt currently (and more types of models soon), while llmfit is more focussed on llms specifically.
That also looks like a tui whereas this is a site and can be used with curl. Thanks for sharing it. I didn't know that existed and looks pretty cool.
I'm currently trying to get TPS from crowdsourcing (there is a bench script users can run and it automatically helps them add github issues to contribute to the leaderboard).
Very cool, I have 2x 3080 20gbs and am already running Qwen 3.8. It would be interesting to include a custom ram/vram button as 40/80 GB is not a valid format.
I tested it for my system, it says i can run qwen 3.8 flash next through ollama, which was marked as slow. While i run it through strata which is actually really fast. Soon this mtp will also be available in llama.cpp btw. Maybe you can add this new technique to the mix, since i'm quite sure smart offloading into ram is the future for local models.
Just a few models use this architecture, since it's much more efficient than loading entire models into vram or ram. It even uses ssd for some parts of the model. Qwen 3.8 flash next uses this as a preview of qwen 4, which is supposed to come out soon. I also read that a deepseek model supports this, but i haven't worked with that yet. The thing is that this makes me run a 125b model on my pc with about 100 token/s output. Which is really fast for a model that is loaded into ram for the most part.
How are the RAM numbers calculated? You say they include context, but not how much. If it’s the model’s max context (e.g. 1M tokens), that would explain why a 4B model shows ~10 GB, which is misleading. Could you state the context size used, or add a slider for it?
The RAM numbers look low to me, I'm assuming not full context. I would also like to know.
Also noticing the scores do not change with the quantization level, are the scores just for full precision? since it mixes them that could be misleading.
You're right for both. The scores are mostly full precision for now (depends on what the study or database provided and many weirdly don't even provide precision frustratingly enough).
I would like to improve that and hope others contributing can improve how quantisation reduces score. Not sure if I should have a score range in those cases or just on the model page, have each score for each quantisation (although that will require even more data)
Good point. It's not the max context and is an estimate from the download size and an approximate modest context. A context slider is a really good idea and I'll add it.
I also have a benchmark script on the site that I hope users can use to add more real-world data so those estimates aren't needed. If you have any data to share, each page has a contribute button to quickly add it
good point! just put in as "apple silicon" for now and set to 64gb or 128gb or 256 if you have 2 128gb sparks (whatever RAM you have). Apple silicon is also unified memory so will filter the same.
But will add a dedicated "unified memory" option for this
I love this interface! It's simple and elegant. The info density is high, and it's based on live, open, crowd-sourced data. What's not to love? It let's you drill down gracefully to unpack deeper levels of detail. The fit and finish on the TUI may be the icing on the cake, but I think the TUI aesthetic is just a hint for those paying attention that the real gem baked inside is the curl interface.
The author has clearly put a lot of thought and effort into this project, and I love the style!
You keep remaining chunk in ram and either swap or use cpu for computation. It's typically much slower due to lower ram bandwidth. For llm you need to read all weights each token, so if you have 20gb model and 40 gb/s ram speed, you will get 2 t/s max.
Right now that means it fits using plain old RAM instead of VRAM or unified memory, and that can be overridden if I get a data point saying someone ran X model using the same or a smaller RAM/VRAM combo
However, I'm working on revamping that to account for approaches like strata. If you have any suggested improvements or additions for the data there's also contribute buttons everywhere
Why do you even have Closed models on this? You don't need to care about RAM or GPU in that case. Once you turn on "RAM" or "GPU" values it should immediately filter all closed models from the list...I shouldn't even have to select "Local" it should do it automatically once you've selected any of the ram or gpu values.
It's also wrong anyway because it says I can run Qwen 3.8 Next Flash 2 bit for my configuration, but I'm able to run 3 bit just fine (using Strata)
Because its useful to know alternatives and how they rank on the list? Also have you never heard of YMMV? Its brilliant to say that something works for your configuration without stating said configuration
Main reason is for a comparison (is this as good as GPT-4, 5, 6 etc). And I agree the local not being immediately selected is a bit confusing there. Changing that behaviour now.
Each page can be contributed to. If you can run 3-bit please click the model page and contribute.
The reason it won't be there is literally i couldn't find a public source saying it can. And I'm trying to crowdsource the data to add missing info like this, so any help is much appreciated
I like this. E.g. I can see that a local model I can run is similar to Sonnet 4.6 for example, this comparison actually helps me way more than just seeing an some score.
should do. The best approximate will be from putting in your total VRAM amount after selecting graphics card.
If you have any ideas, different commands or workflows you usually need to run for dual, please contribute with a github issue (links are everywhere on the site)
Well can you even do anything useful with 2 x RTX 3090 and 128 Gb DD4 Ram can you build software with this or basically well not even worth building what do you suggest ?
yeah you should be able to run a variant of Qwen3.8 27B I believe (e.g. with ollama, llama.cpp or lm studio). That should likely be the best model. And is similar to something like GPT-5 in performance from benchmarks
22
u/asankhs Llama 3.1 21h ago
You can fetch the data we already have from https://huggingface.co/spaces/LocalLLaMA/local-model-explorer to make it better the dataset is also open - https://huggingface.co/datasets/LocalLLaMA/local-model-explorer-data