NVIDIA’s 9-Year-Old Tesla V100 Still Packs Some Inference Muscle; 4 GPUs On An NVLink Board Can Outperform M5 Ultra, DGX Spark In Various AI Benchmarks By Up To 230%

[u/chemist_slime]

https://wccftech.com/nvidia-tesla-v100-inference-m5-ultra-dgx-spark-ai-benchmarks/

Didn't realize the V100 were still so fast, anyone have first hand experience on these?

[u/exaknight21]

If you want V100 to be 1200 a pop, just say so.

[u/thestillwind → u/exaknight21]

Already is

[u/--Spaci--]

common knowledge

[u/Trademarkd]

I started a whole sub /r/v100

[u/chemist_slime → u/Trademarkd]

Just joined, thanks for letting us know.

[u/sleight42]

So let's see:

3090s go for about $1500 each

I could sell my 2 for $3000

I could buy 4 V100 for that though that's with 150 or more of sales tax!

Another 800 for the quad board

Then cooling and another psu.

Plug this into my antiquated z490 16x slot

... and then leap from 48gb vram to 128gb madness?

What am I missing here? I know it's Volta and doesn't support CUDA 13. How f'd will Volta users be in the next year or two?

[u/rpkarma → u/sleight42]

  How f'd will Volta users be in the next year or two?

With frontier models changing our ability to back port things… less fucked than before 

[u/sleight42 → u/rpkarma]

That's been my gut feeling too. Really really has me reconsidering selling my 3090s for this janky awesome approach. Granted, 500W undervolted compared to.... a lot more.

[u/jaegernut → u/rpkarma]

Easy, just ask AI to backport things

[u/rpkarma → u/jaegernut]

That’s what I said

[u/jjusko20]

Yeah I have a few they're fast

[u/IngwiePhoenix]

And now that everyone knows, those buggers are expensive as hell. x) Happy for those that picked them for cheap tho, this is quite an interesting "hack". Super scrappy, but super cool, too! =)

[u/TheThiefMaster]

Yeah they're surprisingly powerful. I have one of the 32 GiB PCIe cards and the big advantage they have is VRAM speed - 900 GiB/s HBM2 is competitive with a 4090 or 5080, while only a 5090 can match the amount (32 GiB). They're no slouch at compute either, though that's less important for inference than the sheer amount and speed of their VRAM.

[u/watcholic]

M5U at 128k ctx running oMLX: 4500 pp tok/s, 129 tg tok/s

https://omlx.ai/benchmarks/performance?chip=M5&context=131072&device=%5B%22M5%22%2C%22Ultra%22%5D&hide_specprefill=1&model=qwen3.8-flash-Next&order=desc&quantization=4bit&sort=tg_tps

[u/Pyrolistical]

The m5 ultra numbers are not right. There is so sort of bug or misconfiguration. The memory bw should allow well over 100 tg/s

[u/power97992 → u/Pyrolistical]

 They need to use splash or mlx d flash +mtp, it should get  around 220-250tk/s , perhaps even 300 t/s if u use dflash+mtp. The m5 max can already 125 tk/s with qwen 3.8 next using mtp. Even with base mlx with 60% bandwidth utilization , u should get 200-220 tk/s since flash only has int4 6b active parameters, but the architecture is still new which  could decrease the utilization .