r/hardware • u/SirActionhaHAA • 15h ago

News AMD debuts Helion reference design for agentic AI servers with up to 32,768 Zen 6 ‘Venice’ cores — rack design ships next year in CPU-only and all-in-one configurations

https://www.tomshardware.com/pc-components/cpus/amd-debuts-helion-reference-design-for-agentic-ai-servers-with-up-to-32-768-zen-6-venice-cores-rack-design-ships-next-year-in-cpu-only-and-all-in-one-configurations
74 Upvotes

91% Upvoted

17 comments sorted by

30

u/Ultium 15h ago

AI Enterprise runs at such a ridiculous scale that I don’t know if this is actually a competitive offering or not. What use case is this for? Inference?

20

u/Ready_Acanthaceae830 15h ago

The use case could be closer to running Muse like agents 24*7. But I can't imagine the cost for such a thing

9

u/EloquentPinguin 14h ago

I don't know how the CPU only compares but CNBC reported one Helios rack to be $5 million. But at a price of ~$60-$70 per Venice CPU core, a 32k core rack would cost ~$2 million in CPUs alone. Add motherboards, ram, storage, (especially ram and storage) and you see that the answer is "expensive"

9

u/goldcakes 14h ago

Basically a rounding error to hyperscalers. More density means you need smaller DCs.

Main use case is probably always on agents, and RL rollouts. Modern LLM training is really in post-training: tens of thousands of agents try to complete a task, get scored/graded; the ones that succeed and proceed the most efficiently have their traces/trajectory boosted. It’s kinda evolutionary in a way.

2

u/comparmentaliser 14h ago

The power costs alone would be a boon

1

u/BlackenedGem 7h ago

No way it's $70/core for cutting edge 2nm silicon sold to hyperscalers during the AI boom where margins are insane. I imagine rack vs system is being confused.

Purely looking at the image there's 32 racks (from the 44 total), and in a CPU only configuration having 4 CPUs/rack seems reasonable. That would then have it be:

  • 1 rack = 4 x 256 cores = 1024 cores
  • Full helion = 32 x 1024 cores = 32,768

If it's a $5m rack then that would be $5k/core, which is insane but probably more realistic given the AI markup and if it includes motherboard and RAM. Then a full cabinet of 32k cores would be ~160m.

2

u/Qesa 3h ago edited 2h ago

We're not buying those specific models, but at my work our quoted all-in cost (including all components and DC space/power/support for the server's lifetime) comes out to about $400 per core. And most of that is from the RAM and SSDs. You're vastly overestimating the price there.

1

u/HighestLevelRabbit 4h ago

AMD's msrp for the individual cpus are $14904 for the 256c 9996 and $10116 for the 96c 9676F. Not sure what ram capacity it would be looking at or what every other component + service contracts would come too.

I don't work in a data centre but do sometimes work with servers, in my experience when people say rack they are referring to the physical rack itself including every physical system in it. Is that not how the terminology is used elsewhere?

9

u/RealPjotr 14h ago

I read earlier this summer that early last year, the GPU market was expected to have 5-10x more importance than the CPU market. With the agentic revolution, it is now expected to become 1 to 1, making AMD the big play here. They have nearly half the enterprise CPU market by $$$ already.

8

u/ElectronicStretch277 15h ago

Agentic AI and CPU heavy workloads.

3

u/spiral6 10h ago

Right now, everything's competitive because demand is extremely high, and supply is low. Speaking from the work I'm doing on Helios right now. The customers are who you'd expect, at least whatever's been publicly shown on the press releases... Unfortunately, the demand doesn't seem to be going down anytime soon (unfortunately in the regards for us as the consumer segment and consumer hardware).

2

u/john0201 11h ago

It’s for agents that use a different server to run inference (or a different part of this rack with GPUs).

27

u/toxygen001 15h ago

I just want to be able to afford to build a new computer. :-(

10

u/FranciumGoesBoom 11h ago

have you tried not being poor?

4

u/Ruff_Ryda 2h ago

Remember the time when people rooted for AMD so that Intel could come to its senses on desktop cpu pricing.

-2

u/JunosArmpits 9h ago

ASRock Rack haha, AsRack heehe, ASSCRack hohohoo

0

u/AutoModerator 15h ago

Hello SirActionhaHAA! Please double check that this submission is original reporting and is not an unverified rumor or repost that does not rise to the standards of /r/hardware. If this link is reporting on the work of another site/source or is an unverified rumor, please delete this submission. If this warning is in error, please report this comment and we will remove it.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.