Community Contributor
Member since 2026
Views
0 Total number of views
Fair Values Set
0 Total number of fair values set
Comments
2 Total number of comments
Followers
0 Total number of followers

No bio added yet

No link added
CBRS logo
CBRS·The Wafer Giant Threatening NVIDIA's GPU Hegemony
TL:DR -NVDA already bought the next-best tech... Groq I really can't intelligently comment on the financial analysis provided, this article's view of technological superiority is supported by technical analyses. Training prefers GPU-type architectures, but for inference nothing much can compare. These things are physically impressive- a 12 inch chip. The magic sauce really is that there are tons (900,000... yes, 0.9x10^6) CPU cores of various flavors physically on the same chip as a pile of fast SRAM memory, structurally guaranteeing memory access bandwidth that HBM cannot approach. Check out commentary and analysis over the last 3 years at nextplatform dot com. The importance of mega-core, on-chip memory chips for inference is shown by NVDA's $20B aquihire of Groq- kinda the same thing, but not as huge (only needs one hand to hold). NVDA bought the closest thing to cerebras that existed, and immediately redesigned their accelerator systems to include the Groq chips, replacing their in-house designed ARM "cpu". Within months, dumping their own high performance custom chip. So yes, its real in every sense. The tech absolutely rocks. More memory would further improve tokens/watt; they were really optimized for simpler queries, not chapter length replies. To handle bigger modern models, they chain a few together - just so they have enough fast memory to hold the model. The real technological challenge going forward is that one simply cannot just add memory to their design- there is limited space, period, and more memory would mean fewer cores. So how can it get more fast memory? Off-chip is way too slow. Likely by gluing memory on top... just like AMD does does with its CPUs.
2
|
0