NVIDIA Groq 3: SRAM, Disaggregation, and Determinism in the New AI Platform

NVIDIA has completed the largest deal in its history, acquiring Groq for $20 billion, and integrated architecturally alien LPU AI accelerators into its Vera Rubin platform to sharply accelerate inference, making it disaggregated and the platform heterogeneous. The new rack-mounted NVIDIA Groq 3 LPX accelerator, consuming 160 kW, contains 256 Groq 3 (LP30) AI chips, each with 96 billion transistors, combined into 32 1U nodes with liquid cooling and a cable-free MGX design. The Groq 3 LPU chip is built on 500 MB of SRAM memory with 150 TB/s bandwidth, without caches or hierarchy, allowing explicit data movement under compiler control. The defining characteristic of the LPU is determinism—no runtime variability due to transferring all decisions to the compiler and using a pleisochronous C2C protocol. The key innovation is Attentive Front-end Decoupling (AFD), which separates the attention mechanism, executed on the GPU, from FFN/MoE operations processed on the LPX. This allows the GPU to absorb growing context volumes while the LPU workload remains constant and independent of context length. For practical use of heterogeneous decoding, NVIDIA Dynamo provides request orchestration, workload distribution, and maintains stable latency under heavy traffic. In combination, Vera Rubin NVL72 with Groq 3 LPX delivers 35x higher throughput compared to Grace Blackwell NVL72 at 400 TPS per user, and up to 10x higher revenue per MW for latency-sensitive workloads.