DiffusionGemma: Speeding Up Text Generation by Up to 4 Times

Google DeepMind has announced DiffusionGemma, an experimental open-source model (Apache 2.0 license) that uses text diffusion for generation. Unlike traditional autoregressive LLMs, which process tokens sequentially, DiffusionGemma generates blocks of text simultaneously, achieving up to 4x speedup on GPUs (over 1000 tokens per second on a single NVIDIA H100, over 700 tokens per second on an NVIDIA GeForce RTX 5090). The model features a Mixture of Experts (MoE) architecture with 26 billion parameters, but activates only 3.8 billion during inference, allowing it to fit into 18 GB of video memory with quantization. Key features include bidirectional attention (256 tokens per pass), iterative self-correction, and a focus on nonlinear tasks such as code editing, infilling, or working with amino acid sequences. However, output quality is lower than that of the standard Gemma 4; developers requiring maximum accuracy are advised to use the autoregressive version. DiffusionGemma is particularly effective for local inference with low concurrency; in the cloud with high throughput, the advantage diminishes. The model is already available for download on Hugging Face, with support in MLX, vLLM, and Hugging Face Transformers; integration with llama.cpp is planned. DeepMind collaborated with NVIDIA to optimize for consumer and enterprise GPUs (GeForce RTX 5090/4090, Hopper, Blackwell, NVIDIA DGX Spark, DGX Station, RTX PRO); the 4-bit NVFP4 format is supported for acceleration. Additionally, a developer guide, visual aid, and fine-tuning tutorial using Hackable Diffusion (JAX) have been published.