DiffusionGemma: 4x faster text generation

Google DeepMind released DiffusionGemma, an experimental 26B parameter Mixture of Experts (MoE) model under Apache 2.0 license, which achieves up to 4x faster text generation by using a diffusion-based approach instead of sequential token prediction. The model activates only 3.8B parameters per inference, fitting within 18GB VRAM when quantized, and achieves over 1000 tokens per second on NVIDIA H100 and 700+ on GeForce RTX 5090. It uses bi-directional attention to generate 256 tokens in parallel, enabling non-linear tasks like code infilling and Sudoku solving. DiffusionGemma is optimized for local low-concurrency inference, where its parallel decoding provides maximum benefit, while autoregressive models remain better for high-QPS cloud serving. The model iteratively refines its output, and can be fine-tuned using tools like Unsloth, Hugging Face, vLLM, MLX, JAX, NVIDIA NeMo, with llama.cpp support upcoming. Google DeepMind collaborated with NVIDIA for hardware optimizations including NVFP4 kernels, and the model is available on Hugging Face, Gemini Enterprise Agent Platform and NVIDIA NIM.