Nunchaku Lite: 4-bit Inference of Diffusion Models in Diffusers

On the Hugging Face blog, the integration of the Nunchaku quantization method (based on SVDQuant) into the Diffusers library has been announced. SVDQuant works with 4-bit weights and activations (W4A4) and handles outliers by moving them into the weights, leaving a small 16-bit low-rank branch. The original Nunchaku engine required separate integrations for each architecture. Nunchaku Lite eliminates this problem by replacing the nn.Linear modules of the diffusion model with special linear layers using CUDA kernels (svdq_w4a4 for main computations and awq_w4a16 for normalization layers). The checkpoint is loaded with a standard call to from_pretrained(), and the quantization configuration is stored in the model's config.json. Nunchaku Lite provides a speed improvement of approximately 30% and a VRAM reduction of up to 50% compared to BF16. With torch.compile, the speedup reaches 1.8x. The method supports Blackwell (NVFP4) and Turing/Ampere/Ada (INT4) GPUs. For quantizing custom models, the diffuse-compressor toolkit is offered, which automatically determines quantization targets and packages the result in Diffusers format.