Nunchaku Lite Brings 4-bit Diffusion Inference to Diffusers

Nunchaku Lite is a new integration path in Hugging Face Diffusers that brings 4-bit inference for diffusion models, based on the SVDQuant quantization method behind the popular Nunchaku inference engine. Unlike standard weight-only quantization, SVDQuant runs main transformer layers with 4-bit weights and activations (W4A4), reducing memory and speeding up the denoising loop. With Nunchaku Lite, loading a quantized checkpoint is as simple as calling from_pretrained() with no local CUDA compilation required, thanks to the Hugging Face kernels package. Two kernel families are used: svdq_w4a4 for attention and MLP projections (available in INT4 and NVFP4 variants) and awq_w4a16 for normalization layers. NVFP4 checkpoints require an NVIDIA Blackwell GPU (RTX 50 series, RTX PRO 6000, B200), while INT4 variants support Turing/Ampere/Ada GPUs (RTX 30 & 40 series, A100, L40S). Benchmarks on an RTX PRO 6000 show a 1.35x speedup and up to 50% VRAM reduction compared to BF16 baseline; combining with torch.compile yields a 1.8x speedup. The companion diffuse-compressor toolkit allows users to quantize their own models and publish them as regular Diffusers repositories.