From DeepSeek V3 to V3.2: Architecture, Sparse Attention, and RL Updates

DeepSeek released V3.2, an open-weight flagship model with performance comparable to GPT-5 and Gemini 3.0 Pro. The model introduces DeepSeek Sparse Attention (DSA) for efficiency, and evolves from a dedicated reasoning model (R1) to a hybrid reasoning model (V3.1, V3.2). The timeline includes V3, R1, R1-0528, V3.1, V3.2-Exp, and V3.2, with architectural highlights like Multi-Head Latent Attention (MLA) and RLVR training.

DeepSeek V3.2, released over a US holiday weekend, matches GPT-5 and Gemini 3.0 Pro performance while being open-weight. It builds on V3.1, which added hybrid reasoning (instruct and reasoning modes) to the V3 base. The key new feature is DeepSeek Sparse Attention (DSA), a learned sparse attention mechanism with a lightning indexer and token selector that improves efficiency, especially for long contexts. DSA replaces sliding-window attention by dynamically selecting which past tokens to attend to. V3.2-Exp (September 2025) was an experimental release to prepare infrastructure for V3.2. Earlier models: V3 (December 2024) used MoE and MLA; R1 (January 2025) added reasoning via RLVR with GRPO; R1-0528 was a minor upgrade. The company reportedly switched from NVIDIA to Huawei chips and back to NVIDIA. The V3.2 release aims for a versatile hybrid model, while a dedicated R2 may still be in development.

Сокращения: MLA = Multi-Head Latent Attention — Многоголовое латентное внимание; MoE = Mixture of Experts — Смесь экспертов; RLVR = Reinforcement Learning with Verifiable Rewards — Обучение с подкреплением с проверяемыми вознаграждениями; GRPO = Group Relative Policy Optimization — Групповая относительная оптимизация политики; PPO = Proximal Policy Optimization — Проксимальная оптимизация политики; RLHF = Reinforcement Learning with Human Feedback — Обучение с подкреплением на основе человеческой обратной связи; DSA = DeepSeek Sparse Attention — Разреженное внимание DeepSeek

Source: Sebastian Raschka — original