Allen Institute for AI

Berita AI terbaru, model, dan rilis dari Allen Institute for AI. ['MolmoAct2', 'OLMo 2', 'OLMo 2 7B', 'OLMo 3', 'OLMo 3 7B', 'Tülu 3']

Situs resmi ↗

Model 🇺🇸

Comparison of Large Language Model Architectures: From DeepSeek V3 to OLMo 2

Sebastian Raschka published a detailed comparison of the architectures of modern open LLMs, highlighting key innovations: Multi-Head Latent Attention (MLA) and Mixture-of-Experts (MoE) in DeepSeek V3, as well as normalization features in OLMo 2. The article covers the evolution from GPT-2 to 2025 models.

DeepSeek

Allen Institute for AI

Sebastian Raschka27.07 · 18:04

Riset 🇺🇸

Beyond Standard LLMs: Linear Attention Hybrids, Text Diffusion Models, Code World Models, and Small Recurrent Transformers

The article explores alternatives to standard autoregressive transformer-based LLMs: linear attention hybrids (MiniMax-M1, Qwen3-Next, DeepSeek V3.2, Kimi Linear), text diffusion models, code world models, and small recurrent transformers. The author notes a return to classical attention in MiniMax-M2 and the complexity of linear attention in production.

Moonshot AI

MiniMax

Alibaba/Qwen

DeepSeek

Google/DeepMind

Mistral