NVIDIA NeMo AutoModel Accelerates MoE Fine-Tuning 3.7x with Expert Parallelism and DeepEP

NVIDIA has introduced NeMo AutoModel, an open library that builds on Hugging Face Transformers v5, delivering 3.4-3.7x higher training throughput and 29-32% less GPU memory for MoE models via Expert Parallelism, DeepEP fused all-to-all dispatch, and TransformerEngine kernels—using the same from_pretrained() API with only an import change.

NVIDIA Hugging Face Moonshot AI DeepSeek

From DeepSeek to K3: How Chinese Large Language Models Enter the Global Top League

Chinese large language models (LLMs) are rapidly advancing, with models like DeepSeek and K3 from Moonshot AI reaching the level of global leaders. The article analyzes how Chinese companies are closing the gap with Western competitors.

Moonshot AI DeepSeek