2.8 trillion parameters is not the end: Domestic model Kimi K4 aims for extreme scale, GPUs become bottleneck
Moonshot AI
Domestic AI model startup Moonshot AI has released Kimi K4, a large language model with reportedly 2.8 trillion parameters, and plans to scale even further. The company states that GPU supply is now the primary bottleneck for training larger models.
Moonshot AI, the Chinese startup behind the Kimi K series, announced the release of its latest model, Kimi K4, claiming it has 2.8 trillion parameters. This makes it one of the largest dense models reported. The company indicated that this is not the ceiling; they plan to continue scaling the model size. However, they cited the availability of high-performance GPUs as the main limiting factor for further expansion. The existing model was trained using thousands of NVIDIA H800 GPUs, and acquiring more advanced hardware is challenging due to export restrictions and domestic supply constraints.
- Сокращения
- GPU = Graphics Processing Unit — графический процессор
Source: DeepSeek (GNews) —
original