How Kimi K3 Works: 2.8 Trillion Parameters, Linear Attention, and Million-Token Agents
Moonshot AI has released the weights of its Kimi K3 model, a multimodal model based on a Mixture of Experts architecture with 2.8 trillion parameters, of which 104 billion are activated per token. In certain programming and tool-use benchmarks, the model outperforms competitors, but on average it lags behind some of them.
Moonshot AI
Anthropic
OpenAI

