Moonshot AI Releases Kimi K3： A 2.8 Trillion Parameter Open MoE Model With Kimi Delta Attention and 1M Context

Moonshot AI 刚刚发布了 Kimi K3：https://www.kimi.com/blog/kimi-k3。它是一个拥有 2.8 万亿参数的模型，具备原生视觉功能和 100 万 token 的上下文窗口。Moonshot 称它为全球首个开放的 3T 级模型。

Kimi K3 是一个稀疏专家混合（MoE）模型，基于两个架构更新构建。这两个更新是 Kimi Delta Attention（KDA）和 Attention Residuals（AttnRes）。两者都改变了信息在序列长度和模型深度上的流动方式。K3 面向长远编码、知识工作和推理任务。

Moonshot 团队表示，K3 是首个达到 2.8 万亿参数的开放模型。在过去的十二个月中，有九个月 Kimi 模型刷新了开放模型规模的上限。

Moonshot 也直接说明了 K3 的定位。总体性能仍然落后于最强大的专有模型 Claude Fable 5 和 GPT 5.6 Sol。在 Moonshot 自己的评测套件中，K3 一直优于其他测试模型。

Kimi Delta Attention（KDA）是一种混合线性注意机制。Moonshot 表示，它能够在百万 token 上下文中实现最多 6.3 倍的解码加速。

AttnRes 作用于另一个轴，即深度。它选择性地跨深度提取表示，而不是均匀累积表示。Moonshot 表示，AttnRes 在额外成本不到 2% 的情况下，可提供约 25% 的训练效率提升。

稀疏性是第三个杠杆。K3 使用 Stable LatentMoE，有效激活 896 个专家中的 16 个。在这种稀疏度下，路由和优化成为一阶挑战。Quantile Balancing 直接从路由器得分分位数得出专家分配，这消除了启发式更新和敏感的平衡超参数。Per-Head Muon 通过独立优化注意头扩展了 Muon。Sigmoid Tanh Unit（SiTU）和 Gated MLA 分别改进了激活控制和注意选择性。

这些结构性变化伴随着精炼的训练和数据方法。综合起来，它们比 Kimi K2 的整体扩展效率大约提高了 2.5 倍。

这些选择会延续到部署阶段。K3从SFT阶段开始应用量化感知训练。它使用MXFP4权重和MXFP8激活，以实现广泛的硬件兼容性。Moonshot团队推荐拥有64个或更多加速器的超级节点配置。由于KDA在前缀缓存方面带来了新挑战，Moonshot向vLLM贡献了一个实现。

在机制建立后，已公布的分数更易于解读。所有K3结果都将推理强度设置为最大值。不同基准测试使用的工具不同：KimiCode、Claude Code或Codex。

这个表格有两个注意事项。'With fallback'表示Fable 5在其使用策略下拒绝的请求会被路由到Opus 4.8。此外，BrowseComp在触发300K令牌时使用了上下文压缩。没有该上下文管理，K3得分为90.4。

因此，K3在Program Bench、SWE Marathon、BrowseComp、Automation Bench和OmniDocBench中领先。在FrontierSWE和HLE-Full中落后于Fable 5，在DeepSWE中落后于GPT 5.6 Sol。

Moonshot团队表示，一个本地多模态架构可以同时处理文本、图像和视频。

K3已在Kimi.com、Kimi Work、Kimi Code和API上上线。访问通过OpenAI SDK并连接Moonshot基础URL进行。

四条规则很重要。reasoning_effort仅支持max，且不能使用K2.x的thinking参数。temperature、top_p和n固定，因此可忽略它们。max_completion_tokens默认值为131072，可达1048576。在多轮对话和工具调用中，返回完整的助手消息。

定价为固定制，不按上下文长度分级。缓存命中输入为$0.30/千令牌，缓存未命中为$3.00/千令牌，输出为$15.00/千令牌。因此，缓存命中率是需要关注的指标。Moonshot团队报告编码工作负载中缓存命中率超过90%。

需要与我们合作以推广您的GitHub仓库、Hugging Face页面、产品发布或网络研讨会等吗？请联系：https://forms.gle/wbash1wF6efRj8G58

Asif Razzaq 是 Marktechpost Media Inc. 的首席执行官。作为一个有远见的企业家和工程师，Asif 致力于利用人工智能的潜力为社会带来积极影响。他最近的努力是推出人工智能媒体平台 Marktechpost，该平台以对机器学习和深度学习新闻的深入报道而著称，这些报道既技术上可靠，又易于广大受众理解。该平台每月浏览量超过 200 万次，显示了其在观众中的受欢迎程度。

构建一个自主事件场地运营商 [完整代码]：https://pxllnk.co/twdn5

谢谢！我们的团队会尽快与您联系

Moonshot AI just released Kimi K3：https://www.kimi.com/blog/kimi-k3 . It is a 2.8-trillion-parameter model with native vision and a 1-million-token context window. Moonshot calls it the world’s first open 3T-class model.

Kimi K3 is a sparse Mixture-of-Experts (MoE) model built on two architectural updates. Those are Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). Both change how information flows across sequence length and model depth. K3 targets long-horizon coding, knowledge work, and reasoning.

Moonshot team states K3 is the first open model to reach 2.8 trillion parameters. For nine of the past twelve months, Kimi models set the upper bound of open-model sizes.

Moonshot is also direct about where K3 sits. Overall performance still trails the most powerful proprietary models, Claude Fable 5 and GPT 5.6 Sol. Across Moonshot’s own evaluation suite, K3 consistently outperformed other tested models.

Kimi Delta Attention (KDA) is a hybrid linear attention mechanism. Moonshot states it enables up to 6.3x faster decoding in million-token contexts.

AttnRes works along the other axis, which is depth. It selectively retrieves representations across depth rather than accumulating them uniformly. Moonshot states AttnRes delivers roughly 25% higher training efficiency at under 2% additional cost.

Sparsity is the third lever. K3 uses Stable LatentMoE, effectively activating 16 of 896 experts. At that sparsity, routing and optimization become first-order challenges. Quantile Balancing derives expert allocation directly from router-score quantiles. That eliminates heuristic updates and a sensitive balancing hyperparameter. Per-Head Muon extends Muon by optimizing attention heads independently. Sigmoid Tanh Unit (SiTU) and Gated MLA improve activation control and attention selectivity respectively.

Refined training and data recipes accompany those structural changes. Together they yield roughly 2.5x better overall scaling efficiency than Kimi K2.

Those choices carry into serving. K3 applies quantization-aware training from the SFT stage onward. It uses MXFP4 weights with MXFP8 activations for broad hardware compatibility. Moonshot team recommends supernode configurations with 64 or more accelerators. Because KDA poses new challenges for prefix caching, Moonshot contributed an implementation to vLLM.

With the mechanics established, the published scores are easier to read. All K3 results use reasoning effort set to max. Harnesses differ per benchmark: KimiCode, Claude Code, or Codex.

Two caveats shape this table. 'With fallback' means requests Fable 5 refuses under its usage policy route to Opus 4.8. Also, BrowseComp used context compaction triggered at 300K tokens. Without that context management, K3 scores 90.4.

So K3 leads Program Bench, SWE Marathon, BrowseComp, Automation Bench, and OmniDocBench. It trails Fable 5 on FrontierSWE and HLE-Full, and GPT 5.6 Sol on DeepSWE.

Moonshot team states one native multimodal architecture handles text, images, and video together.

K3 is live on Kimi.com, Kimi Work, Kimi Code, and the API. Access runs through the OpenAI SDK against a Moonshot base URL.

Four rules matter. reasoning_effort supports only max , and the K2.x thinking parameter must not be used. temperature , top_p , and n are fixed, so omit them. max_completion_tokens defaults to 131072 and reaches 1048576. In multi-turn and tool calls, return the complete assistant message.

Pricing is flat, with no tiering by context length. Cache-hit input is $0.30/MTok, cache-miss is $3.00/MTok, and output is $15.00/MTok. The cache-hit rate is therefore the number to watch. Moonshot team reports above 90% cache hits in coding workloads.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us ：https://forms.gle/wbash1wF6efRj8G58

Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.

Build an Agentic Event Venue Operator [Full Codes]：https://pxllnk.co/twdn5

Thanks! Our team will contact you soon