Kimi K3 发布：2.8T 参数开源模型，具备原生视觉与百万 token 上下文窗口

今天，我们推出 Kimi K3——我们最强大的模型。Kimi K3 是一个拥有 2.8 万亿参数的模型，基于我们的 Kimi Delta Attention 和 Attention Residuals 架构，具备原生视觉能力和 100 万令牌上下文窗口。它是全球首个开放的 3T 级别模型，旨在为长程编程、知识工作和推理提供前沿智能支持。

虽然其整体性能仍落后于最强大的专有模型，如 Claude Fable 5 和 GPT 5.6 Sol，但 Kimi K3 在我们的评估套件中展示了前沿水平的表现，并持续优于其他测试模型。

Kimi K3 今日已在 Kimi.com（https://www.kimi.com/）、Kimi Work（https://www.kimi.com/products/kimi-work）、Kimi Code（https://www.kimi.com/code）以及 Kimi API（https://platform.kimi.ai/）上线。发布时，Kimi K3 默认使用最大计算努力，低/高计算努力模式将在后续更新中引入。我们目前正与推理合作伙伴和开源维护者紧密合作，以对齐技术细节并确保生态系统内的可靠部署。完整模型权重将于 2026 年 7 月 27 日发布。关于架构、训练及评估的更多细节将随 Kimi K3 技术报告一并发布。

Kimi K3 是首个达到 2.8 万亿参数的开放模型。这标志着 Kimi 在规模化前沿持续推进的最新一步：在过去 12 个月内的 9 个月里，Kimi 模型设定了开放模型规模的上限。

Kimi K3 构建于 Kimi Delta Attention（KDA）和 Attention Residuals（AttnRes）之上，这两项架构更新旨在改善信息在序列长度和模型深度之间的流动。我们还扩大了专家混合（MoE）的稀疏性，当与稳定的 LatentMoE 框架配合使用时，实际激活 896 个专家中的 16 个。结合改进的训练和数据方案，这些结构性变化带来了整体规模化效率约 2.5 倍的提升，相比 Kimi K2，更有效地将计算转化为智能。

Kimi K3 具有强大的长程编程能力。在最少人工监督下，它能够维持长时间的工程任务，浏览大型代码库，并协调终端工具的使用。

Kimi K3 在融合软件工程与视觉推理的任务中也表现出色——它利用截图和视觉信息来优化游戏开发、前端开发和 CAD。

下面的案例研究展示了 Kimi K3 的编码能力如何转化为开放式的软件创作和科学研究。

我们测试了各模型优化 GPU 内核的能力。每个模型在相同的沙箱环境中独立工作，最多有 24 小时来分析、重写并基准测试四个任务，涵盖 AttnRes、KDA 和一个 512 头维度的 MLA 内核，运行于 NVIDIA H200 及其他供应商的 GPGPU 上。Kimi K3 的表现与 Fable 5（带回退）相当，并显著优于 Opus 4.8、GPT 5.6 Sol 和 GPT 5.5。

Claude Fable 5 经第三方评估，其结果可能包括回退行为。在大多数模型中，一些轨迹包含小幅、可接受的精度取舍，这仍在我们的数值容差范围内。GPGPU 指用于图形渲染之外计算的一般用途 GPU。

在 Kimi K3 开发的后期阶段，Kimi K3 的早期版本处理了团队大部分内核优化工作。

我们进一步测试了 Kimi K3 是否能够从零构建 GPU 编程系统。Kimi K3 开发了 MiniTriton，这是一个紧凑的类似 Triton 的编译器，在 MLIR 上建立了自己的 tile 级 IR 层、优化流水线和 PTX 代码生成管线。在支持的 roofline 基准测试中，MiniTriton 的性能与 Triton 和 torch.compile 相当甚至更优——在某些工作负载上超过了 Triton。除了微基准测试之外，MiniTriton 能够持续进行端到端的 nanoGPT 训练，并保持稳定收敛，损失曲线与参考曲线高度一致，仅有轻微偏差——验证了真实工作负载下完整流水线的有效性。这些结果表明，Kimi K3 能够构建一个连贯的端到端编译器——从 DSL 前端和 IR 优化到 PTX 代码生成和运行时——而非仅处理孤立内核；其从零实现的 Tensor Core 路径已经可与 Triton 的高度优化栈媲美。

Kimi K3结合了强大的三维推理、编码和视觉能力，将概念、图像和视频转化为完全可玩的互动体验。Kimi K3通过在代码和实时截图之间无缝迭代，实现了真正的“循环视觉”—即时查看并优化输出。

Kimi K3使用Three.js WebGPU和GPU计算构建了一个完全程序化的基于浏览器的3D探索游戏。它程序化生成环境，同时使用三维资产生成工具创建骑手和马的模型，生成了一个广阔的开放世界，包括森林、木屋村庄、雪山和动态天气。使用的外部资产：动画牛仔和马模型及地形数据。

作为早期概念验证，Kimi K3设计了一块芯片，以服务于其自身架构构建的纳米模型。在一次48小时的自主运行中，K3使用开源EDA工具在Nangate 45nm库上构建、优化并验证了芯片。在4 mm²的面积内，该芯片在100 MHz时闭合时序，并在仿真中维持超过8,700 tokens/s的解码吞吐量，封装了1.46M个标准单元、0.277 MB SRAM和带融合去量化的INT4 MAC阵列。一块由模型为模型构建的芯片，体现了K3的长远自主代理能力。

Kimi K3桥接了科学文献与可执行代码，能够自主实现、验证并分析复杂的计算研究工作流程。

在一个案例中，Kimi K3用了大约两小时完成了通常需要经验丰富的研究人员一到两周才能完成的工作。为了重现计算天体物理中的I–Love–Q普适关系，它审阅并交叉验证了20篇论文，实施了完整的数值管线，评估了300种状态方程，识别出已发表公式中的不一致性，生成了3,000行Python代码，并制作了一个用于探索结果的互动HTML仪表板。

Kimi K3推进了端到端的知识工作。除了公共基准之外，Kimi K3 (max)在我们的内部评估中也展示了持续的性能提升，这些评估源自真实用户代理工作流程中观察到的反复模式和挑战。在不同面向生产的工作流程中，这些持续优势反映了Kimi K3自主知识工作能力的广泛提升。

以下是 Kimi Work 中的 Kimi K3 在金融咨询和科学研究方面可以生成的一些示例：

您可以深入钻研的互动研究报告：42 年的 ASIC 行业数据，通过 120 轮递归自我优化生成。Kimi K3 将证据转化为定制图表、动画图解和互动可视化叙事。它通过 2800 次网页搜索/抓取和 1100 次终端数据获取，从涵盖 87 个季度报告和 99 个原始 PDF 的 1.1 万页数据中提取信息。

带有互动可视化的咨询风格行业报告——包括时间线、漏斗图、范围条形图、甘特图以及出版级幻灯片。

使用 20 个并行子代理分析 391 个引力波事件，生成 7 个科学可视化图、2 个表格，以及来自 10 篇论文的文献综述。

Kimi K3 在生成信息图表式演示方面也特别有效，例如下面显示的完全可编辑的热图和年度报告：

在 Kimi Work 中，我们推出了两个新功能——小组件（Widgets）和仪表板（Dashboard），让与 Kimi K3 的交互更加可视化和持久化。小组件允许您直接在聊天中生成交互组件，并与本地数据或外部插件连接以实现持续更新。仪表板将您最关心的小组件整合到一个持久的、个性化视图中，围绕特定主题、项目或目标进行组织。

Kimi K3 擅长动态设计、动画和视频编辑，因为其原生多模态架构能够在同一模型中理解文本、图像和视频。

在一个示例中，K3 创建了一个类似 3Blue1Brown 风格的动态图形解说，讲解其自身架构，将技术概念转化为动画图解和过渡效果。

在另一个示例中，Kimi K3 从 56 个源片段中编辑了自己的预告视频，处理片段选择、动作匹配剪辑、帧精确节拍同步、音频处理以及多轮修订。像这样的高密度短视频通常需要一位经验丰富的编辑一到两个工作日，或者初学者三到五天。

Kimi K3 建立在 Kimi Delta Attention (KDA) 和 Attention Residuals (AttnRes) 的基础上。KDA 为扩展注意力提供了高效的基础，而 AttnRes 则选择性地跨深度提取表示，而不是均匀地累积它们。两者结合，构成了一个设计用于远超一万亿参数规模的模型的架构骨干。

Kimi K3 使用 Stable LatentMoE，有效激活 896 个专家中的 16 个。在这个稀疏水平下，路由和优化成为一阶挑战。Quantile Balancing 直接从路由评分分位数派生专家分配，消除启发式更新和敏感的平衡超参数，而 Per-Head Muon 通过独立优化注意力头扩展了 Muon，以实现大规模下更自适应的学习。Sigmoid Tanh 单元 (SiTU) 和 Gated MLA 分别改进了激活控制和注意力选择性。综合这些进展，使得在 2.8 万亿参数规模下仍能稳定高效地训练。

Kimi K3 从 SFT 阶段起应用量化感知训练，使用 MXFP4 权重和 MXFP8 激活，实现广泛的硬件兼容性。为了防止专家不平衡在大规模专家并行时降低吞吐量，我们引入了全平衡专家并行训练方法，具备静态形状且关键路径上无主机同步。由于推理效率同样受益于更大的高带宽通信域，我们建议在拥有 64 个或以上加速器的超级节点配置上部署 Kimi K3。最后，由于 KDA 对传统前缀缓存提出了新挑战，我们已为 vLLM 社区贡献了相应实现，将随模型一起发布。带有预填充缓存的 KDA 使我们能够在其规模和长上下文的情况下，以极具竞争力的 token 成本提供 Kimi K3 服务。

更多技术细节将在我们即将发布的报告中提供。

以下报告的所有 Kimi K3 结果均在推理努力设置为 'max'、温度 = 1.0 和 top-p = 1.0 的情况下获得。根据基准测试的不同，每个模型在三种智能代理测试环境之一进行评估——KimiCode、Claude Code 或 Codex，具体见下方注释说明。

生产力和智能代理基准

Today, we are introducing Kimi K3 — our most capable model. Kimi K3 is a 2.8T-parameter model built on our Kimi Delta Attention and Attention Residuals, with native vision capabilities and a 1-million-token context window. It is the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning.

While its overall performance still trails the most powerful proprietary models, Claude Fable 5 and GPT 5.6 Sol, Kimi K3 demonstrated frontier-level performance across our evaluation suite, consistently outperforming other tested models.

Kimi K3 is available today on Kimi.com：https://www.kimi.com/, Kimi Work：https://www.kimi.com/products/kimi-work, Kimi Code：https://www.kimi.com/code, and the Kimi API：https://platform.kimi.ai/. At launch, Kimi K3 will use max thinking effort by default, with low- and high-effort modes to be introduced in subsequent updates. We are currently working closely with inference partners and open-source maintainers to align technical details and ensure a reliable rollout across the ecosystem. The full model weights will be released by July 27, 2026. Further details on the architecture, training, and evaluations will be released alongside the Kimi K3 technical report.

Kimi K3 is the first open model to reach 2.8 trillion parameters. It marks the latest step in Kimi's sustained push at the scaling frontier: for nine of the past twelve months, Kimi models have set the upper bound of open-model sizes.

Kimi K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), two architectural updates designed to improve how information flows across sequence length and model depth. We have also scaled up Mixture of Experts (MoE) sparsity, effectively activating 16 out of 896 experts when paired with a Stable LatentMoE framework. Together with refined training and data recipes, these structural changes yield an approximate 2.5× improvement in overall scaling efficiency compared to Kimi K2, allowing the model to convert compute into intelligence more effectively.

Kimi K3 has strong long-horizon coding performance. Operating with minimal human oversight, it can sustain long engineering sessions, navigate massive repositories, and orchestrate terminal tools.

Kimi K3 also excels in tasks blending software engineering with visual reasoning — it leverages screenshots and visuals to optimize game dev, frontend, and CAD.

The case studies below show how Kimi K3's coding capability translates into open-ended software creation and scientific research.

We tested the models' capability to optimize GPU kernels. Each model works independently in an identical sandbox, with up to 24 hours to profile, rewrite, and benchmark four tasks spanning AttnRes, KDA, and a 512-head-dimension MLA kernel across NVIDIA H200 and GPGPU from an alternative vendor. Kimi K3 performed competitively with Fable 5 (with fallback) and substantially outperformed Opus 4.8, GPT 5.6 Sol, and GPT 5.5.

Claude Fable 5 was evaluated by a third party, and its results may include fallback behavior. Across most models, some trajectories include small, acceptable precision shortcuts that remain within our numerical tolerance. GPGPU denotes general-purpose GPUs used for computation beyond graphics rendering.

In the late stages of Kimi K3 development, an early version of Kimi K3 handled the majority of the team's kernel optimization works.

We further tested whether Kimi K3 could build a GPU programming system from scratch. Kimi K3 developed MiniTriton, a compact Triton-like compiler with its own tile-level IR layer over MLIR, optimization passes, and a PTX code-generation pipeline. Across supported roofline benchmarks, MiniTriton delivers performance on par with or better than Triton and torch.compile — beating Triton on certain workloads. Beyond microbenchmarks, MiniTriton sustains end-to-end nanoGPT training with stable convergence, the loss curve closely tracking the reference with only minor divergence — validating the full pipeline on a realistic workload. These results demonstrate that Kimi K3 can build a coherent end-to-end compiler — from DSL frontend and IR passes to PTX codegen and runtime — rather than isolated kernels; its from-scratch Tensor Core path already rivals Triton’s extensively optimized stack.

Kimi K3 combines strong 3D reasoning, coding, and vision capabilities to turn concepts, images, and videos into fully playable interactive experiences. Kimi K3 achieves true "vision in the loop" by seamlessly iterating between code and live screenshots—instantly seeing and refining outputs.

Kimi K3 built a fully procedural browser-based 3D exploration game using Three.js WebGPU and GPU compute. It procedurally generated the environment, while using a 3D asset generation tool to create the rider and horse models, producing an expansive open world with forests, a log-cabin village, snowy mountains, and dynamic weather. External assets used: animated cowboy and horse models and terrain data.

As an early proof of concept, Kimi K3 designed a chip to serve a nano model built on its own architecture. In a single 48-hour autonomous run, K3 built, optimized, and verified the chip using open-source EDA tools on the Nangate 45nm library. Within 4 mm², the chip closes timing at 100 MHz and sustains over 8,700 tokens/s decode throughput in simulation, packing 1.46M standard cells, 0.277 MB of SRAM, and an INT4 MAC array with fused dequantization. A chip built by a model, for a model, reflects K3's long-horizon agentic capabilities.

Kimi K3 bridges scientific literature and executable code, autonomously implementing, validating, and analyzing complex computational research workflows.

In one case, Kimi K3 completed in about two hours what would typically require one to two weeks of work by an experienced researcher. To reproduce the I–Love–Q universal relations in computational astrophysics, it reviewed and cross-validated 20+ papers, implemented the full numerical pipeline, evaluated 300+ equations of state, identified inconsistencies in published formulas, generated 3,000+ lines of Python code, and produced an interactive HTML dashboard for exploring the results.

Kimi K3 advances end-to-end knowledge work. Beyond public benchmarks, Kimi K3 (max) demonstrates consistent gains across our internal evaluations, which are derived from recurring patterns and challenges observed in real-world user-agent workflows. These consistent advantages across distinct production-oriented workflows reflect a broad improvement in Kimi K3's agentic knowledge work capabilities.

Below are a few examples of what Kimi K3 in Kimi Work can produce across financial consulting and scientific research:

An interactive research report you can drill into: 42 years of the ASIC industry, created through 120+ rounds of recursive self-improvement. Kimi K3 transforms evidence into bespoke charts, animated diagrams, and interactive visual narratives. It pulled data via 2.8k+ web searches/fetches and 1.1k+ terminal data pulls, across 11k+ pages spanning 87 quarterly reports and 99 original PDFs.

A consulting-style industry report with interactive visualizations—including timelines, Funnel Chart, Range Bar Chart, Gantt Charts, and publication-quality slides.

An analysis of 391 gravitational-wave events using 20+ concurrent subagents, producing 7 scientific visualizations, 2 tables, and a literature synthesis from 10+ papers.

Kimi K3 is also particularly effective at producing infographic-style presentations, such as the fully editable heatmap and annual report shown below:

In Kimi Work, we introduce two new features - Widgets and Dashboard - which make interactions with Kimi K3 more visual and persistent. Widgets let you generate interactive components directly within a chat, with connections to local data or external plugins for continuous updates. Dashboard brings the widgets you care about most into one persistent, personalized view organized around a topic, project, or goal.

Kimi K3 excels at motion design, animation, and video editing because its native multimodal architecture understands text, images, and video within the same model.

In one example, K3 created a 3Blue1Brown-style motion-graphics explainer of its own architecture, translating technical ideas into animated diagrams and transitions.

In another, Kimi K3 edited its own teaser video from 56 source clips, handling clip selection, motion-matched cuts, frame-accurate beat synchronization, audio processing, and multiple rounds of revision. A high-density short video like this would typically take an experienced editor one to two working days, or a beginner three to five.

Kimi K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). KDA provides an efficient foundation for scaling attention, while AttnRes selectively retrieves representations across depth rather than accumulating them uniformly. Together, they form the architectural backbone of a model designed to scale well beyond the trillion-parameter regime.

Kimi K3 uses Stable LatentMoE, effectively activating 16 of 896 experts. At this level of sparsity, routing and optimization become first-order challenges. Quantile Balancing derives expert allocation directly from router-score quantiles, eliminating heuristic updates and a sensitive balancing hyperparameter, while Per-Head Muon extends Muon by optimizing attention heads independently for more adaptive learning at scale. Sigmoid Tanh Unit (SiTU) and Gated MLA improve activation control and attention selectivity respectively. Together, these advances enable stable and efficient training at the 2.8-trillion-parameter scale.

Kimi K3 applies quantization-aware training from the SFT stage onward, using MXFP4 weights with MXFP8 activations for broad hardware compatibility. To prevent expert imbalance from degrading throughput at large expert-parallel scales, we introduce a fully balanced expert-parallel training method with static shapes and no host synchronization on the critical path. Since inference efficiency likewise benefits from larger high-bandwidth communication domains, we recommend deploying Kimi K3 on supernode configurations with 64 or more accelerators. Finally, as KDA poses new challenges for conventional prefix caching, we have contributed a corresponding implementation to the vLLM community, to be released alongside the model. KDA with prefill cache allows us to serve Kimi K3 at a highly competitive token price despite its scale and long context.

More technical details will be available in our coming report.

All Kimi K3 results reported below are obtained with the reasoning effort set to 'max', setting temperature = 1.0 and top-p = 1.0. Depending on the benchmark, each model is evaluated under one of three agentic harnesses — KimiCode, Claude Code, or Codex — as specified in the notes below.

Productivity and agentic benchmarks