然而，专家对蒸馏——通过查询 LLM 来确定其内部工作原理并复制其能力的过程——能否让 Kimi K3 展现出先进能力表示怀疑。
“我认为在 Fable 严格进行蒸馏之后，你不可能这么快得到如此强大的模型，”Laude Institute 的研究员、Snorkel AI 联合创始人布拉登·汉考克告诉 TechCrunch。“坦白说，根本没有时间，对吧？Fable 公开发布仅从7月1日开始。你不可能在两周内完成如此多的数据蒸馏、模型训练并发布。”
“我一直认为，随着中国模型越来越接近前沿，训练体系转向[强化学习]，蒸馏的影响正变得越来越小，”Allen Institute for AI 的 AI 研究员内森·兰伯特在昨天发布的播客中说道：https://www.interconnects.ai/p/open-models-recap-more-on-kimi-k3。“如果真是这样，每个人都可以轻松通过蒸馏使用其数据来追赶 GLM 或 K3。但我们没有，也不会仅通过监督微调看到这种情况。”
特斯拉的机器人出租车正在倒退行驶：https://techcrunch.com/2026/07/23/teslas-robotaxis-are-moving-in-reverse/ Sean O'Kane：https://techcrunch.com/author/sean-okane/
Jack Dorsey 正在通过 Buzz 挑战 Slack，这是一款为团队及其 AI 代理设计的群聊平台：https://techcrunch.com/2026/07/21/jack-dorsey-is-taking-on-slack-with-buzz-a-group-chat-platform-for-teams-and-their-ai-agents/ Amanda Silberling：https://techcrunch.com/author/amanda-silberling/
据 Have I Been Pwned，AI 音乐生成器 Suno 的数据泄露影响了 5500 万用户：https://techcrunch.com/2026/07/21/ai-music-generator-suno-breach-affects-55m-users-per-have-i-been-pwned/ Zack Whittaker：https://techcrunch.com/author/zack-whittaker/
White House science advisor Michael Kratsios said that Moonshot, the Chinese company behind Kimi K3, the largest available open-weight LLM, built its model by copying Anthropic’s Fable LLM while using chips that aren’t cleared for export to China.
“Large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable,” Kratsios wrote：https://x.com/mkratsios47/status/2079933645888880708, amid reported discussions about banning Chinese open-weight models：https://techcrunch.com/2026/07/20/openai-is-scared-of-open-weight-models-should-the-us-be/ that have roiled the AI sector. Moonshot did not respond to questions about its training process, and Kratsios did not share more details about the sources of his allegations.
Kratsios’ tweet echoed comments from Treasury Secretary Scott Bessent that “we are finding watermarks of our U.S. large language models on many of the Chinese models, and that that’s unacceptable.” It’s not clear what those watermarks consist of, and the Treasury Department did not respond to a query.
However, experts are skeptical that distillation — the process of querying an LLM to determine its inner workings and copy its capabilities — is responsible for the advanced capabilities that Kimi K3 displays.
“I don’t think you get a model this strong and this quickly on the heels of Fable doing strictly distillation,” Braden Hancock, a researcher at the Laude Institute and co-founder of Snorkel AI, told TechCrunch. “There’s just not even frankly time, right? Fable’s only been publicly available since July 1st. You can’t distill that much data, train a model, and release it in two weeks.”
“I’ve been of the opinion that distillation is becoming less and less impactful over time as the Chinese models get closer to the frontier and the training regime shifts to [reinforcement learning],” Nathan Lambert, an AI researcher at the Allen Institute for AI, said in a podcast：https://www.interconnects.ai/p/open-models-recap-more-on-kimi-k3 released yesterday. “[I]f it were the case, everyone would be easily able to catch up to a GLM or to a K3 by using its data for distillation. But we have not, or we won’t see this, from supervised fine-tuning alone.”
Performing distillation requires a lab to systematically query its target model in order to generate data that can be used for post-training. Sometimes this explicitly involves asking the model to articulate its chain-of-thought to understand how it solves problems. Other times, the prompts and responses from a model are used to train a new model in a process called supervised fine-tuning, or SFT.
It’s this fine-tuning process that can result in a model ostensibly created by a third party claiming that it is Claude. Fine tuning is where, in Lambert’s view, the “model picks up its manners.”
But Lambert says that the benefits of SFT are becoming less important as models become more complex. To distill Fable-like capabilities would likely require reinforcement learning techniques. In many cases, that means having an agent of the larger model grade the smaller model’s responses, and adjusting based on the grade.
The more advanced techniques also require more significant infrastructure. Large reinforcement learning runs can require tens of millions of agents. Using a frontier lab’s API to do that “would be insanely expensive and potentially it would probably be a time bottleneck because these models are pretty slow and to be frank might not even give you a performance uplift.”
It seems likely that previous frontier models might have contributed to Kimi; Anthropic publicly accused：https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks Moonshot, DeepSeek, and MiniMax of systematically distilling its models earlier this year. Anthropic said it discovered millions of exchanges between its models and users it identified at those companies through IP addresses and other meta data. Those queries were “distinct from normal usage patterns, reflecting deliberate capability extraction rather than legitimate use.” Anthropic didn’t respond to TechCrunch’s queries about Fable distillation.
However, distillation is seen as common among AI companies, not just in China. Elon Musk testified：https://techcrunch.com/2026/04/30/elon-musk-testifies-that-xai-trained-grok-on-openai-models/ earlier this year that his company SpaceXAI distilled OpenAI models to develop Grok, and that the practice was common in the industry. The line between distillation and developing synthetic datasets, for example, can be fairly blurry.
“[I]n general, Americans are understating the technical expertise of these Chinese teams,” Hancock said. “One of the founders of Moonshot was a CMU PhD student. These are legitimate researchers and engineers doing solid work. …if American models ground to a halt, I think China’s progress would slow, but would still continue. They’re not just riding coattails here.”
It’s also hard to disentangle distillation from the second part of Kratsios’ comment — that Moonshot had obtained advanced Nvidia Chips, Grace Blackwell 300s, and also accessed GB300 equipped-servers in Thailand. Those chips are banned from export to China, but a black market exists, according to Sam Bresnick, a research fellow at Georgetown’s Center for Security and Emerging Technology. In May, the founder of Supermicro, a U.S. server builder, was indicted for smuggling advanced chips into China.
“I am a proponent of know your customer laws for data centers across the world,” Bresnick said. “If you are letting a company conduct huge training runs on your state-of-the-art hardware, there needs to be a reporting mechanism for who that company is and what they’re doing.”
President Joe Biden’s Department of Commerce proposed：https://www.federalregister.gov/documents/2024/01/29/2024-01580/taking-additional-steps-to-address-the-national-emergency-with-respect-to-significant-malicious federal know-your-customer rules for data centers in 2024, but no further progress appears to have been made under Donald Trump. Exporters shipping advanced chips abroad, however, are supposed to ensure：https://www.mayerbrown.com/en/insights/publications/2026/01/administration-policies-on-advanced-ai-chips-codified they are only used for approved purposes.
When you purchase through links in our articles, we may earn a small commission：https://techcrunch.com/techcrunch-affiliate-monetization-standards/. This doesn’t affect our editorial independence.
Scale faster. Grow your portfolio. Gain practical expertise. No matter your goal, Disrupt can empower you. Save up to $330 toda y!
SpaceX launches new V3 Starlink satellites but suffers another booster failure：https://techcrunch.com/2026/07/24/spacex-launches-new-v3-starlink-satellites-but-suffers-another-booster-failure/ Sean O'Kane：https://techcrunch.com/author/sean-okane/
US accuses American of allegedly wiping his phone using a ‘duress’ password during border search：https://techcrunch.com/2026/07/24/us-accuses-american-of-allegedly-wiping-his-phone-using-a-duress-password-during-border-search/ Zack Whittaker：https://techcrunch.com/author/zack-whittaker/
Anduril reportedly in talks to raise funding at $100B valuation, more than 3x last year’s mark：https://techcrunch.com/2026/07/24/anduril-reportedly-in-talks-to-raise-funding-at-100b-valuation-more-than-3x-last-years-mark/ Ram Iyer：https://techcrunch.com/author/ram-iyer/
Tesla’s robotaxis are moving in reverse：https://techcrunch.com/2026/07/23/teslas-robotaxis-are-moving-in-reverse/ Sean O'Kane：https://techcrunch.com/author/sean-okane/
Jack Dorsey is taking on Slack with Buzz, a group chat platform for teams and their AI agents：https://techcrunch.com/2026/07/21/jack-dorsey-is-taking-on-slack-with-buzz-a-group-chat-platform-for-teams-and-their-ai-agents/ Amanda Silberling：https://techcrunch.com/author/amanda-silberling/
Light made a flip phone — it’s colorful and it’s cheap：https://techcrunch.com/2026/07/21/light-is-modernizing-the-flip-phone-with-light-flip/ Amanda Silberling：https://techcrunch.com/author/amanda-silberling/
AI music generator Suno breach affects 55M users, per Have I Been Pwned：https://techcrunch.com/2026/07/21/ai-music-generator-suno-breach-affects-55m-users-per-have-i-been-pwned/ Zack Whittaker：https://techcrunch.com/author/zack-whittaker/
扫码直达文章详情