{"@context":"https://schema.org","@type":"NewsArticle","generatedAt":"2026-10-24T05:00:40.349Z","headline":"Baseten 工程师实测：LLM 生成的推理引擎比 vLLM 快最多 90%","description":"Baseten 工程师参考 MetaInfer 论文，让 Claude Code（Fable 5）为 Qwen-3.6-35B-A3B（NVFP4，单张 B200）自动构建推理引擎 VibeQwen，单流解码比 vLLM 0.25.1 快 90%，首 token 从 28ms 降至 12ms，并发 32 时吞吐高 71%。","url":"https://www.aioga.com/news/uwm1ml1igd8k2u81g14uf8pc3/","mainEntityOfPage":"https://www.aioga.com/news/uwm1ml1igd8k2u81g14uf8pc3/","datePublished":"2026-10-02T21:09:34.000Z","dateModified":"2026-10-02T21:09:34.000Z","inLanguage":"zh-CN","publisher":{"@type":"NewsMediaOrganization","name":"Aioga","url":"https://www.aioga.com"},"citation":["https://www.baseten.co/blog/agentic-inference-optimization-faster-than-sota","https://aihot.news/items/uwm1ml1igd8k2u81g14uf8pc3"],"canonicalUrl":"https://www.aioga.com/news/uwm1ml1igd8k2u81g14uf8pc3/","directAnswer":{"@type":"Answer","text":"Baseten 工程师称，Claude Code（Fable 5）参考 MetaInfer，为单张 B200 上 NVFP4 精度的 Qwen-3.6-35B-A3B 构建了 VibeQwen；单流解码较 vLLM 0.25.1 快最多 90%，摘要还报告首 token 延迟由 28ms 降至 12ms、并发 32 时吞吐高 71%。","url":"https://www.aioga.com/news/uwm1ml1igd8k2u81g14uf8pc3/","dateCreated":"2026-10-02T21:09:34.000Z","author":{"@type":"Organization","@id":"https://www.aioga.com/authors/aioga-editorial/#editorial-team","name":"Aioga Editorial Team","url":"https://www.aioga.com/authors/aioga-editorial/"}},"evidence":[{"@type":"CreativeWork","name":"Baseten 工程博客 source article","url":"https://www.baseten.co/blog/agentic-inference-optimization-faster-than-sota","datePublished":"2026-10-02T21:09:34.000Z","provider":{"@type":"Organization","name":"Baseten 工程博客","url":"https://www.baseten.co/blog/agentic-inference-optimization-faster-than-sota"}},{"@type":"CreativeWork","name":"AIHot archive record","url":"https://aihot.news/items/uwm1ml1igd8k2u81g14uf8pc3","datePublished":"2026-10-02T21:09:34.000Z","provider":{"@type":"Organization","name":"AIHot","url":"https://aihot.news/items/uwm1ml1igd8k2u81g14uf8pc3"}}],"aggregationSource":"Baseten 工程博客","originalPublisher":{"name":"Baseten 工程博客","url":"https://www.baseten.co/blog/agentic-inference-optimization-faster-than-sota"},"geoDeepAnswer":null,"article":{"id":"uwm1ml1igd8k2u81g14uf8pc3","slug":"uwm1ml1igd8k2u81g14uf8pc3","url":"https://www.aioga.com/news/uwm1ml1igd8k2u81g14uf8pc3/","title":"Baseten 工程师实测：LLM 生成的推理引擎比 vLLM 快最多 90%","title_en":"","summary":"Baseten 工程师参考 MetaInfer 论文，让 Claude Code（Fable 5）为 Qwen-3.6-35B-A3B（NVFP4，单张 B200）自动构建推理引擎 VibeQwen，单流解码比 vLLM 0.25.1 快 90%，首 token 从 28ms 降至 12ms，并发 32 时吞吐高 71%。","source":"Baseten 工程博客","sourceUrl":"https://www.baseten.co/blog/agentic-inference-optimization-faster-than-sota","aiHotUrl":"https://aihot.news/items/uwm1ml1igd8k2u81g14uf8pc3","publishedAt":"2026-10-02T21:09:34.000Z","category":"行业动态","score":72,"selected":true,"articleBody":["Claude Code with Fable 5 built a Qwen 3.6 inference engine that beat vLLM by up to 90% on decode speed and 2.3x on TTFT, using MetaInfer skills on one B200.","Baseten is the kind of place where we have Slack channels like #ai-papers-discuss and #model-performance-reading-group , and it was in one of these channels that a paper caught my attention recently: MetaInfer: A Knowledge Only LLM Inference Engine Generator SKILL Toolbox：https://arxiv.org/abs/2607.12875v1.","In it, the authors provide a skills-only framework for building custom inference engines from scratch, claiming their results beat SoTA open solutions like vLLM on common inference performance metrics like time per output token (TPOT). “Skills-only” here means no model post-training, and no specialized harnesses. I’ll admit, I was skeptical, but they provided a repo with the skill toolkit, and we’re still living that unlimited-token good life here, so I thought I’d give it a try on a popular model just to see how it did.","With Qwen-3.6-35B-A3B in NVFP4 precision on a single B200, our LLM-generated inference engine outperformed vLLM 0.25.1 (latest at the time of the experiment) by up to 90% on single-stream decode speed, and with 2.33x faster TTFT, opening up new ultra-low-latency use cases for the model that had not been practical before.","There are a few large-scale trends that are converging right now, creating fertile ground for discoveries and techniques like MetaInfer.","Models are getting smarter and better at long-horizon tasks, with the release of Anthropic’s Mythos 5 and Fable 5 models marking a meaningful step forward on the kind of tasks that can be reliably delegated to AI systems. The last such moment I remember was the release of Opus 4.5 in late 2025, at which point many engineers, myself included, moved from using AI coding tools as a spicy autocomplete to driving coding harnesses like Claude Code as our primary work interface.","At the same time, agent coordination workflows and primitives are getting packaged directly into popular harnesses, making complex multi-agent workflows more accessible and easier to operate. One important such feature is “Goals”, provided by both Claude Code and Codex, which keep models working and on-task toward measurable goals over long periods of time. I first encountered this experiment-measure-iterate loop in Karpathy’s Autoresearch：https://github.com/karpathy/autoresearch project about 6 months ago, but it's now just part of normal coding harnesses.","As the capabilities of AI systems keep increasing, so does the market demand for inference services, creating enormous pressure to continuously improve both cost and performance.","With all of these factors combined, it starts making a lot of sense to point LLMs at well-defined optimization problems and just let them grind away at solutions. Inference performance turns out to be an ideal domain for this kind of automated optimization. The problem space is well-defined in terms of quantifiable metrics like TTFT, TPOT, throughput-per-chip, and required memory. Even correctness can be numerically verified against full-precision reference implementations of the model, to ensure that performance gains do not come at the expense of accuracy.","This objective verifiability is critical to the success of autonomous optimization. We also know that there is a huge gap：https://arxiv.org/pdf/2606.26383 between current SoTA performance and speed-of-light theoretical maximum performance. Additionally, inference is enormously complex, and the search space for optimizations is very large, ranging from low-level CUDA kernel authoring all the way to serving-layer optimizations like dynamic batching.","LLMs may be very good at solving well-defined optimization problems, but now the challenge shifts to how to frame your problem as a well-defined optimization problem. You need to be able to define your constraints well, and choose your optimization targets carefully. The agent will absolutely try to game the system to “pass” the goal, so it’s up to you as the engineer to design the game in such a way that the agent’s success actually aligns with what you need. As an example, if you don’t set accuracy-preserving constraints, your “model” will converge on:","Such a simple model will have unbelievably good TTFT and TPOT scores, but is no longer useful for anything.","Existing open inference engines like vLLM, SGLang and TensorRT-LLM do an overall great job at serving a wide range of models on a large variety of accelerators at commercially acceptable performance levels. However, the benefits they offer in generality and ease of use come with a tradeoff: they are not hyper-optimized for any specific (model + accelerator + workload) tuple. The premise behind MetaInfer is that for any specific deployment, a highly specialized (i.e., constrained) engine can outperform these generalized engines.","In the original MetaInfer paper, they challenge off-the-shelf agents to build inference engines from a knowledge base of contracts and constraints, with no direct access to source code from open engines. It’s extremely likely that such source code was present during model training, but agents did not have direct file-level access. Agents expand the knowledge base as they work, proactively identifying gaps and filling them through structured experimentation and verification.","While the experimental purity taken on by the authors is admirable and interesting, we’re very outcome-oriented at Baseten. I did allow my agents to access SoTA open solutions for reference when needed, including finding pre-optimized kernels from vLLM and TensorRT-LLM where available, and only writing their own kernels when needed. Additionally, while the original paper focused on just the inference engine component, I broadened the scope to the full serving stack, with final outcomes measured against deployed multi-replica services behind a load balancer, using AIPerf：https://github.com/ai-dynamo/aiperf to generate load and measure performance.","In both the original paper and my extension, the result is two deliverables: the inference engine itself, and reusable extensions to the knowledge base that can be used in subsequent runs.","In kicking off the experiment, I gave Claude Code with Fable 5 access to the paper and the MetaInfer Repository：https://github.com/HuangPuStar/MetaInfer, as well as access to an NVIDIA B200 workstation via SSH. I told it the target hardware and provided a link to the Hugging Face repo for both the NVFP4 quantized weights：https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4 I wanted to use for Qwen-3.6-35B-A3B, as well as the original full-precision weights：https://huggingface.co/Qwen/Qwen3.6-35B-A3B to be used as an accuracy oracle. The agent was also able to deploy engine candidates to Baseten, and to run AIPerf profiles against the deployed endpoint.","We have internal agent skills and MCPs to facilitate all of the required Baseten interactions, and the agent did have access to those as well. I set a /goal telling it I wanted to beat vLLM by 20% on all performance metrics without accuracy loss vs. the NVFP4 baseline.","I was surprised and impressed both by the final results and the relatively low amount of human intervention required. Claude worked mostly autonomously for roughly 1 week on this project, with me providing occasional steering to keep it from over-fixating on one particular traffic shape or another, and to keep it focused on production inference performance, and not just isolated engine benchmarks. The MetaInfer framework uses a series of immutable gates and checks that enable productive long-horizon work like this.","Claude would periodically pause to ask me to approve any changes that came with accuracy differences. I ultimately allowed it to have subtle numerically different outputs from the reference NVFP4 implementation, as long as overall accuracy vs. the BF16 baseline was at least as good.","I did very little to manually manage agent coordination or context management other than requesting that it use Fable 5 for all subagents after some disappointing early results from smaller models. It underwent automatic context compaction many times, and consumed roughly 1.7 billion tokens (overwhelmingly cached input tokens) and ~200 B200 hours. It actually reached parity with vLLM relatively quickly, within the first few days, but I had it keep grinding away for the sake of science.","By the end, the generated engine, called VibeQwen , outperformed vLLM across all tested traffic patterns, and by very large margins in some cases. In head-to-head comparisons on a single B200, VibeQwen achieved 1,792 tokens per second (TPS) on single-stream, speculator-friendly text (repetitive and structured), vs. 943 TPS on a well-tuned vLLM deployment (a 90% improvement). It also returned its first token in 12ms on the tested corpus, vs 28ms with vLLM, a 2.3x improvement. In single-replica throughput at concurrency of 32, VibeQwen beat vLLM by 71%, at 10,307 output TPS vs 6,030 output TPS in vLLM.","The results from this first experiment were so encouraging that I decided to try it out on another model, of a completely different type, using the newly expanded knowledge base generated from the VibeQwen experiment. This time I wanted an optimized inference server for SAM 3.1：https://huggingface.co/facebook/sam3.1, a state of the art image segmentation model.","In this case, I used Facebook’s reference server：https://github.com/facebookresearch/sam3 as the baseline. Despite entirely different model architectures and modalities, the process worked very well, and we achieved 91 images/second on a single H100, a 50% throughput improvement vs. the baseline in just a couple of days and ~200M tokens.","This second experiment, called Sammie , was much faster and less expensive than the initial VibeQwen project, but due to different model architectures, accelerator types, and no control testing, it provides only suggestive, not definitive, evidence that the knowledge base reuse has value.","While VibeQwen and Sammie are still very much experimental, with neither serving production traffic currently, the performance improvements vs. SoTA baselines are quite remarkable. They were also delivered in relatively little time (days), with very little human labor (hours), and a low enough cost ($100s for Sammie to $1000s for VibeQwen) to be very compelling in a world where companies spend tens or even hundreds of millions of dollars annually on inference. At such large scales, even single-digit percentage improvements can represent millions of dollars in savings, making the investment easy to justify.","The opportunity space for AI to create value in optimization problems is vast, and AI systems themselves are prime candidates for these autonomous optimizations, as we’ve seen. The cost of doing so will continue to come down as inference systems get more efficient, and as more models achieve the level of intelligence required to succeed at the task.","While VibeQwen and Sammie were built with Fable 5 in late July, it’s October now, and the open frontier has nearly caught up, with models like Kimi K3：https://www.baseten.co/library/kimi-k3/ and GLM-5.3：https://www.baseten.co/library/glm-53/ scoring very close to the best closed models on many benchmarks. Between rising model intelligence, falling inference costs, and efficiency gains from the reusable knowledge base, it’s easy to imagine a not-so-distant future where these optimizations become so cheap and reliable that they are a standard part of the model deployment lifecycle.","Join us in building the inference platform for production AI.","Dedicated Inference：/products/dedicated-inference/","Model Labs：/products/baseten-for-model-labs/","Model Runtimes：/platform/model-performance/","Infrastructure：/platform/cloud-native-infrastructure/","Multi-cloud：/products/multi-cloud-capacity-management/","Self-hosted：/deployments/baseten-self-hosted/","Hybrid：/deployments/baseten-hybrid/","Forward deployed engineers：/platform/embedded-engineering/","Transcription：/solutions/transcription/","Image Generation：/solutions/image-generation/","Text-to-speech：/solutions/text-to-speech/","Large language models：/solutions/llms/","Compound AI：/solutions/compound-ai/","Documentation：https://docs.baseten.co/","Savings calculator：/resources/calculator/","Trust Center：https://trust.baseten.co","Terms and Conditions：/terms-and-conditions/","Service Level Agreement：/service-level-agreement/","DeepSeek V4.1 Flash：/library/deepseek-v41-flash/","GLM-5.3-Flash：/library/glm-53-flash/"],"articleImages":[{"sourceUrl":"https://www.baseten.co/_next/image/?url=https%3A%2F%2Fwww.datocms-assets.com%2F104802%2F1790974437-baseten-blog-2026-thumbnails-20.png%3Fauto%3Dformat%26fit%3Dcrop%26h%3D630%26w%3D1200&w=3840&q=100","alt":"Baseten engineers leverage agentic inference optimization techniques for up to 90% faster inference","afterParagraph":0,"url":"/media/articles/uwm1ml1igd8k2u81g14uf8pc3/d89ba7c48b779ab1.avif"},{"sourceUrl":"https://www.baseten.co/_next/image/?url=https%3A%2F%2Fwww.datocms-assets.com%2F104802%2F1790965649-converging-trends-baseten-brand.png%3Fauto%3Dformat%26w%3D1200&w=3840&q=75","alt":"Four converging trends: smarter models, agent coordination, measurable optimization, and inference demand.","afterParagraph":4,"url":"/media/articles/uwm1ml1igd8k2u81g14uf8pc3/7e988c79be2a8e29.avif"},{"sourceUrl":"https://www.baseten.co/_next/image/?url=https%3A%2F%2Fwww.datocms-assets.com%2F104802%2F1790966949-ai-inference-demand-baseten-brand.png%3Fauto%3Dformat%26w%3D1200&w=3840&q=75","alt":"AI inference demand is compounding. Source: Google I/O 2026.","afterParagraph":6,"url":"/media/articles/uwm1ml1igd8k2u81g14uf8pc3/b18999db4250f4d5.avif"},{"sourceUrl":"https://www.baseten.co/_next/image/?url=https%3A%2F%2Fwww.datocms-assets.com%2F104802%2F1790968273-vibeqwen-vs-vllm-baseten-brand.png%3Fauto%3Dformat%26w%3D1200&w=3840&q=75","alt":"VibeQwen, the inference engine Claude generated for Qwen-3.6-35B-A3B in NVFP4, was tested against a tuned vLLM 0.25.1 deployment on a single B200. It decoded 90% faster on single-stream text and cut time to first token from 28 ms to 12 ms. At concurrency 32, it delivered 71% more output throughput.","afterParagraph":21,"url":"/media/articles/uwm1ml1igd8k2u81g14uf8pc3/d54e6fd5b6020d23.avif"},{"sourceUrl":"https://www.baseten.co/_next/image/?url=https%3A%2F%2Fwww.datocms-assets.com%2F104802%2F1790968939-sammie-segmentation-baseten-brand.png%3Fauto%3Dformat%26w%3D1200&w=3840&q=75","alt":"Sammie, the inference server Claude built for SAM 3.1, takes an image and a short text prompt and returns a mask for each matching object. On a single H100, Sammie processed 91 images per second, 50% more than Meta's reference server.","afterParagraph":24,"url":"/media/articles/uwm1ml1igd8k2u81g14uf8pc3/d13dd4af5504a604.avif"},{"sourceUrl":"https://www.baseten.co/_next/image/?url=%2F_next%2Fstatic%2Fmedia%2Fsoc2-type2.660a008c.png&w=640&q=75","alt":"SOC 2 Type II Certified","afterParagraph":46,"url":"/media/articles/uwm1ml1igd8k2u81g14uf8pc3/fb6c79c61bd4463a.avif"}],"mediaStatus":"ok","articleBodyZh":["Claude Code 与 Fable 5 构建了一个 Qwen 3.6 推理引擎，在解码速度上比 vLLM 提高了最多 90%，在 TTFT 上提高了 2.3 倍，使用 MetaInfer 技能在一台 B200 上实现。","Baseten 是那种地方，我们有像 #ai-papers-discuss 和 #model-performance-reading-group 这样的 Slack 频道，最近就是在其中一个频道里，一篇论文吸引了我的注意：MetaInfer: A Knowledge Only LLM Inference Engine Generator SKILL Toolbox：https://arxiv.org/abs/2607.12875v1。","在论文中，作者提供了一个仅技能框架，用于从零构建自定义推理引擎，并声称其结果在常见的推理性能指标上，如每输出标记时间（TPOT），优于 vLLM 等 SoTA 开源解决方案。这里的“仅技能”意味着不进行模型后训练，也不使用专门的工具。我得承认，我起初持怀疑态度，但他们提供了一个带有技能工具包的仓库，而我们现在仍然在享受无限 token 的美好生活，因此我决定在一个流行模型上尝试一下，看看效果如何。","使用 Qwen-3.6-35B-A3B，NVFP4 精度，在单个 B200 上，我们生成的 LLM 推理引擎在单流解码速度上比 vLLM 0.25.1（实验时的最新版本）提高了最多 90%，TTFT 提高了 2.33 倍，为模型开启了此前不实用的超低延迟新应用场景。","目前有几个大规模趋势正在汇聚，为像 MetaInfer 这样的发现和技术创造了肥沃的土壤。","模型变得更聪明，对长周期任务的处理能力更强，Anthropic 发布的 Mythos 5 和 Fable 5 模型标志着在可以可靠委托给 AI 系统的任务类型上迈出了重要一步。我记得上一次类似的时刻是 2025 年底 Opus 4.5 的发布，到那时，许多工程师，包括我自己，都从将 AI 编码工具作为辅助自动补全，转向使用像 Claude Code 这样的编码工具作为主要工作接口。","与此同时，代理协调工作流和原语正被直接打包进流行的开发框架，使复杂的多代理工作流程更易于访问和操作。其中一个重要功能是由 Claude Code 和 Codex 提供的“Goals（目标）”，它能让模型在长期内保持工作并专注于可衡量的目标。我大约在六个月前第一次在 Karpathy 的 Autoresearch：https://github.com/karpathy/autoresearch 项目中遇到过这种实验-测量-迭代的循环，但现在它已经成为常规编码框架的一部分。","随着 AI 系统能力的不断提高，对推理服务的市场需求也在增长，从而带来了不断提高成本和性能的巨大压力。","将所有这些因素结合起来，将大型语言模型指向定义明确的优化问题，让它们专注于寻找解决方案开始变得非常合理。推理性能证明是这类自动化优化的理想领域。问题空间在可量化指标方面如 TTFT、TPOT、每芯片吞吐量和所需内存都是明确定义的。即使是正确性，也可以通过与模型的全精度参考实现进行数值验证，以确保性能提升不会以牺牲准确性为代价。","这种目标的可验证性对自主优化的成功至关重要。我们还知道当前最先进性能与光速理论最大性能之间存在巨大差距：https://arxiv.org/pdf/2606.26383。此外，推理极其复杂，优化的搜索空间非常大，从低级 CUDA 内核编写到服务层优化，如动态批处理，范围广泛。","大型语言模型可能非常擅长解决定义明确的优化问题，但挑战现在转向如何将你的问题构建为定义明确的优化问题。你需要能够很好地定义你的约束，并仔细选择优化目标。代理绝对会尝试利用系统来“通过”目标，因此作为工程师，你需要设计游戏方式，使代理的成功实际上与你的需求保持一致。例如，如果你没有设置保持准确性的约束，你的“模型”将收敛于：","这样一个简单的模型会有难以置信的优秀TTFT和TPOT评分，但已经没有实际用途了。","现有的开源推理引擎如vLLM、SGLang和TensorRT-LLM在为各种加速器提供广泛模型的服务方面总体表现出色，并且性能达到商业可接受水平。然而，它们在通用性和易用性上的优势也伴随着一个权衡：它们不会针对任何特定的（模型 + 加速器 + 工作负载）组合进行超级优化。MetaInfer的前提是，对于任何特定部署，一个高度专门化（即受限的）引擎可以超越这些通用引擎的性能。","在最初的MetaInfer论文中，他们挑战现成的代理从合同和约束的知识库构建推理引擎，而无法直接访问开源引擎的源代码。在模型训练期间，这类源代码很可能存在，但代理无法直接进行文件级访问。代理在工作中会扩展知识库，主动识别空白并通过结构化实验和验证加以填补。","尽管作者在实验上的纯粹性令人钦佩且有趣，但我们在Baseten非常关注结果。我确实允许我的代理在需要时访问最新的开源解决方案以供参考，包括在可用的情况下从vLLM和TensorRT-LLM找到预优化的内核，仅在必要时编写自己的内核。此外，虽然原论文只关注推理引擎组件，我将范围扩展到了完整的服务栈，最终结果通过在负载均衡器后的多副本部署服务进行衡量，使用AIPerf：https://github.com/ai-dynamo/aiperf 来生成负载并测量性能。","在原论文和我的扩展中，结果是两个可交付成果：推理引擎本身，以及可在后续运行中使用的可重用知识库扩展。","在启动实验时，我让 Claude Code 与 Fable 5 获取了论文和 MetaInfer 仓库的访问权限：https://github.com/HuangPuStar/MetaInfer，以及通过 SSH 访问 NVIDIA B200 工作站的权限。我告诉它目标硬件，并提供了 Hugging Face 仓库的链接，用于获取 NVFP4 量化权重：https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4，我希望将其用于 Qwen-3.6-35B-A3B，同时提供了原始全精度权重的链接：https://huggingface.co/Qwen/Qwen3.6-35B-A3B，用作准确性参考。该代理还能将引擎候选部署到 Baseten，并对已部署的端点运行 AIPerf 性能分析。","我们内部有用于协助所有 Baseten 交互的代理技能和 MCPs，并且该代理也能够访问这些资源。我设置了一个 /goal，告诉它我希望在不降低准确度的前提下，相比 NVFP4 基线，在所有性能指标上超过 vLLM 20%。","最终结果以及所需人工干预相对较少，这让我既惊讶又印象深刻。Claude 在这个项目上大部分时间都是自主工作的，大约持续了一周，我偶尔提供指导，以防它过度专注于某种特定的流量形态，并确保它关注的是生产环境推理性能，而不仅仅是孤立的引擎基准测试。MetaInfer 框架使用一系列不可变的门控和检查机制，使得像这样的长时间、高效工作成为可能。","Claude 会定期暂停，请我批准任何可能影响准确性的更改。我最终允许它在与参考 NVFP4 实现存在细微数值差异的情况下工作，只要总体准确性相较 BF16 基线至少保持不变。","我几乎没有手动管理代理协调或上下文管理，除了在早期一些小模型结果不理想时，要求所有子代理都使用 Fable 5。它多次进行自动上下文压缩，并消耗了大约 17 亿个代币（绝大多数是缓存的输入代币）和约 200 个 B200 小时。实际上，它在前几天就相对快速地达到了与 vLLM 的同等水平，但我让它继续努力，以科学为目的。","到最后，名为 VibeQwen 的生成引擎在所有测试的流量模式下都优于 vLLM，在某些情况下优势非常显著。在单个 B200 上的正面比较中，VibeQwen 在单流、预测者友好的文本（重复且结构化）上实现了每秒 1,792 个令牌 (TPS)，而经过良好调优的 vLLM 部署为每秒 943 个令牌（提高了 90%）。在测试语料库上，VibeQwen 返回第一个令牌的时间为 12 毫秒，而 vLLM 为 28 毫秒，提升了 2.3 倍。在并发为 32 的单副本吞吐量测试中，VibeQwen 比 vLLM 高出 71%，输出 TPS 为 10,307，而 vLLM 为 6,030。","首次实验的结果如此令人鼓舞，以至于我决定用从 VibeQwen 实验生成的新扩展知识库在另一种完全不同类型的模型上进行尝试。这一次，我希望为 SAM 3.1（https://huggingface.co/facebook/sam3.1，一种最先进的图像分割模型）构建一个优化的推理服务器。","在这种情况下，我使用了 Facebook 的参考服务器（https://github.com/facebookresearch/sam3）作为基线。尽管模型架构和模态完全不同，这一过程仍然非常有效，我们在单个 H100 上达到了每秒 91 张图像的速度，比基线提高了 50%的吞吐量，仅用了几天时间和约 2 亿个令牌。","这第二次实验名为 Sammie，比最初的 VibeQwen 项目快得多也更便宜，但由于模型架构、加速器类型不同且没有控制测试，它只能提供暗示性的证据，而非确定性的证据，表明知识库重用具有价值。","虽然 VibeQwen 和 Sammie 仍处于实验阶段，目前都没有服务生产流量，但与最先进基线相比的性能提升非常显著。它们也是在相对短的时间内（几天）、极少的人力（数小时）和可接受的低成本（Sammie 仅需几百美元，VibeQwen 需几千美元）实现的，这在一个公司每年在推理上花费数千万甚至数亿美元的世界里非常具有吸引力。在如此大规模下，即便是个位数百分比的提升，也可能意味着数百万美元的节省，使得投资变得容易合理化。","在优化问题中，人工智能创造价值的机会空间是巨大的，正如我们所见，人工智能系统本身就是这些自主优化的主要候选对象。随着推理系统效率的提高，以及更多模型达到完成任务所需的智能水平，相关成本将持续下降。","虽然 VibeQwen 和 Sammie 是在七月底用 Fable 5 构建的，但现在已经是十月，开放前沿几乎赶上了，像 Kimi K3：https://www.baseten.co/library/kimi-k3/ 和 GLM-5.3：https://www.baseten.co/library/glm-53/ 这样模型在许多基准测试中的得分已经非常接近最优秀的封闭模型。随着模型智能的提升、推理成本的下降以及可重用知识库带来的效率提升，我们很容易想象一个不太遥远的未来，这些优化将变得如此低成本且可靠，以至于成为模型部署生命周期中的标准部分。","加入我们，共同构建用于生产人工智能的推理平台。","专用推理：/products/dedicated-inference/","Model Labs：/products/baseten-for-model-labs/","模型运行时：/platform/model-performance/","基础设施：/平台/云原生基础设施/","多云：/products/multi-cloud-capacity-management/","自宿主：/deployments/baseten-self-hosted/","混合：/deployments/baseten-hybrid/","前置部署工程师：/平台/嵌入式工程/","转录：/solutions/transcription/","图像生成：/解决方案/图像生成/","文本转语音：/solutions/text-to-speech/","大型语言模型：/solutions/llms/","复合人工智能：/solutions/compound-ai/","文档：https://docs.baseten.co/","储蓄计算器：/resources/calculator/","信任中心：https://trust.baseten.co","条款和条件：/terms-and-conditions/","服务等级协议：/service-level-agreement/","DeepSeek V4.1 闪存：/library/deepseek-v41-flash/","GLM-5.3-Flash：/library/glm-53-flash/"],"translationStatus":"translated","bodyOrigin":"source-page","editorial":{"summary":"Baseten 工程师称，Claude Code（Fable 5）参考 MetaInfer，为单张 B200 上 NVFP4 精度的 Qwen-3.6-35B-A3B 构建了 VibeQwen；单流解码较 vLLM 0.25.1 快最多 90%，摘要还报告首 token 延迟由 28ms 降至 12ms、并发 32 时吞吐高 71%。","background":"文章介绍，MetaInfer 提供仅依靠技能工具箱生成推理引擎的框架，作者称其无需模型后训练或专用测试框架。Baseten 工程师表示，实验选择 Qwen-3.6-35B-A3B，并将生成的引擎与实验时最新的 vLLM 0.25.1 比较。","viewpoint":"Aioga 判断：这次测试展示了技能工具与代码模型参与推理引擎构建的实践案例，但结果对应特定模型、精度、硬件和基准条件，不能据此认定其他模型或部署环境也会获得同等提升。","implications":"可能影响：推理服务团队可关注自动生成引擎在特定工作负载上的表现，并需要结合自身模型、硬件和测试口径复核；单次对比不代表普遍性能优势，也不足以证明整体成本或可靠性改善。","nextStep":"后续观察：可关注 Baseten 是否披露更完整的测试方法、重复实验结果及不同负载下的数据，并留意其他模型与硬件配置能否复现文中所述表现；现有材料未说明这些验证结果。","evidenceRefs":["title","summary","articleBody","source"],"status":"published","aiGenerated":true,"autoApproved":true,"generatedBy":"aioga-editorial:gpt-5.6-sol","reviewedBy":"aioga-editorial-review:gpt-5.6-sol","generatedAt":"2026-10-03T15:19:07.011Z","sourceHash":"e4ba74ddfeea04b1","review":{"approved":true,"groundedness":92,"clarity":91,"duplicationRisk":10,"blockingIssues":[],"notes":["“专用测试框架”可考虑改为“专用 harness（测试或执行框架）”，以更贴近来源中的 specialized harnesses。"]},"validation":{"passed":true,"mode":"ai-auto","revisions":0,"checks":["schema","length","source-attribution","editorial-labels","inference-boundary","low-source-overlap","no-html","independent-ai-review"]}},"tags":["行业动态","Baseten 工程博客"],"translations":{"zh-CN":{"title":"Baseten 工程师实测：LLM 生成的推理引擎比 vLLM 快最多 90%","summary":"Baseten 工程师参考 MetaInfer 论文，让 Claude Code（Fable 5）为 Qwen-3.6-35B-A3B（NVFP4，单张 B200）自动构建推理引擎 VibeQwen，单流解码比 vLLM 0.25.1 快 90%，首 token 从 28ms 降至 12ms，并发 32 时吞吐高 71%。","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Baseten 工程师实测：LLM 生成的推理引擎比 vLLM 快最多 90% - Aioga AI资讯","description":"Baseten 工程师参考 MetaInfer 论文，让 Claude Code（Fable 5）为 Qwen-3.6-35B-A3B（NVFP4，单张 B200）自动构建推理引擎 VibeQwen，单流解码比 vLLM 0.25.1 快 90%，首 token 从 28ms 降至 12ms，并发 32 时吞吐高 71%。","url":"https://www.aioga.com/news/uwm1ml1igd8k2u81g14uf8pc3/","articleBody":["Claude Code 与 Fable 5 构建了一个 Qwen 3.6 推理引擎，在解码速度上比 vLLM 提高了最多 90%，在 TTFT 上提高了 2.3 倍，使用 MetaInfer 技能在一台 B200 上实现。","Baseten 是那种地方，我们有像 #ai-papers-discuss 和 #model-performance-reading-group 这样的 Slack 频道，最近就是在其中一个频道里，一篇论文吸引了我的注意：MetaInfer: A Knowledge Only LLM Inference Engine Generator SKILL Toolbox：https://arxiv.org/abs/2607.12875v1。","在论文中，作者提供了一个仅技能框架，用于从零构建自定义推理引擎，并声称其结果在常见的推理性能指标上，如每输出标记时间（TPOT），优于 vLLM 等 SoTA 开源解决方案。这里的“仅技能”意味着不进行模型后训练，也不使用专门的工具。我得承认，我起初持怀疑态度，但他们提供了一个带有技能工具包的仓库，而我们现在仍然在享受无限 token 的美好生活，因此我决定在一个流行模型上尝试一下，看看效果如何。","使用 Qwen-3.6-35B-A3B，NVFP4 精度，在单个 B200 上，我们生成的 LLM 推理引擎在单流解码速度上比 vLLM 0.25.1（实验时的最新版本）提高了最多 90%，TTFT 提高了 2.33 倍，为模型开启了此前不实用的超低延迟新应用场景。","目前有几个大规模趋势正在汇聚，为像 MetaInfer 这样的发现和技术创造了肥沃的土壤。","模型变得更聪明，对长周期任务的处理能力更强，Anthropic 发布的 Mythos 5 和 Fable 5 模型标志着在可以可靠委托给 AI 系统的任务类型上迈出了重要一步。我记得上一次类似的时刻是 2025 年底 Opus 4.5 的发布，到那时，许多工程师，包括我自己，都从将 AI 编码工具作为辅助自动补全，转向使用像 Claude Code 这样的编码工具作为主要工作接口。","与此同时，代理协调工作流和原语正被直接打包进流行的开发框架，使复杂的多代理工作流程更易于访问和操作。其中一个重要功能是由 Claude Code 和 Codex 提供的“Goals（目标）”，它能让模型在长期内保持工作并专注于可衡量的目标。我大约在六个月前第一次在 Karpathy 的 Autoresearch：https://github.com/karpathy/autoresearch 项目中遇到过这种实验-测量-迭代的循环，但现在它已经成为常规编码框架的一部分。","随着 AI 系统能力的不断提高，对推理服务的市场需求也在增长，从而带来了不断提高成本和性能的巨大压力。","将所有这些因素结合起来，将大型语言模型指向定义明确的优化问题，让它们专注于寻找解决方案开始变得非常合理。推理性能证明是这类自动化优化的理想领域。问题空间在可量化指标方面如 TTFT、TPOT、每芯片吞吐量和所需内存都是明确定义的。即使是正确性，也可以通过与模型的全精度参考实现进行数值验证，以确保性能提升不会以牺牲准确性为代价。","这种目标的可验证性对自主优化的成功至关重要。我们还知道当前最先进性能与光速理论最大性能之间存在巨大差距：https://arxiv.org/pdf/2606.26383。此外，推理极其复杂，优化的搜索空间非常大，从低级 CUDA 内核编写到服务层优化，如动态批处理，范围广泛。","大型语言模型可能非常擅长解决定义明确的优化问题，但挑战现在转向如何将你的问题构建为定义明确的优化问题。你需要能够很好地定义你的约束，并仔细选择优化目标。代理绝对会尝试利用系统来“通过”目标，因此作为工程师，你需要设计游戏方式，使代理的成功实际上与你的需求保持一致。例如，如果你没有设置保持准确性的约束，你的“模型”将收敛于：","这样一个简单的模型会有难以置信的优秀TTFT和TPOT评分，但已经没有实际用途了。","现有的开源推理引擎如vLLM、SGLang和TensorRT-LLM在为各种加速器提供广泛模型的服务方面总体表现出色，并且性能达到商业可接受水平。然而，它们在通用性和易用性上的优势也伴随着一个权衡：它们不会针对任何特定的（模型 + 加速器 + 工作负载）组合进行超级优化。MetaInfer的前提是，对于任何特定部署，一个高度专门化（即受限的）引擎可以超越这些通用引擎的性能。","在最初的MetaInfer论文中，他们挑战现成的代理从合同和约束的知识库构建推理引擎，而无法直接访问开源引擎的源代码。在模型训练期间，这类源代码很可能存在，但代理无法直接进行文件级访问。代理在工作中会扩展知识库，主动识别空白并通过结构化实验和验证加以填补。","尽管作者在实验上的纯粹性令人钦佩且有趣，但我们在Baseten非常关注结果。我确实允许我的代理在需要时访问最新的开源解决方案以供参考，包括在可用的情况下从vLLM和TensorRT-LLM找到预优化的内核，仅在必要时编写自己的内核。此外，虽然原论文只关注推理引擎组件，我将范围扩展到了完整的服务栈，最终结果通过在负载均衡器后的多副本部署服务进行衡量，使用AIPerf：https://github.com/ai-dynamo/aiperf 来生成负载并测量性能。","在原论文和我的扩展中，结果是两个可交付成果：推理引擎本身，以及可在后续运行中使用的可重用知识库扩展。","在启动实验时，我让 Claude Code 与 Fable 5 获取了论文和 MetaInfer 仓库的访问权限：https://github.com/HuangPuStar/MetaInfer，以及通过 SSH 访问 NVIDIA B200 工作站的权限。我告诉它目标硬件，并提供了 Hugging Face 仓库的链接，用于获取 NVFP4 量化权重：https://huggingface.co/nvidia/Qwen3.6-35B-A3B-NVFP4，我希望将其用于 Qwen-3.6-35B-A3B，同时提供了原始全精度权重的链接：https://huggingface.co/Qwen/Qwen3.6-35B-A3B，用作准确性参考。该代理还能将引擎候选部署到 Baseten，并对已部署的端点运行 AIPerf 性能分析。","我们内部有用于协助所有 Baseten 交互的代理技能和 MCPs，并且该代理也能够访问这些资源。我设置了一个 /goal，告诉它我希望在不降低准确度的前提下，相比 NVFP4 基线，在所有性能指标上超过 vLLM 20%。","最终结果以及所需人工干预相对较少，这让我既惊讶又印象深刻。Claude 在这个项目上大部分时间都是自主工作的，大约持续了一周，我偶尔提供指导，以防它过度专注于某种特定的流量形态，并确保它关注的是生产环境推理性能，而不仅仅是孤立的引擎基准测试。MetaInfer 框架使用一系列不可变的门控和检查机制，使得像这样的长时间、高效工作成为可能。","Claude 会定期暂停，请我批准任何可能影响准确性的更改。我最终允许它在与参考 NVFP4 实现存在细微数值差异的情况下工作，只要总体准确性相较 BF16 基线至少保持不变。","我几乎没有手动管理代理协调或上下文管理，除了在早期一些小模型结果不理想时，要求所有子代理都使用 Fable 5。它多次进行自动上下文压缩，并消耗了大约 17 亿个代币（绝大多数是缓存的输入代币）和约 200 个 B200 小时。实际上，它在前几天就相对快速地达到了与 vLLM 的同等水平，但我让它继续努力，以科学为目的。","到最后，名为 VibeQwen 的生成引擎在所有测试的流量模式下都优于 vLLM，在某些情况下优势非常显著。在单个 B200 上的正面比较中，VibeQwen 在单流、预测者友好的文本（重复且结构化）上实现了每秒 1,792 个令牌 (TPS)，而经过良好调优的 vLLM 部署为每秒 943 个令牌（提高了 90%）。在测试语料库上，VibeQwen 返回第一个令牌的时间为 12 毫秒，而 vLLM 为 28 毫秒，提升了 2.3 倍。在并发为 32 的单副本吞吐量测试中，VibeQwen 比 vLLM 高出 71%，输出 TPS 为 10,307，而 vLLM 为 6,030。","首次实验的结果如此令人鼓舞，以至于我决定用从 VibeQwen 实验生成的新扩展知识库在另一种完全不同类型的模型上进行尝试。这一次，我希望为 SAM 3.1（https://huggingface.co/facebook/sam3.1，一种最先进的图像分割模型）构建一个优化的推理服务器。","在这种情况下，我使用了 Facebook 的参考服务器（https://github.com/facebookresearch/sam3）作为基线。尽管模型架构和模态完全不同，这一过程仍然非常有效，我们在单个 H100 上达到了每秒 91 张图像的速度，比基线提高了 50%的吞吐量，仅用了几天时间和约 2 亿个令牌。","这第二次实验名为 Sammie，比最初的 VibeQwen 项目快得多也更便宜，但由于模型架构、加速器类型不同且没有控制测试，它只能提供暗示性的证据，而非确定性的证据，表明知识库重用具有价值。","虽然 VibeQwen 和 Sammie 仍处于实验阶段，目前都没有服务生产流量，但与最先进基线相比的性能提升非常显著。它们也是在相对短的时间内（几天）、极少的人力（数小时）和可接受的低成本（Sammie 仅需几百美元，VibeQwen 需几千美元）实现的，这在一个公司每年在推理上花费数千万甚至数亿美元的世界里非常具有吸引力。在如此大规模下，即便是个位数百分比的提升，也可能意味着数百万美元的节省，使得投资变得容易合理化。","在优化问题中，人工智能创造价值的机会空间是巨大的，正如我们所见，人工智能系统本身就是这些自主优化的主要候选对象。随着推理系统效率的提高，以及更多模型达到完成任务所需的智能水平，相关成本将持续下降。","虽然 VibeQwen 和 Sammie 是在七月底用 Fable 5 构建的，但现在已经是十月，开放前沿几乎赶上了，像 Kimi K3：https://www.baseten.co/library/kimi-k3/ 和 GLM-5.3：https://www.baseten.co/library/glm-53/ 这样模型在许多基准测试中的得分已经非常接近最优秀的封闭模型。随着模型智能的提升、推理成本的下降以及可重用知识库带来的效率提升，我们很容易想象一个不太遥远的未来，这些优化将变得如此低成本且可靠，以至于成为模型部署生命周期中的标准部分。","加入我们，共同构建用于生产人工智能的推理平台。","专用推理：/products/dedicated-inference/","Model Labs：/products/baseten-for-model-labs/","模型运行时：/platform/model-performance/","基础设施：/平台/云原生基础设施/","多云：/products/multi-cloud-capacity-management/","自宿主：/deployments/baseten-self-hosted/","混合：/deployments/baseten-hybrid/","前置部署工程师：/平台/嵌入式工程/","转录：/solutions/transcription/","图像生成：/解决方案/图像生成/","文本转语音：/solutions/text-to-speech/","大型语言模型：/solutions/llms/","复合人工智能：/solutions/compound-ai/","文档：https://docs.baseten.co/","储蓄计算器：/resources/calculator/","信任中心：https://trust.baseten.co","条款和条件：/terms-and-conditions/","服务等级协议：/service-level-agreement/","DeepSeek V4.1 闪存：/library/deepseek-v41-flash/","GLM-5.3-Flash：/library/glm-53-flash/"]},"en":{"title":"Actual measurement by Baseten engineers: The inference engine generated by LLM is up to 90% faster than vLLM","summary":"Baseten engineers used the MetaInfer paper as a reference and had Claude Code (Fable 5) automatically build the VibeQwen inference engine for Qwen-3.6-35B-A3B (NVFP4, single B200). Single-stream decoding is 90% faster than vLLM 0.25.1, time to first token drops from 28 ms to 12 ms, and throughput is 71% higher at concurrency 32.","category":"Industry","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Actual measurement by Baseten engineers: The inference engine generated by LLM is up to 90% faster than vLLM - Aioga AI News","description":"Baseten engineers used the MetaInfer paper as a reference and had Claude Code (Fable 5) automatically build the VibeQwen inference engine for Qwen-3.6-35B-A3B (NVFP4, single B200)....","url":"https://www.aioga.com/en/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:27:12.141Z"},"ja":{"title":"Baseten エンジニアによる実測: LLM によって生成された推論エンジンは vLLM より最大 90% 高速","summary":"Baseten のエンジニアは MetaInfer の論文を参照し、Qwen-3.6-35B-A3B (NVFP4、シングル B200) 用の推論エンジン VibeQwen を自動的に構築するように Claude Code (表 5) に依頼しました。シングルストリームのデコードは vLLM 0.25.1 より 90% 高速になり、最初のトークンは 28 ミリ秒から 12 ミリ秒に短縮され、同時実行数が 32 の場合のスループットは 71% 向上します。","category":"業界動向","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Baseten エンジニアによる実測: LLM によって生成された推論エンジンは vLLM より最大 90% 高速 - Aioga AIニュース","description":"Baseten のエンジニアは MetaInfer の論文を参照し、Qwen-3.6-35B-A3B (NVFP4、シングル B200) 用の推論エンジン VibeQwen を自動的に構築するように Claude Code (表 5) に依頼しました。シングルストリームのデコードは vLLM 0.25.1 より 90% 高速になり、最初のトークンは 28 ミ...","url":"https://www.aioga.com/ja/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:27:56.384Z"},"ko":{"title":"Baseten 엔지니어들이 직접 테스트한 결과: LLM이 생성한 추론 엔진은 vLLM보다 최대 90% 빠르다","summary":"Baseten 엔지니어들은 MetaInfer 논문을 참조하여 Claude Code(Fable 5)에게 Qwen-3.6-35B-A3B(NVFP4, 단일 B200)용 추론 엔진 VibeQwen을 자동으로 구축하도록 요청했습니다. 단일 스트림 디코딩은 vLLM 0.25.1보다 90% 빠르고, 첫 번째 토큰은 28ms에서 12ms로 줄었으며, 동시성이 32일 때 처리량은 71% 더 높습니다.","category":"업계 동향","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Baseten 엔지니어들이 직접 테스트한 결과: LLM이 생성한 추론 엔진은 vLLM보다 최대 90% 빠르다 - Aioga AI 뉴스","description":"Baseten 엔지니어들은 MetaInfer 논문을 참조하여 Claude Code(Fable 5)에게 Qwen-3.6-35B-A3B(NVFP4, 단일 B200)용 추론 엔진 VibeQwen을 자동으로 구축하도록 요청했습니다. 단일 스트림 디코딩은 vLLM 0.25.1보다 90% 빠르고, 첫 번째 토큰은 28ms에서 12m...","url":"https://www.aioga.com/ko/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:27:53.733Z"},"es":{"title":"Pruebas reales de los ingenieros de Baseten: el motor de inferencia generado por LLM es hasta un 90 % más rápido que vLLM","summary":"Los ingenieros de Baseten se refieren al artículo de MetaInfer, permitiendo que Claude Code (Fable 5) construya automáticamente el motor de inferencia VibeQwen para Qwen-3.6-35B-A3B (NVFP4, una sola B200), decodificando un solo flujo un 90% más rápido que vLLM 0.25.1, con el primer token reducido de 28 ms a 12 ms, y con una concurrencia de 32 el rendimiento aumenta un 71%.","category":"Industria","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Pruebas reales de los ingenieros de Baseten: el motor de inferencia generado por LLM es hasta un 90 % más rápido que vLLM - Aioga Noticias de IA","description":"Los ingenieros de Baseten se refieren al artículo de MetaInfer, permitiendo que Claude Code (Fable 5) construya automáticamente el motor de inferencia VibeQwen para Qwen-3.6-35B-A3...","url":"https://www.aioga.com/es/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:33:42.087Z"},"fr":{"title":"Testé par les ingénieurs de Baseten : le moteur de raisonnement généré par LLM est jusqu'à 90 % plus rapide que vLLM","summary":"Les ingénieurs de Baseten se sont référés à l'article MetaInfer pour que Claude Code (Fable 5) construise automatiquement le moteur d'inférence VibeQwen pour Qwen-3.6-35B-A3B (NVFP4, carte B200 unique), avec un décodage en flux unique 90% plus rapide que vLLM 0.25.1, le premier token passant de 28 ms à 12 ms, et un débit supérieur de 71% en cas de concurrence de 32.","category":"Industrie","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Testé par les ingénieurs de Baseten : le moteur de raisonnement généré par LLM est jusqu'à 90 % plus rapide que vLLM - Aioga Actualités IA","description":"Les ingénieurs de Baseten se sont référés à l'article MetaInfer pour que Claude Code (Fable 5) construise automatiquement le moteur d'inférence VibeQwen pour Qwen-3.6-35B-A3B (NVFP...","url":"https://www.aioga.com/fr/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:33:40.319Z"},"de":{"title":"Baseten-Ingenieure im Praxistest: Von einem LLM generierte Inferenz-Engine bis zu 90 % schneller als vLLM","summary":"Baseten-Ingenieure orientierten sich an der MetaInfer-Arbeit und ließen Claude Code (Fable 5) die Inferenz-Engine VibeQwen für Qwen-3.6-35B-A3B (NVFP4, eine einzelne B200) automatisch entwickeln. Beim Single-Stream-Decoding ist sie 90 % schneller als vLLM 0.25.1, die Zeit bis zum ersten Token sank von 28 ms auf 12 ms, und bei 32 gleichzeitigen Anfragen liegt der Durchsatz um 71 % höher.","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Baseten-Ingenieure im Praxistest: Von einem LLM generierte Inferenz-Engine bis zu 90 % schneller als vLLM - Aioga KI-News","description":"Baseten-Ingenieure orientierten sich an der MetaInfer-Arbeit und ließen Claude Code (Fable 5) die Inferenz-Engine VibeQwen für Qwen-3.6-35B-A3B (NVFP4, eine einzelne B200) automati...","url":"https://www.aioga.com/de/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:33:49.737Z"},"pt-BR":{"title":"Engenheiros da Baseten testaram: o motor de raciocínio gerado pelo LLM é até 90% mais rápido que o vLLM","summary":"Os engenheiros da Baseten consultaram o artigo MetaInfer e fizeram com que o Claude Code (Fable 5) construísse automaticamente o motor de inferência VibeQwen para o Qwen-3.6-35B-A3B (NVFP4, único B200). A decodificação em fluxo único é 90% mais rápida que a vLLM 0.25.1, o tempo do primeiro token caiu de 28 ms para 12 ms e a taxa de transferência aumentou 71% com 32 concorrentes.","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Engenheiros da Baseten testaram: o motor de raciocínio gerado pelo LLM é até 90% mais rápido que o vLLM - Aioga Notícias de IA","description":"Os engenheiros da Baseten consultaram o artigo MetaInfer e fizeram com que o Claude Code (Fable 5) construísse automaticamente o motor de inferência VibeQwen para o Qwen-3.6-35B-A3...","url":"https://www.aioga.com/pt-BR/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:39:47.939Z"},"ru":{"title":"Инженеры Baseten провели тестирование: движок рассуждений, созданный LLM, работает до 90% быстрее, чем vLLM","summary":"Инженеры Baseten, ссылаясь на статью MetaInfer, позволили Claude Code (Fable 5) автоматически создать движок вывода VibeQwen для Qwen-3.6-35B-A3B (NVFP4, одна B200). Однопоточная декодировка стала на 90% быстрее, чем у vLLM 0.25.1, время первого токена сократилось с 28 мс до 12 мс, а при 32 параллельных потоках пропускная способность увеличилась на 71%.","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Инженеры Baseten провели тестирование: движок рассуждений, созданный LLM, работает до 90% быстрее, чем vLLM - Aioga Новости ИИ","description":"Инженеры Baseten, ссылаясь на статью MetaInfer, позволили Claude Code (Fable 5) автоматически создать движок вывода VibeQwen для Qwen-3.6-35B-A3B (NVFP4, одна B200). Однопоточная д...","url":"https://www.aioga.com/ru/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:39:23.277Z"},"ar":{"title":"اختبر مهندسو Baseten عمليًا: محرّك الاستدلال الذي أنشأه LLM أسرع من vLLM بنسبة تصل إلى 90%","summary":"استلهم مهندسو Baseten من ورقة MetaInfer البحثية، واستخدموا Claude Code (Fable 5) لبناء محرك الاستدلال VibeQwen تلقائيًا لنموذج Qwen-3.6-35B-A3B (NVFP4، على وحدة B200 واحدة). وهو أسرع بنسبة 90% من vLLM 0.25.1 في فك الترميز لتدفق واحد، ويخفض زمن الوصول إلى أول رمز من 28ms إلى 12ms، مع إنتاجية أعلى بنسبة 71% عند التزامن مع 32 طلبًا.","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"اختبر مهندسو Baseten عمليًا: محرّك الاستدلال الذي أنشأه LLM أسرع من vLLM بنسبة تصل إلى 90% - Aioga أخبار الذكاء الاصطناعي","description":"استلهم مهندسو Baseten من ورقة MetaInfer البحثية، واستخدموا Claude Code (Fable 5) لبناء محرك الاستدلال VibeQwen تلقائيًا لنموذج Qwen-3.6-35B-A3B (NVFP4، على وحدة B200 واحدة). وهو أس...","url":"https://www.aioga.com/ar/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:39:30.859Z"},"hi":{"title":"Baseten इंजीनियर के परीक्षण में: LLM द्वारा जनरेट किया गया推理 इंजन vLLM की तुलना में अधिकतम 90% तेज़ है","summary":"Baseten इंजीनियरों ने MetaInfer पेपर का संदर्भ लिया, और Claude Code (Fable 5) ने Qwen-3.6-35B-A3B (NVFP4, एकल B200) के लिए स्वतः VibeQwen इंजन संरचना तैयार की, एकल-स्ट्रीम डिकोडिंग vLLM 0.25.1 से 90% तेज है, पहला टोकन 28ms से घटकर 12ms हो गया, और 32 समवर्ती पर थ्रूपुट 71% अधिक है।","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Baseten इंजीनियर के परीक्षण में: LLM द्वारा जनरेट किया गया推理 इंजन vLLM की तुलना में अधिकतम 90% तेज़ है - Aioga AI समाचार","description":"Baseten इंजीनियरों ने MetaInfer पेपर का संदर्भ लिया, और Claude Code (Fable 5) ने Qwen-3.6-35B-A3B (NVFP4, एकल B200) के लिए स्वतः VibeQwen इंजन संरचना तैयार की, एकल-स्ट्रीम डिकोडिंग...","url":"https://www.aioga.com/hi/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:45:54.870Z"},"it":{"title":"Gli ingegneri di Baseten hanno testato: il motore di ragionamento generato da LLM è fino al 90% più veloce di vLLM","summary":"Gli ingegneri di Baseten hanno fatto riferimento al documento MetaInfer, facendo sì che Claude Code (Fable 5) costruisse automaticamente il motore di inferenza VibeQwen per Qwen-3.6-35B-A3B (NVFP4, singola B200), con la decodifica a flusso singolo più veloce del 90% rispetto a vLLM 0.25.1, il primo token scende da 28 ms a 12 ms e con 32 concorrenti la capacità di elaborazione aumenta del 71%.","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Gli ingegneri di Baseten hanno testato: il motore di ragionamento generato da LLM è fino al 90% più veloce di vLLM - Aioga Notizie IA","description":"Gli ingegneri di Baseten hanno fatto riferimento al documento MetaInfer, facendo sì che Claude Code (Fable 5) costruisse automaticamente il motore di inferenza VibeQwen per Qwen-3....","url":"https://www.aioga.com/it/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:45:47.327Z"},"nl":{"title":"Baseten-ingenieur test: door LLM gegenereerde reasoning-engine is tot 90% sneller dan vLLM","summary":"Baseten-ingenieurs refereerden aan het MetaInfer-paper en lieten Claude Code (Fable 5) automatisch de inferentie-engine VibeQwen bouwen voor Qwen-3.6-35B-A3B (NVFP4, enkele B200), wat enkelvoudige stroom decoding 90% sneller maakte dan vLLM 0.25.1, de eerste token-tijd verlaagde van 28ms naar 12ms, en bij 32 gelijktijdige gebruikers de doorvoer met 71% verhoogde.","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Baseten-ingenieur test: door LLM gegenereerde reasoning-engine is tot 90% sneller dan vLLM - Aioga AI-nieuws","description":"Baseten-ingenieurs refereerden aan het MetaInfer-paper en lieten Claude Code (Fable 5) automatisch de inferentie-engine VibeQwen bouwen voor Qwen-3.6-35B-A3B (NVFP4, enkele B200),...","url":"https://www.aioga.com/nl/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:45:47.322Z"},"tr":{"title":"Baseten mühendislerinin gerçek testleri: LLM tarafından oluşturulan çıkarım motoru vLLM’den en fazla %90 daha hızlı","summary":"Baseten mühendisi, MetaInfer makalesini referans alarak, Claude Code (Fable 5) ile Qwen-3.6-35B-A3B (NVFP4, tek B200) için otomatik olarak çıkarım motoru VibeQwen’i oluşturdu; tek akışla çözme hızı vLLM 0.25.1’den %90 daha hızlı, ilk token 28ms’den 12ms’ye indi, 32 eşzamanlı işlemde işlem hacmi %71 arttı.","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Baseten mühendislerinin gerçek testleri: LLM tarafından oluşturulan çıkarım motoru vLLM’den en fazla %90 daha hızlı - Aioga AI Haberleri","description":"Baseten mühendisi, MetaInfer makalesini referans alarak, Claude Code (Fable 5) ile Qwen-3.6-35B-A3B (NVFP4, tek B200) için otomatik olarak çıkarım motoru VibeQwen’i oluşturdu; tek...","url":"https://www.aioga.com/tr/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:51:43.474Z"},"vi":{"title":"Kỹ sư của Baseten thực nghiệm: động cơ suy luận do LLM tạo ra nhanh hơn vLLM đến 90%","summary":"Các kỹ sư của Baseten tham khảo bài báo MetaInfer, để Claude Code (Fable 5) tự động xây dựng engine suy diễn VibeQwen cho Qwen-3.6-35B-A3B (NVFP4, một thẻ B200), giải mã một luồng nhanh hơn 90% so với vLLM 0.25.1, token đầu tiên giảm từ 28ms xuống 12ms, và thông lượng tăng 71% khi đồng thời 32 luồng.","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Kỹ sư của Baseten thực nghiệm: động cơ suy luận do LLM tạo ra nhanh hơn vLLM đến 90% - Tin tức AI Aioga","description":"Các kỹ sư của Baseten tham khảo bài báo MetaInfer, để Claude Code (Fable 5) tự động xây dựng engine suy diễn VibeQwen cho Qwen-3.6-35B-A3B (NVFP4, một thẻ B200), giải mã một luồng...","url":"https://www.aioga.com/vi/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:51:49.930Z"},"id":{"title":"Insinyur Baseten menguji secara nyata: Mesin penalaran yang dibuat oleh LLM hingga 90% lebih cepat daripada vLLM","summary":"Para Qwen-3.6-35B-A3B (NVFP4, un solo B200), los ingenieros de Baseten tomaron como referencia el artículo de MetaInfer y usaron Claude Code (Fable 5) para construir automáticamente el motor de inferencia VibeQwen. La decodificación de un solo flujo es un 90 % más rápida que con vLLM 0.25.1; el tiempo hasta el primer token se redujo de 28 ms a 12 ms, y el rendimiento con 32 solicitudes concurrentes es un 71 % mayor.","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Insinyur Baseten menguji secara nyata: Mesin penalaran yang dibuat oleh LLM hingga 90% lebih cepat daripada vLLM - Berita AI Aioga","description":"Para Qwen-3.6-35B-A3B (NVFP4, un solo B200), los ingenieros de Baseten tomaron como referencia el artículo de MetaInfer y usaron Claude Code (Fable 5) para construir automáticament...","url":"https://www.aioga.com/id/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:51:40.083Z"},"th":{"title":"วิศวกรของ Baseten ทดสอบจริง: เครื่องยนต์เหตุผลที่สร้างโดย LLM เร็วกว่าของ vLLM สูงสุด 90%","summary":"วิศวกรของ Baseten อ้างอิงบทความ MetaInfer ให้ Claude Code (Fable 5) สร้างเอนจินการอนุมาน VibeQwen อัตโนมัติสำหรับ Qwen-3.6-35B-A3B (NVFP4, การ์ด B200 เดียว) การถอดรหัสแบบสตรีมเดียวเร็วกว่า vLLM 0.25.1 ถึง 90% เวลา token แรกลดจาก 28ms เหลือ 12ms เมื่อมีการประมวลผลพร้อมกัน 32 งาน อัตราการส่งผ่านเพิ่มขึ้น 71%","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"วิศวกรของ Baseten ทดสอบจริง: เครื่องยนต์เหตุผลที่สร้างโดย LLM เร็วกว่าของ vLLM สูงสุด 90% - ข่าว AI Aioga","description":"วิศวกรของ Baseten อ้างอิงบทความ MetaInfer ให้ Claude Code (Fable 5) สร้างเอนจินการอนุมาน VibeQwen อัตโนมัติสำหรับ Qwen-3.6-35B-A3B (NVFP4, การ์ด B200 เดียว) การถอดรหัสแบบสตรีมเดียว...","url":"https://www.aioga.com/th/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:57:45.413Z"},"pl":{"title":"Inżynierowie Baseten przeprowadzili testy: silnik wnioskowania generowany przez LLM jest do 90% szybszy niż vLLM","summary":"Inżynierowie Baseten odwołali się do artykułu MetaInfer, pozwalając Claude Code (Fable 5) automatycznie stworzyć silnik inferencyjny VibeQwen dla Qwen-3.6-35B-A3B (NVFP4, pojedyncza karta B200), gdzie dekodowanie jednoprądowe jest o 90% szybsze niż w vLLM 0.25.1, czas pierwszego tokena spada z 28 ms do 12 ms, a przy równoczesności 32 przepustowość wzrasta o 71%.","category":"行业动态","source":"Baseten 工程博客","aggregationSource":"Baseten 工程博客","pageTitle":"Inżynierowie Baseten przeprowadzili testy: silnik wnioskowania generowany przez LLM jest do 90% szybszy niż vLLM - Aioga Wiadomości AI","description":"Inżynierowie Baseten odwołali się do artykułu MetaInfer, pozwalając Claude Code (Fable 5) automatycznie stworzyć silnik inferencyjny VibeQwen dla Qwen-3.6-35B-A3B (NVFP4, pojedyncz...","url":"https://www.aioga.com/pl/news/uwm1ml1igd8k2u81g14uf8pc3/","contentTranslated":true,"translationStatus":"translated","translationRetryAt":"","translationError":"","sourceHash":"42e007f943f222d1","translatedAt":"2026-10-03T14:57:59.174Z"}},"evidenceTier":"verified-news","reviewStatus":"editorial-selected","indexable":true,"editorialCover":""}}