PrismML 发布 Bonsai 27B：27B 参数开源推理模型，可装入 iPhone

PrismML 发布了 Bonsai 27B，这是一款 AI 模型，据称能安装在智能手机上，同时不牺牲推理能力和代理能力。据报道，苹果正在测试这项技术。

PrismML 是一家由加州理工学院研究团队创立的公司，发布了 Bonsai 27B，这是一个拥有 270 亿参数的 AI 模型，设计用于直接在 iPhone 上运行。盆景27B基于阿里巴巴的Qwen3.6-27B：https：//the-decoder.com/qwen3-6-27b-beats-much-larger-predecessor-on-most-coding-benchmarks/。PrismML 表示支持多步推理、工具使用、图像理解和基于代理的任务。

PrismML认为，现代AI应用越来越需要强大的模型来本地运行。代理可能连续进行数百次模型调用，每次都承载上下文，产生结构化输出，并传递到下一步。在云端，每个令牌的成本会累积，每次通话都会增加网络延迟，中间结果、工具调用以及屏幕内容或文档等私有数据都会从设备中流出。

根据CNBC报道：https：//www.cnbc.com/2026/07/14/apple-prismml-ai-compression-iphone.html，PrismML已经在与苹果洽谈Bonsai背后的压缩技术。PrismML 首席执行官 Babak Hassibi 确认，苹果及其他公司正在测试这些型号的速度、功耗和性能。谈判“还很早”，但“进展顺利”。

这种尺寸的型号通常占用大约54GB的存储空间。即使采用标准压缩，仍然需要大约18 GB的内存。PrismML 提供两个更小版本：注重质量的版本约占 5.9 GB，面向笔记本电脑，尽管目前发货的软件包可能更大，具体取决于运行时间。白皮书列出llama.cpp版约7.2GB，MLX版约8.49GB。

较小版本约为3.9 GB，足够容纳iPhone 17 Pro Max有限的存储空间。根据PrismML的数据，配备12GB内存的iPhone实际上只为单个应用提供了大约6GB内存，分配在机型和缓存之间。

PrismML 并没有将每个神经网络权重存储为 16 位，而是只使用一位或略低于两位。在最激进的版本中，每个权重只有两个状态。在稍大一些的版本中，则有三个状态。这种方法在整个语言模型中应用。作为常见标签问题的一个例子，PrismML 指出了白皮书中比较的 Qwen3.6-27B-IQ2_XXS 构建版本，尽管标为“2-bit”，但每个权重的平均位数为 2.8 位。

在 PrismML 自己对 15 个基准的评估中，较大的版本保持了原始模型 95% 的性能。较小的版本保持了 90%。据 PrismML 称，数学和编码几乎“未受影响”。

较大幅度的性能下降出现在更激进的压缩中，尤其是在图像理解、指令执行和基于代理的工具使用方面。一个常规压缩的 Qwen3.6-27B 模型为 9.4 GB，只得 72.7 分，而较小的 Bonsai 版本为 3.9 GB，得 76.1 分。

根据白皮书，较小的版本在 iPhone 17 Pro Max 上每秒可以生成约 11 个 token。一次电池测试显示每 1% 的电量大约可以生成 672 个 token，外推到满电可生成约 67,000 个 token。芯片在运行略超过五分钟后略微降频。

模型权重可在 Apache 2.0 许可证下使用。Bonsai 27B 可通过 Apple 的 MLX 框架在 Apple 设备上运行：https://github.com/PrismML-Eng/Bonsai-demo/tree/main，并可运行在 NVIDIA GPU 上。PrismML 还提供有限时间的免费开发者预览 API：https://www.together.ai/models/prism-ml-ternary-bonsai-27b，以及 HuggingFace 上的实时演示：https://huggingface.co/spaces/webml-community/bonsai-webgpu-kernels。

该公司在 Khosla Ventures、Cerberus 和 Google 的支持下成立，三星继续支持其发展。PrismML 计划将其压缩技术应用到 Google 的 Gemma 模型系列：https://the-decoder.com/google-deepminds-gemma-4-12b-squeezes-multimodal-ai-onto-a-laptop-with-just-16-gb-of-ram/，其中的小版本已经可以在智能手机上运行：https://the-decoder.com/googles-gemma-4-puts-free-agentic-ai-on-your-phone-and-no-data-ever-leaves-the-device/。

获得许可的压缩技术对苹果也很重要，因为到目前为止，其自家的模型在基准测试中落后于竞争对手：https://the-decoder.com/emergency-meetings-and-failed-billion-dollar-talks-reveal-the-chaos-behind-apples-pivot-to-google-gemini/。在 2026 年 WWDC 上：https://the-decoder.com/apple-intelligence-gets-a-second-go-with-the-help-of-google-and-nvidia/，苹果发布了基于谷歌使用 Gemini 技术开发的基础模型的改进版 Siri。最强大的设备端模型已经需要至少 12 GB 内存的 iPhone，而复杂查询则在苹果云中的 Nvidia GPU 上运行。

保持对人工智能的关注。清晰、有用，无冗余。

关注 The Decoder，获取 AI 新闻、背景故事及专家分析。

The Decoder：https://the-decoder.com/

PrismML has released Bonsai 27B, an AI model that it says fits on a smartphone without sacrificing reasoning or agent capabilities. Apple is reportedly testing the technology.

PrismML, a company founded by a team of Caltech researchers, has released Bonsai 27B, a 27-billion-parameter AI model designed to run directly on an iPhone. Bonsai 27B is based on Alibaba's Qwen3.6-27B：https://the-decoder.com/qwen3-6-27b-beats-much-larger-predecessor-on-most-coding-benchmarks/. PrismML says it supports multistep reasoning, tool use, image understanding, and agent-based tasks.

PrismML argues that modern AI apps increasingly need powerful models to run locally. An agent may make hundreds of model calls in sequence, each carrying context, producing structured output, and feeding into the next step. In the cloud, per-token costs pile up, every call adds network latency, and intermediate results, tool calls, and private data such as screen content or documents all leave the device.

According to a CNBC report：https://www.cnbc.com/2026/07/14/apple-prismml-ai-compression-iphone.html, PrismML is already in talks with Apple about the compression technology behind Bonsai. PrismML CEO Babak Hassibi confirmed that Apple and other companies are testing the models for speed, power draw, and performance. The talks are "very early," but "things are progressing nicely."

A model this size typically takes up about 54 GB of storage. Even with standard compression, it still needs around 18 GB. PrismML offers two much smaller versions: The quality-focused variant takes up about 5.9 GB and is meant for laptops, though the packages currently shipping may be larger depending on the runtime. The white paper lists about 7.2 GB for the llama.cpp version and 8.49 GB for the MLX version.

The smaller variant comes in at about 3.9 GB, small enough to fit within the limited storage of an iPhone 17 Pro Max. According to PrismML, an iPhone with 12 GB of RAM actually makes only about 6 GB available to a single app, split between the model and the cache.

Instead of storing each neural network weight as 16 bits, PrismML uses only one or just under two bits. In the most aggressive variant, each weight has only two states. In the slightly larger one, three. This approach is applied across the entire language model. As an example of common labeling issues, PrismML points to the Qwen3.6-27B-IQ2_XXS build compared in the white paper, which averages 2.8 bits per weight despite its "2-bit" label.

In PrismML's own evaluation across 15 benchmarks, the larger variant keeps 95 percent of the original model's performance. The smaller one keeps 90 percent. Math and coding stayed "virtually unaffected," according to PrismML.

The bigger drops showed up with the more aggressive compression, especially in image understanding, instruction following, and agent-based tool use. A conventionally compressed Qwen3.6-27B model at 9.4 GB scores only 72.7 points, while the smaller Bonsai variant at 3.9 GB scores 76.1.

According to the white paper, the smaller variant generates about 11 tokens per second on an iPhone 17 Pro Max. A battery test yielded roughly 672 generated tokens per percentage point of battery charge, which extrapolates to about 67,000 tokens on a full charge. The chip throttled slightly after a little over five minutes.

The model weights are available under the Apache 2.0 license. Bonsai 27B runs on Apple devices：https://github.com/PrismML-Eng/Bonsai-demo/tree/main via Apple's MLX framework and on NVIDIA GPUs. PrismML also provides a time-limited, free Developer Preview API：https://www.together.ai/models/prism-ml-ternary-bonsai-27b and a live demo on HuggingFace：https://huggingface.co/spaces/webml-community/bonsai-webgpu-kernels.

The company was founded with backing from Khosla Ventures, Cerberus, and Google, and Samsung continues to support it. PrismML plans to apply its compression tech to Google's Gemma model：https://the-decoder.com/google-deepminds-gemma-4-12b-squeezes-multimodal-ai-onto-a-laptop-with-just-16-gb-of-ram/ series next, smaller versions of which already run on smartphones：https://the-decoder.com/googles-gemma-4-puts-free-agentic-ai-on-your-phone-and-no-data-ever-leaves-the-device/.

A licensed compression technology would also matter to Apple, whose own models have trailed the competition in benchmarks so far：https://the-decoder.com/emergency-meetings-and-failed-billion-dollar-talks-reveal-the-chaos-behind-apples-pivot-to-google-gemini/. At WWDC 2026：https://the-decoder.com/apple-intelligence-gets-a-second-go-with-the-help-of-google-and-nvidia/, Apple unveiled a revamped Siri built on foundation models developed with Google using Gemini technology. The most powerful on-device model already requires an iPhone with at least 12 GB of RAM, and complex queries run on Nvidia GPUs in Apple's cloud.

Stay in the loop on AI. Clear, useful, no fluff.

Follow The Decoder for AI news, background stories and expert analyses.

The Decoder：https://the-decoder.com/