PrismML 发布 Bonsai 27B：Qwen3.6-27B 的低比特版本，可在笔记本和手机上运行

PrismML 发布 Bonsai 27B，这是 Qwen3.6-27B 的低比特表示，非全新预训练模型。

三值版使用 {−1， 0， +1} 权重，每权重 1.71 bits，大小 5.9GB；

1-bit 版使用 {−1， +1} 权重，每权重 1.125 bits，大小 3.9GB。

两者均支持多模态，上下文窗口 262K tokens。

三值版保留 FP16 基线 94.6% 的推理性能，1-bit 版保留 89.5%。

1-bit 版是首个可装入手机的 27B 级模型。

PrismML 还提供 DSpark 推测解码 drafter，在 H100 上实现 1.37 倍加速。

两者均以 Apache 2.0 许可证发布，支持 llama.cpp 和 MLX。

PrismML just released Bonsai 27B：https://prismml.com/news/bonsai-27b . It is a low-bit representation of Qwen3.6-27B, not a new pretrain. The architecture is unchanged.

Two variants ship under Apache 2.0. Ternary Bonsai 27B uses {−1, 0, +1} weights at a true 1.71 bits per weight. Its ideal size is 5.9GB. 1-bit Bonsai 27B uses binary {−1, +1} weights at 1.125 bits per weight, for 3.9GB.

Both are multimodal. The split is ~24.8B language weights, a 0.46B vision tower, and 2.5B in embeddings and the LM head. The vision tower is held separately at 4-bit (HQQ). Context is 262K tokens, kept practical because ~75% of Qwen3.6-27B attention is linear.

That architecture shapes the compression method below.

Each weight is a code, with one shared FP16 scale per group of 128. The effective weight is w_i = s_g · t_i .

A ternary value carries log2(3) ≈ 1.585 bits. One FP16 scale per 128 weights adds 16/128 , giving ≈1.71 bits per weight. That is a ~9.4× reduction against FP16. Binary costs 1 + 16/128 = 1.125 bits, a ~14.2× reduction.

The representation runs end to end across the matrix-heavy components. Those are embeddings, attention projections, MLP projections, and the LM head. Only a negligible tail of normalization and scale parameters stays higher precision.

Measured as a true average, the Qwen3.6-27B “4-bit” build (Q4_K_XL) is 5.2 bits per weight. The “2-bit” build (IQ2_XXS) is 2.8. Bonsai also departs from BitNet, which avoids collapse only by pretraining from scratch.

The obvious question is what compression costs in accuracy.

PrismML evaluated 15 benchmarks in thinking mode, using EvalScope with vLLM on H100 GPUs. Ternary Bonsai 27B retains 94.6% of the FP16 baseline, and 1-bit Bonsai 27B retains 89.5%.

Conventional sub-4-bit builds fail differently. IQ2_XXS falls to 57.5 on AIME26 and 56.4 on LiveCodeBench. It still scores 88.93 on MMLU-Redux, so short-form benchmarks mask the collapse. Gemma-4-31B Q2_K_XL repeats that pattern on a second base model.

Scores alone, however, do not explain the release. Memory does.

Fitting a phone is stricter than storage numbers suggest. iOS limits a single app to roughly half of physical memory. A 12GB iPhone therefore exposes about 6GB.

The KV cache is the second budget. Only 16 of 64 layers carry a growing full-attention cache, so FP16 costs ≈64 KiB/token. A 262K window costs ≈17.2GB, and a 4-bit KV cache cuts that to ≈4.3GB.

Tolerance is measured. Against its own FP16-KV baseline, Ternary Bonsai shows 0.0011 nats of output forward-KL on MATH-500. Q4_K_XL shows 0.0146.

Peaks follow. At 100K tokens with an FP16 cache, 1-bit peaks at 11.6GB and ternary at 14.7GB. The derived Q4_K_XL row needs ≈25.6GB.

Once a model fits, throughput is the next question.

Generation is memory-bandwidth-bound, so fewer bytes per step means more tokens per second. Prefill is compute-bound and gains less.

PrismML also ships a DSpark drafter trained against the Bonsai 27B target. On an H100 at draft depth k=4, the binary target reaches accepted length τ=3.6. That is 143.8 tok/s, a 1.37× speedup. Verification is lossless, so output stays distribution-identical. On Apple Silicon the drafter is off by default at batch size 1.

Ternary 27B is the demo repo default. Start the server, or generate directly:

Tool calling uses the standard OpenAI-style tools array:

The call returns in choices[0].message.tool_calls . Thinking mode is on by default; thinking_budget_tokens toggles it per request.

That maps onto four deployment patterns.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us ：https://forms.gle/wbash1wF6efRj8G58

Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.