SGLang 为 Muse Glimmer 提供 Day-0 支持，针对本地智能体工作流优化推理

我们很高兴与Meta Superintelligence Labs合作，将Muse Glimmer：https：//ai.meta.com/的Day-0支持带到SGLang，并针对本地硬件上高性能推断代理工作流进行专门优化。

Muse Glimmer 是一个 30B 型号，由一个 27.9B 密集文本解码器、一个 1.9B 的 ViT 和一个基于 GELU 的多模态投影仪组成。文本解码器有52个变换器层。每层包含分组查询注意力，包含32个查询头和两个键值头，随后是SwiGLU前馈网络。

解码器采用混合注意力模式，将三层2048个令牌的滑动窗口层与每四步一层全序列层交错交错，以提高效率。将局部窗口的RoPE与全注意力层的NoPE结合，使模型能够将上下文长度延长至训练极限之外。

Muse Glimmer 可以与 SGLang 一起部署在开发者常用的多种硬件上，用于本地构建和运行 AI 代理，包括苹果硅芯片设备（Mac mini 和 M 系列 MacBook Pro）、NVIDIA GeForce RTX 5090 GPU 以及 NVIDIA DGX Spark。在NVIDIA SM120平台上，SGLang利用其优化后的GEMM和FlashInfer后端实现高通量推断，而苹果硅芯片则使用原生MLX后端实现高性能本地服务。

为了实现低延迟推断，可以使用SGLang的DFlash实现：

Muse Glimmer 兼容 SGLang 的众多原生优化，包括低开销调度器、RadixAttention 前缀缓存和可破坏的 CUDA 图。我们还将包括前缀缓存在内的多项优化引入了SGLang MLX后台，使得在Apple Silicon上实现代理工作负载的竞争性能。

我们提供多种格式的Muse Glimmer检查点，以满足不同硬件、画质和系统性能需求的用户需求。

需要最大模型保真度的开发者可以在单个 H100 GPU 上运行原生 BF16 检查点。SM120路径包含一个~19.5 GB的混合NVFP4+MXFP8量化配方。18 GB NVFP4 检查点与 5 GB BF16 DFlash Speculator 可舒适地安装在单个 RTX 5090 上，支持在 NVIDIA GeForce RTX 5090 加速器和 DGX Spark 上的高性能部署。

我们还提供两个 GGUF 检查点。其中较小的一个是 Q4KM 格式，组大小为 128。此检查点允许更快的推理速度，并可在内存受限的硬件上部署。Q4K-Dynamic 提供更好的模型质量。对于在 Apple Silicon 设备上开发的开发者，我们提供前面提到的 GGUF 检查点的 MLX 格式。

我们在七个平台上使用 SGLang 测量了 Muse Glimmer，并测试了批量大小 1 到 8。下面报告的是批量为 1 的交互性（tok/s/user）和批量为 8 的整体输出吞吐量（tok/s）。

DFlash 将批量为 1 的交互性提高了 1.9–4.3 倍，具体取决于平台和精度：除 RTX 5090 的 GGUF 路径外，每种配置都达到了 3.0 倍或更高，RTX 5090 的 GGUF 路径获得了 1.9 倍提升。批量为 8 的增益较小且更具变化性，范围为 1.4 倍到 4.2 倍。MLX 后端不支持推测解码。

我们感谢 Meta 超智能实验室团队和 SGLang 社区在 Muse Glimmer 的 Day-0 支持方面的合作。

We're excited to partner with Meta Superintelligence Labs to bring Day-0 support for Muse Glimmer：https://ai.meta.com/ to SGLang, with dedicated optimizations tailored for high-performance inference of agentic workflows on local hardware.

Muse Glimmer is a 30B model, consisting of a 27.9B dense text decoder, a 1.9B ViT, and a GELU-based multimodal projector. The text decoder has 52 transformer layers. Each layer contains grouped-query attention with 32 query heads and two key-value heads, followed by a SwiGLU feed-forward network.

The decoder uses a hybrid attention pattern that interleaves three 2,048-token sliding-window layers with a full-sequence layer every fourth step for efficiency. Combining RoPE on the local windows with NoPE on the full-attention layers allows the model to extend its context length beyond its training limits.

Muse Glimmer can be deployed with SGLang across a wide range of hardware commonly used by developers building and running AI agents locally, including Apple Silicon devices (Mac mini and M-series MacBook Pro), NVIDIA GeForce RTX 5090 GPUs, and the NVIDIA DGX Spark. On NVIDIA SM120 platforms, SGLang leverages its optimized GEMM and FlashInfer backends for high-throughput inference, while Apple Silicon devices use the native MLX backend to deliver high-performance local serving.

To achieve low-latency inference, use SGLang's implementation of DFlash:

Muse Glimmer is compatible with SGLang's many native optimizations, including the low-overhead scheduler, the RadixAttention prefix cache, and breakable CUDA graphs. We have also brought several of these optimizations, including prefix caching, to the SGLang MLX backend, enabling competitive performance on Apple Silicon for agentic workloads.

We provide Muse Glimmer checkpoints in several formats to meet the needs of users with different hardware, fidelity, and system performance requirements.

Developers requiring maximum model fidelity can run the native BF16 checkpoint on a single H100 GPU. The SM120 path includes a ~19.5 GB mixed NVFP4+MXFP8 quantization recipe. An 18 GB NVFP4 checkpoint paired with a 5 GB BF16 DFlash speculator fits comfortably on a single RTX 5090, enabling high-performance deployment on NVIDIA GeForce RTX 5090 accelerators and DGX Spark.

We also provide two GGUF checkpoints. The smaller of the two is in Q4KM format with a group size of 128. This checkpoint allows for faster inference speeds and deployment on hardware with tighter memory constraints. Q4K-Dynamic delivers better model quality. For developers working on Apple Silicon devices, we provide the previously mentioned GGUF checkpoints in MLX format.

We measured Muse Glimmer with SGLang across seven platforms, sweeping batch sizes 1 through 8. Reported below are batch-1 interactivity (tok/s/user) and batch-8 aggregate output throughput (tok/s).

DFlash raises batch-1 interactivity by 1.9–4.3x depending on platform and precision: 3.0x or better on every configuration except the GGUF path on the RTX 5090, which gains 1.9x. The batch-8 gain is smaller and more variable, ranging from 1.4x to 4.2x. Speculative decoding is not available on the MLX backend.

We thank the teams at Meta Superintelligence Labs and the SGLang community for their collaboration in bringing Day-0 support for Muse Glimmer to SGLang.