We're excited to partner with Meta Superintelligence Labs to bring Day-0 support for Muse Glimmer：https://ai.meta.com/ to SGLang, with dedicated optimizations tailored for high-performance inference of agentic workflows on local hardware.

Muse Glimmer is a 30B model, consisting of a 27.9B dense text decoder, a 1.9B ViT, and a GELU-based multimodal projector. The text decoder has 52 transformer layers. Each layer contains grouped-query attention with 32 query heads and two key-value heads, followed by a SwiGLU feed-forward network.

The decoder uses a hybrid attention pattern that interleaves three 2,048-token sliding-window layers with a full-sequence layer every fourth step for efficiency. Combining RoPE on the local windows with NoPE on the full-attention layers allows the model to extend its context length beyond its training limits.

Muse Glimmer can be deployed with SGLang across a wide range of hardware commonly used by developers building and running AI agents locally, including Apple Silicon devices (Mac mini and M-series MacBook Pro), NVIDIA GeForce RTX 5090 GPUs, and the NVIDIA DGX Spark. On NVIDIA SM120 platforms, SGLang leverages its optimized GEMM and FlashInfer backends for high-throughput inference, while Apple Silicon devices use the native MLX backend to deliver high-performance local serving.

To achieve low-latency inference, use SGLang's implementation of DFlash:

Muse Glimmer is compatible with SGLang's many native optimizations, including the low-overhead scheduler, the RadixAttention prefix cache, and breakable CUDA graphs. We have also brought several of these optimizations, including prefix caching, to the SGLang MLX backend, enabling competitive performance on Apple Silicon for agentic workloads.

We provide Muse Glimmer checkpoints in several formats to meet the needs of users with different hardware, fidelity, and system performance requirements.

Developers requiring maximum model fidelity can run the native BF16 checkpoint on a single H100 GPU. The SM120 path includes a ~19.5 GB mixed NVFP4+MXFP8 quantization recipe. An 18 GB NVFP4 checkpoint paired with a 5 GB BF16 DFlash speculator fits comfortably on a single RTX 5090, enabling high-performance deployment on NVIDIA GeForce RTX 5090 accelerators and DGX Spark.

We also provide two GGUF checkpoints. The smaller of the two is in Q4KM format with a group size of 128. This checkpoint allows for faster inference speeds and deployment on hardware with tighter memory constraints. Q4K-Dynamic delivers better model quality. For developers working on Apple Silicon devices, we provide the previously mentioned GGUF checkpoints in MLX format.

We measured Muse Glimmer with SGLang across seven platforms, sweeping batch sizes 1 through 8. Reported below are batch-1 interactivity (tok/s/user) and batch-8 aggregate output throughput (tok/s).

DFlash raises batch-1 interactivity by 1.9–4.3x depending on platform and precision: 3.0x or better on every configuration except the GGUF path on the RTX 5090, which gains 1.9x. The batch-8 gain is smaller and more variable, ranging from 1.4x to 4.2x. Speculative decoding is not available on the MLX backend.

We thank the teams at Meta Superintelligence Labs and the SGLang community for their collaboration in bringing Day-0 support for Muse Glimmer to SGLang.