SGLang and Miles announced day-0 serving and training support for Muse Spark Open on October 15, 2026. The open-weight multimodal mixture-of-experts model uses an 816B-parameter text transformer backbone, with approximately 42 billion active parameters per token, accepts text, image, video and audio inputs, and supports a 1M-token context window. Its checkpoints are available in NVFP4 and BF16. Target platforms include NVIDIA Blackwell GPUs (GB300, B300, GB200 and B200), NVIDIA H200, AMD MI350X and MI355X, and Intel Xeon 6 with P-cores.

In an OpenHands multi-turn agentic coding replay using an NVFP4 checkpoint on 8 NVIDIA B300 GPUs, SGLang reported 14.8K total tokens per second per GPU at 543 tokens per second per user (p50) with concurrency 1. At concurrency 128, it reported 80.3K total tokens per second per GPU at 61 tokens per second per user (p50). SGLang also supports sequence parallelism with NVFP4 MegaMoE, multimodal preprocessing optimizations and DFlash V2 speculative decoding.

For the model’s 1M-token context, grouped-query attention shares key/value heads among query-head groups, while sliding-window layers handle a bounded recent context and periodic global layers retain access to the full sequence. Images and video frames use a shared NaViT visual encoder, while audio uses a separate encoder; their outputs join text embeddings in the decoder. The mixture-of-experts block uses pooler and unpooler projections to compress and restore feature width, and applies per-expert RMSNorm before outputs are weighted and combined.

In tests on four GB300 GPUs, SGLang compared three configurations using an NVFP4 target model, a DFlash V2 draft model, TP4/EP4, FP8 KV cache and 16K prefill chunks. Each test sent eight groups of eight requests, with nominal 80K input tokens and 20 output tokens per request; requests within each group shared a 73,360-token generated prefix, while groups used independently generated prefixes. Across seven concurrency levels, sequence parallelism plus MegaMoE reduced p90 time to first token by 28.3–41.9% versus TP4. At concurrency 64, that measure fell from 29.03 seconds to 19.47 seconds, a 32.9% reduction.

SGLang also described image and video preprocessing on CPUs using Rust or Pillow resizing, optimized attention backends for vision encoding, and reuse of cached results when images recur across conversation turns. Its separate encoder, prefill and decode workers pass encoded media features and KV cache between stages. Other serving options include prefill/decode disaggregation, HiCache’s GPU/host KV-cache hierarchy and breakable CUDA graphs.

DFlash V2 uses a draft model to propose blocks of future tokens for Muse Spark Open to verify. Its dynamic convolutions mix information across neighboring draft positions, and a candidate selector accounts for dependencies among proposed tokens. SGLang integrates drafting and selection into one CUDA graph and supports overlap scheduling. For sampled proposals, block verification evaluates draft prefixes jointly using target and draft probabilities, accepting the longest valid prefix and sampling a correction token after partial acceptance or a target-model bonus token after full acceptance.

The team also detailed kernel changes including fused normalization and residual operations, reduced memory passes, CUDA-based routing, sharded pooler and unpooler projections, and overlapping independent mixture-of-experts work. DeepGEMM’s MegaMoE combines dispatch, expert computations, activation and combine in a persistent kernel. Muse Spark Open support requires NVFP4 weights and activations in 60 layers, BF16 experts in two layers, and per-expert RMS normalization before route weighting. The article’s remaining sections list AMD and Intel platform support, reinforcement learning, a trainable backend, token-consistent multi-turn rollouts, training results, acknowledgements and reproduction instructions.

SGLang and Miles announced day-0 serving and training support for Muse Spark Open on October 15, 2026. The open-weight multimodal mixture-of-experts model uses an 816B-parameter text transformer backbone, with approximately 42 billion active parameters per token, accepts text, image, video and audio inputs, and supports a 1M-token context window. Its checkpoints are available in NVFP4 and BF16. Target platforms include NVIDIA Blackwell GPUs (GB300, B300, GB200 and B200), NVIDIA H200, AMD MI350X and MI355X, and Intel Xeon 6 with P-cores.

In an OpenHands multi-turn agentic coding replay using an NVFP4 checkpoint on 8 NVIDIA B300 GPUs, SGLang reported 14.8K total tokens per second per GPU at 543 tokens per second per user (p50) with concurrency 1. At concurrency 128, it reported 80.3K total tokens per second per GPU at 61 tokens per second per user (p50). SGLang also supports sequence parallelism with NVFP4 MegaMoE, multimodal preprocessing optimizations and DFlash V2 speculative decoding.

For the model’s 1M-token context, grouped-query attention shares key/value heads among query-head groups, while sliding-window layers handle a bounded recent context and periodic global layers retain access to the full sequence. Images and video frames use a shared NaViT visual encoder, while audio uses a separate encoder; their outputs join text embeddings in the decoder. The mixture-of-experts block uses pooler and unpooler projections to compress and restore feature width, and applies per-expert RMSNorm before outputs are weighted and combined.

In tests on four GB300 GPUs, SGLang compared three configurations using an NVFP4 target model, a DFlash V2 draft model, TP4/EP4, FP8 KV cache and 16K prefill chunks. Each test sent eight groups of eight requests, with nominal 80K input tokens and 20 output tokens per request; requests within each group shared a 73,360-token generated prefix, while groups used independently generated prefixes. Across seven concurrency levels, sequence parallelism plus MegaMoE reduced p90 time to first token by 28.3–41.9% versus TP4. At concurrency 64, that measure fell from 29.03 seconds to 19.47 seconds, a 32.9% reduction.

SGLang also described image and video preprocessing on CPUs using Rust or Pillow resizing, optimized attention backends for vision encoding, and reuse of cached results when images recur across conversation turns. Its separate encoder, prefill and decode workers pass encoded media features and KV cache between stages. Other serving options include prefill/decode disaggregation, HiCache’s GPU/host KV-cache hierarchy and breakable CUDA graphs.

DFlash V2 uses a draft model to propose blocks of future tokens for Muse Spark Open to verify. Its dynamic convolutions mix information across neighboring draft positions, and a candidate selector accounts for dependencies among proposed tokens. SGLang integrates drafting and selection into one CUDA graph and supports overlap scheduling. For sampled proposals, block verification evaluates draft prefixes jointly using target and draft probabilities, accepting the longest valid prefix and sampling a correction token after partial acceptance or a target-model bonus token after full acceptance.

The team also detailed kernel changes including fused normalization and residual operations, reduced memory passes, CUDA-based routing, sharded pooler and unpooler projections, and overlapping independent mixture-of-experts work. DeepGEMM’s MegaMoE combines dispatch, expert computations, activation and combine in a persistent kernel. Muse Spark Open support requires NVFP4 weights and activations in 60 layers, BF16 experts in two layers, and per-expert RMS normalization before route weighting. The article’s remaining sections list AMD and Intel platform support, reinforcement learning, a trainable backend, token-consistent multi-turn rollouts, training results, acknowledgements and reproduction instructions.