SGLang and Miles announced day-0 support for Inkling, a 975B-parameter multimodal model with a 1M token context window. Inkling, developed as a frontier model, introduces a unique architecture featuring short convolution, relative positional embedding, and a shared-expert sink in its Mixture-of-Experts (MoE) design. SGLang provides performance optimizations for this architecture on Nvidia Blackwell GPUs, delivering up to 71.7k tok/s input throughput and 171.0 tok/s per-user decode speed. These gains are realized through fused kernels for ShortConv (a causal convolution over token dimensions), specialized attention kernels for relative bias, and linearized weight layouts for the model's shared-expert MoE.

The SGLang optimizations address three specific architectural departures from standard decoder-only models. First, the ShortConv implementation utilizes fused kernels to combine K/V convolution, normalization, and KV-cache writes, achieving a 2.08–3.60× speedup over unfused chains. Second, relative logits are handled through a sheared-bias kernel within a modified FlashAttention-4 integration, allowing bias to be added via tile loads rather than per-score indexing. Finally, the shared-expert sink MoE is optimized by linearizing weight layouts, which converts expert-batched operations into two dense matrix multiplications (GEMMs). This fusion improves input throughput by 5.8–11.1% on B200 W4A16 serving configurations.

The Miles platform implements Inkling within a customized Megatron backend, supporting full-parameter and LoRA RL across text and vision-language tasks. Miles ensures consistency between training and inference through customized kernels and routing replay mechanisms. Additionally, speculative decoding is supported using DFlash, a draft model trained by Modal specifically for Inkling. The software stack includes broad support for distributed training strategies like data, pipeline, and tensor parallelism (DP/PP/TP/SP/EP/CP), enabling efficient reinforcement learning deployments for the 975B-parameter architecture.

SGLang and Miles announced day-0 support for Inkling, a 975B-parameter multimodal model with a 1M token context window. Inkling, developed as a frontier model, introduces a unique architecture featuring short convolution, relative positional embedding, and a shared-expert sink in its Mixture-of-Experts (MoE) design. SGLang provides performance optimizations for this architecture on Nvidia Blackwell GPUs, delivering up to 71.7k tok/s input throughput and 171.0 tok/s per-user decode speed. These gains are realized through fused kernels for ShortConv (a causal convolution over token dimensions), specialized attention kernels for relative bias, and linearized weight layouts for the model's shared-expert MoE.

The SGLang optimizations address three specific architectural departures from standard decoder-only models. First, the ShortConv implementation utilizes fused kernels to combine K/V convolution, normalization, and KV-cache writes, achieving a 2.08–3.60× speedup over unfused chains. Second, relative logits are handled through a sheared-bias kernel within a modified FlashAttention-4 integration, allowing bias to be added via tile loads rather than per-score indexing. Finally, the shared-expert sink MoE is optimized by linearizing weight layouts, which converts expert-batched operations into two dense matrix multiplications (GEMMs). This fusion improves input throughput by 5.8–11.1% on B200 W4A16 serving configurations.

The Miles platform implements Inkling within a customized Megatron backend, supporting full-parameter and LoRA RL across text and vision-language tasks. Miles ensures consistency between training and inference through customized kernels and routing replay mechanisms. Additionally, speculative decoding is supported using DFlash, a draft model trained by Modal specifically for Inkling. The software stack includes broad support for distributed training strategies like data, pipeline, and tensor parallelism (DP/PP/TP/SP/EP/CP), enabling efficient reinforcement learning deployments for the 975B-parameter architecture.