Today'sInfra
SGLang Reports DeepInfer Benchmark Results
- SGLang reports prefill throughput up to 1.04x DeepInfer on 4x GB300 and 4x B300
- On 64x GB300, SGLang decode steps were 1.1% to 7.4% shorter at 16 to 256 requests per GPU
- SGLang integrates DeepSeek kernels and adds KV caching, prefill/decode separation and replica load balancing
LMSYSRead more · 2 min →