SGLang speeds up MiniMax-H3 1.95x with zero quality loss
LMSYS benchmarked MiniMax-H3 on 8x H200 GPUs: SGLang's lossless path beats Diffusers by up to 1.95x, while steeper tiers trade SSIM fidelity for up to 6x speed.
inference optimization News and Analysis
5 verified stories covering inference optimization, product updates and industry developments.
Ant's Ling-3.0-flash: 2.1x Faster Single-Request Decoding on 4 GPUs
SGLang and Ant Ling Infra cut single-request TPOT for Ling-3.0-flash from 3.33ms to 0.78ms on 4 Blackwell GPUs by removing host-side stalls.
Trillion-Parameter Model Restart Cut From 8.8 Minutes to 32 Seconds
Ant Group and SGLang keep quantized weights resident in GPU memory, cutting Ling-2.6-1T restart time from 8.8 minutes to about half a minute.
H20 Runs DeepSeek V4-Pro at 70% of B300 Speed
LMSYS's DeepSeek-V4-Pro benchmarks show Nvidia's China-market H20 chip closing the decode-speed gap with the flagship B300 to just 1.42x.
NVIDIA fuses diffusion and autoregression in new Nemotron model
NVIDIA's Nemotron-Labs-Diffusion introduces a tri-mode decoding approach that boosts throughput up to 6× over Qwen3-8B while maintaining accuracy.