SGLang Info
Overview of SGLang, CI infrastructure, and Docker images
Stable Docker Image
Official SGLang release images for production use.
lmsysorg/sglang
Docker Hub
docker pull lmsysorg/sglang:latestCUDA 12 / CUDA 13 variants are
:dev-cu12 and :dev-cu13;
the released version is on the
releases page.
ROCm Nightly Image
AMD ROCm nightly builds for MI300X/MI325X/MI355X development — the same images AMD CI runs on.
rocm/sgl-dev
Docker Hub
:latest tag. Every build is dated:
<version>-<rocm>-<gpu>-<YYYYMMDD>, eight a night —
rocm720 and rocm724 × mi30x and
mi35x, rocm10 × mi30x,
mi35x and mi45x, plus rocm101 ×
mi45x.
docker pull rocm/sgl-dev:v0.5.17-rocm724-mi35x-20260820What is SGLang?
SGLang is a high-performance serving framework for large language models and multimodal models, from a single GPU up to large distributed clusters. It is hosted by LMSYS and runs on more than 400,000 GPUs worldwide.
Start with the install guide and the AMD GPU platform page; for a worked introduction on AMD Instinct hardware see the ROCm blog post: SGLang on AMD Instinct GPUs.
Key Capabilities
- RadixAttention prefix caching, zero-overhead CPU scheduler, continuous batching, chunked prefill
- Prefill-decode disaggregation and speculative decoding (MTP)
- Tensor, pipeline, expert and data parallelism; multi-LoRA batching
- FP4/FP8/INT4/AWQ/GPTQ quantization; MXFP4 on MI355X
- LLMs, MoE, VLMs, embedding, reward and diffusion models
- NVIDIA GPUs, AMD Instinct (MI300X/MI325X/MI355X), Intel Xeon, TPU, Ascend NPU
SGLang is also the rollout backend for RL post-training frameworks including Miles, slime, verl and AReaL — the Miles AMD nightlies have their own page here.
What is SGLang CI?
SGLang's upstream CI (sgl-project/sglang) runs automated tests on every PR and nightly to validate correctness and performance across GPU platforms.
AMD CI Infrastructure
| Self-hosted runners |
linux-mi300-{1,2,4,8}gpu-sglang (MI300X/MI325X)linux-mi35x-gpu-{1,2,8} (MI355X), plus
linux-mi35x-gpu-8.fabric for the PR gate's 8-GPU disaggregation stage.All are ephemeral ARC pods — see Runner Health for the capacity split and queue. The exception is the MI355X disaggregation nightly's linux-sglang-mi35x-di, a login-node runner that allocates two or four MI355X nodes from Slurm.
|
| ROCm tracks |
pr-test-amd.yml gates every AMD-touching PR on
ROCm 10; its scheduled runs add ROCm 7.2.4 and 7.2.0.
ROCm 7.0 was retired in
sgl-project/sglang#38632.
|
| Execution |
Each job starts a container from that night's rocm/sgl-dev image
(scripts/ci/amd/amd_ci_start_container.sh --rocm-version <flavor>),
installs dependencies, then dispatches a suite with
run_suite.py --hw amd --suite <suite>.
|
| Test registration |
Tests live in test/registered/amd/ and register via register_amd_ci().
What AMD runs against what NVIDIA runs is on the
Coverage page.
|
What is Nightly CI?
Nightly CI runs every night (not on every PR) to test things that are too expensive or slow for per-PR testing:
| Type | What it tests | Example |
|---|---|---|
| Accuracy | Run models end-to-end, check output quality against benchmarks | DeepSeek-V3.2, DeepSeek-V4 Pro, GLM-5.1, MiniMax-M3 |
| Performance | Measure throughput/latency, detect regressions | DeepSeek-V3.2 MTP decode speed, GPT-OSS, Grok-2 |
| Large models | 8-GPU models too big for per-PR runners | Kimi-K2.6, Kimi-K3, Qwen3-235B |
| Quantization | MXFP4 / FP8 / INT4 paths that only exist on some hardware | DeepSeek-R1 MXFP4 on MI355X, Grok-1 INT4 |
| Disaggregation | Prefill/decode split across two MI355X nodes (1P1D) or four (2P1D), on ROCm 10 | nightly-amd-mi355x-disagg.yml |
nightly-test-amd.yml, at 17:30 UTC,
which runs every one of its jobs three times — once on ROCm 10, once on 7.2.4 and
once on 7.2.0. A model registered for both GPU families therefore lands up to six
nightly jobs.
Nightly CI shows how each of them did.
Documentation & Cookbook
docs.sglang.io is the reference documentation — installation, server arguments, the OpenAI-compatible API, and a per-platform section that covers AMD Instinct.
The SGLang Cookbook is a community-maintained site with practical guides for deploying SGLang with various models on different hardware. Includes AMD-specific deployment guides.