SGLang Info
Overview of SGLang, CI infrastructure, and Docker images
Stable Docker Image
Official SGLang release images for production use.
lmsysorg/sglang
Docker Hub
docker pull lmsysorg/sglang:latestCUDA 12 / CUDA 13 variants are
:dev-cu12 and :dev-cu13;
the released version is on the
releases page.
ROCm Nightly Image
AMD ROCm nightly builds for MI300X/MI325X/MI355X development — the same images AMD CI runs on.
rocm/sgl-dev
Docker Hub
:latest tag. Every build is dated:
<version>-<rocm>-<gpu>-<YYYYMMDD>, six a night —
rocm700, rocm720 and rocm724 ×
mi30x and mi35x.
docker pull rocm/sgl-dev:v0.5.17-rocm724-mi35x-20260820What is SGLang?
SGLang is a high-performance serving framework for large language models and multimodal models, from a single GPU up to large distributed clusters. It is hosted by LMSYS and runs on more than 400,000 GPUs worldwide.
Start with the install guide and the AMD GPU platform page; for a worked introduction on AMD Instinct hardware see the ROCm blog post: SGLang on AMD Instinct GPUs.
Key Capabilities
- RadixAttention prefix caching, zero-overhead CPU scheduler, continuous batching, chunked prefill
- Prefill-decode disaggregation and speculative decoding (MTP)
- Tensor, pipeline, expert and data parallelism; multi-LoRA batching
- FP4/FP8/INT4/AWQ/GPTQ quantization; MXFP4 on MI355X
- LLMs, MoE, VLMs, embedding, reward and diffusion models
- NVIDIA GPUs, AMD Instinct (MI300X/MI325X/MI355X), Intel Xeon, TPU, Ascend NPU
SGLang is also the rollout backend for RL post-training frameworks including Miles, slime, verl and AReaL — the Miles AMD nightlies have their own page here.
What is SGLang CI?
SGLang's upstream CI (sgl-project/sglang) runs automated tests on every PR and nightly to validate correctness and performance across GPU platforms.
AMD CI Infrastructure
| Self-hosted runners |
linux-mi300-{1,2,4,8}gpu-sglang (MI300X/MI325X)linux-mi35x-gpu-{1,2,8} (MI355X), plus
linux-mi35x-gpu-8.fabric for the two-node disaggregation nightly.All are ephemeral ARC pods — see Runner Health for the capacity split and queue. |
| ROCm tracks |
ROCm 7.2 is the PR gate (pr-test-amd-rocm720.yml);
ROCm 7.0 (pr-test-amd.yml) was demoted to a scheduled rollback shadow.
Nightlies additionally cover ROCm 7.2.4.
|
| Execution |
Each job starts a container from that night's rocm/sgl-dev image
(scripts/ci/amd/amd_ci_start_container.sh --rocm-version <flavor>),
installs dependencies, then dispatches a suite with
run_suite.py --hw amd --suite <suite>.
|
| Test registration |
Tests live in test/registered/amd/ and register via register_amd_ci().
What AMD runs against what NVIDIA runs is on the
Coverage page.
|
What is Nightly CI?
Nightly CI runs every night (not on every PR) to test things that are too expensive or slow for per-PR testing:
| Type | What it tests | Example |
|---|---|---|
| Accuracy | Run models end-to-end, check output quality against benchmarks | DeepSeek-V3.2, DeepSeek-V4 Pro, GLM-5.1, MiniMax-M3 |
| Performance | Measure throughput/latency, detect regressions | DeepSeek-V3.2 MTP decode speed, GPT-OSS, Grok-2 |
| Large models | 8-GPU models too big for per-PR runners | Kimi-K2.6, Kimi-K3, Qwen3-235B |
| Quantization | MXFP4 / FP8 / INT4 paths that only exist on some hardware | DeepSeek-R1 MXFP4 on MI355X, Grok-1 INT4 |
| Disaggregation | Prefill/decode split across two MI355X nodes (1P1D) | nightly-amd-mi355x-disagg.yml |
nightly-test-amd.yml (ROCm 7.0) and
nightly-test-amd-rocm720.yml, which runs every one of its jobs twice —
once on the ROCm 7.2 image and once on 7.2.4. Both start at 17:30 UTC.
A model registered for both GPU families therefore lands up to six nightly jobs.
Nightly CI shows how each of them did.
Documentation & Cookbook
docs.sglang.io is the reference documentation — installation, server arguments, the OpenAI-compatible API, and a per-platform section that covers AMD Instinct.
The SGLang Cookbook is a community-maintained site with practical guides for deploying SGLang with various models on different hardware. Includes AMD-specific deployment guides.