ATOM Documentation

ATOM (Accelerated Training and Optimization for Models) is AMD’s high-performance LLM serving framework optimized for ROCm platforms. Find the source code at https://github.com/ROCm/ATOM.

Getting Started

• Installation
  • Requirements
  • Installation methods
  • Environment variables
  • Verify the installation
  • Troubleshooting
• Quick start
  • Offline inference
  • Distributed inference
  • API server
  • Performance tips
  • Next steps
• Model run guide
  • Quick start
  • Supported models
  • Nightly CI benchmark configurations
  • Live dashboard

User Guides

• ATOM architecture guide
  • Quick reference
  • System overview
  • Component architecture
  • Request lifecycle
  • Forward context pattern
  • Multi-process architecture
  • Sequence lifecycle
  • Speculative decoding (MTP / EAGLE)
  • Source files
• ATOM configuration guide
  • Quick reference
  • Master configuration (Config)
  • Compilation configuration (CompilationConfig)
  • Quantization configuration (QuantizationConfig & LayerQuantConfig)
  • Parallel configuration (ParallelConfig)
  • Speculative decoding configuration (SpeculativeConfig)
  • Sampling parameters (SamplingParams)
  • CLI arguments (EngineArgs)
  • Environment variables
  • Decision tree — choosing a compilation level
  • Source files
• ATOM Model Support Guide
  • Quick Reference
  • Supported model architectures
  • Model architecture details
  • Weight loading
  • Adding a new model
  • Model-specific optimizations
  • Source files
• ATOM model operations guide
  • Quick reference
  • AITER integration overview
  • Linear operations
  • Attention operations
  • Mixture of experts (MoE)
  • Normalization
  • Activation functions
  • Embedding and output head
  • Rotary position embedding (RoPE)
  • Sampling
  • Fused kernel chains
  • Source files
• ATOM scheduling & KV cache guide
  • Quick reference
  • Scheduling algorithm
  • ScheduledBatch structure
  • Block manager
  • Prefix caching
  • Postprocessing
  • Speculative decoding integration
  • Sequence management
  • Source files
• ATOM Distributed Inference Guide
  • Quick Reference
  • Tensor Parallelism (TP)
  • Data Parallelism (DP)
  • Expert Parallelism (EP)
  • Environment variables
  • Multi-GPU deployment examples
  • Combined parallelism strategies
  • Source files
• Prefill Context Parallel (PCP) Guide
  • When to use PCP
  • Quick Reference
  • CLI usage
  • Launching server
  • Tuning for long sequences
  • Performance baseline
  • Constraints & Compatibility
  • How it works
  • Source Files
• ATOM compilation & CUDA graphs guide
  • Compilation levels
  • CUDA graph modes
  • CUDA graph capture
  • Piecewise compilation
  • Forward context & stateless dispatch
  • Compiler backend
  • Configuration options
  • Decision tree
  • Source files
• ATOM serving & benchmarking guide
  • Quick reference
  • OpenAI-compatible server
  • Programmatic API (LLMEngine)
  • Simple inference
  • Benchmarking
  • Profiling
  • Speculative decoding (MTP)
  • Deployment examples
  • Accuracy validation
  • Source files

Framework Integrations

• vLLM-ATOM backend
  • Architecture
  • Configuration translation
  • Attention integration
  • Supported models
  • Installation and quick start
  • Environment variables

API Reference

• Serving API
  • LLMEngine class
  • SamplingParams
  • Return values
  • Example
• Supported models
  • Llama models
  • GPT models
  • Mixtral
  • Other architectures
  • Model configuration
  • Performance by model size
  • Quantization

Features

• High Performance: Optimized kernels for AMD Instinct GPUs

• Model Support: Wide range of LLM architectures (Llama, GPT, etc.)

• Distributed Serving: Multi-GPU and multi-node deployment

• Compilation: CUDAGraph and ROCm optimizations

• Benchmarking: Built-in performance measurement tools

Supported GPUs

GPU	Architecture	Memory	Status
AMD Instinct MI355X	CDNA 4 (gfx950)	288 GB HBM3e	✅ Fully Supported (primary CI target)
AMD Instinct MI300X	CDNA 3 (gfx942)	192 GB HBM3	✅ Fully Supported
AMD Instinct MI250X	CDNA 2 (gfx90a)	128 GB HBM2e	✅ Fully Supported

Quick links

• GitHub: https://github.com/ROCm/ATOM

• ROCm Documentation: https://rocm.docs.amd.com

• Issues: https://github.com/ROCm/ATOM/issues