ATOM Documentation

ATOM (Accelerated Training and Optimization for Models) is AMD’s high-performance LLM serving framework optimized for ROCm platforms.

Getting Started

• Installation
  • Requirements
  • Installation Methods
  • Environment Variables
  • Verification
  • Troubleshooting
• Quickstart
  • Serving a Model
  • Batch Inference
  • Distributed Serving
  • API Server
  • Performance Tips
  • Next Steps

User Guides

• ATOM Architecture Guide
  • 1. System Overview
  • 2. Component Architecture
  • 3. Request Lifecycle
  • 4. Forward Context Pattern
  • 5. Multi-Process Architecture
  • 6. Sequence Lifecycle
  • Source Files
• ATOM Configuration Guide
  • Quick Reference
  • 1. Master Configuration (Config)
  • 2. Compilation Configuration (CompilationConfig)
  • 3. Quantization Configuration (QuantizationConfig & LayerQuantConfig)
  • 4. Parallel Configuration (ParallelConfig)
  • 5. Speculative Decoding Configuration (SpeculativeConfig)
  • 6. Sampling Parameters (SamplingParams)
  • 7. CLI Arguments (EngineArgs)
  • 8. Environment Variables
  • 9. Decision Tree – Choosing a Compilation Level
  • Source Files
• ATOM Model Support Guide
  • Quick Reference
  • 1. Supported Model Architectures
  • 2. Model Architecture Details
  • 3. Weight Loading
  • 4. Adding a New Model
  • 5. Model-Specific Optimizations
  • Source Files
• ATOM Model Operations Guide
  • Quick Reference
  • 1. AITER Integration Overview
  • 2. Linear Operations
  • 3. Attention Operations
  • 4. Mixture of Experts (MoE)
  • 5. Normalization
  • 6. Activation Functions
  • 7. Embedding & Output Head
  • 8. Rotary Position Embedding (RoPE)
  • 9. Sampling
  • 10. Fused Kernel Chains
  • Source Files
• ATOM Scheduling & KV Cache Guide
  • Quick Reference
  • 1. Scheduling Algorithm
  • 2. ScheduledBatch Structure
  • 3. Block Manager
  • 4. Prefix Caching
  • 5. Postprocessing
  • 6. Speculative Decoding Integration
  • 7. Sequence Management
  • Source Files
• ATOM Distributed Inference Guide
  • Quick Reference
  • 1. Tensor Parallelism (TP)
  • 2. Data Parallelism (DP)
  • 3. Expert Parallelism (EP)
  • 4. Environment Variables
  • 5. Multi-GPU Deployment Examples
  • 6. Combined Parallelism Strategies
  • Source Files
• ATOM Compilation & CUDA Graphs Guide
  • 1. Compilation Levels
  • 2. CUDA Graph Modes
  • 3. CUDA Graph Capture
  • 4. Piecewise Compilation
  • 5. Forward Context & Stateless Dispatch
  • 6. Compiler Backend
  • 7. Configuration Options
  • 8. Decision Tree
  • Source Files
• ATOM Serving & Benchmarking Guide
  • Quick Reference
  • 1. OpenAI-Compatible Server
  • 2. Programmatic API (LLMEngine)
  • 3. Simple Inference
  • 4. Benchmarking
  • 5. Profiling
  • 6. Speculative Decoding (MTP)
  • 7. Deployment Examples
  • 8. Accuracy Validation
  • Source Files

API Reference

• Serving API
  • LLMEngine Class
  • SamplingParams
  • Return Values
  • Example
• Supported Models
  • Llama Models
  • GPT Models
  • Mixtral
  • Other Architectures
  • Model Configuration
  • Performance by Model Size
  • Quantization

Features

• High Performance: Optimized kernels for AMD Instinct GPUs

• Model Support: Wide range of LLM architectures (Llama, GPT, etc.)

• Distributed Serving: Multi-GPU and multi-node deployment

• Compilation: CUDAGraph and ROCm optimizations

• Benchmarking: Built-in performance measurement tools

Supported GPUs

GPU	Architecture	Memory	Status
AMD Instinct MI300X	CDNA 3 (gfx942)	192 GB HBM3	✅ Fully Supported
AMD Instinct MI250X	CDNA 2 (gfx90a)	128 GB HBM2e	✅ Fully Supported
AMD Instinct MI300A	CDNA 3 (gfx950)	128 GB HBM3	🧪 Experimental

Quick Links

• GitHub: https://github.com/ROCm/ATOM

• ROCm Documentation: https://rocm.docs.amd.com

• Issues: https://github.com/ROCm/ATOM/issues

Getting Help

• Documentation: https://rocm.github.io/ATOM/

• GitHub Issues: https://github.com/ROCm/ATOM/issues

• ROCm Community: https://github.com/ROCm/ROCm/discussions

Indices and tables

• Index

• Search Page