Tutorials

Learn AITER through hands-on examples.

Getting Started

• Basic Usage
  • Installation Check
  • Hello World: Flash Attention
  • Working with Different Precisions
  • RMSNorm Example
  • Batched Operations
  • Error Handling
  • Memory Management
  • Next Steps
  • Common Gotchas

Advanced Topics

• How to Add a New Operator
  • Overview
  • Step 1: Define the Operator Interface
  • Step 2: Implement the ROCm Kernel
  • Step 3: Create Python Bindings
  • Step 4: Update Build Configuration
  • Step 5: Add Tests
  • Step 6: Build and Install
  • Step 7: Register in Main Module
  • Advanced: Optimizations
  • Common Patterns
  • Best Practices
  • Debugging Tips
  • Example: Complete RMSNorm Implementation
  • Next Steps
  • Contributing

Tutorial Overview

Basic Tutorials

• Basic Usage - Your first AITER program

• attention_tutorial - Understanding attention kernels

• variable_length - Handling variable-length sequences

Advanced Topics

• How to Add a New Operator - How to add a new operator (step-by-step guide)

• moe_tutorial - Mixture of Experts optimization

• custom_kernels - Writing custom ROCm kernels

• quantization - INT8 quantization for inference

• triton_comms - Triton-based communication primitives

Integration Guides

• vllm_integration - Using AITER with vLLM

• pytorch_lightning - PyTorch Lightning integration

• deepspeed - DeepSpeed integration

Prerequisites

All tutorials assume:

• Python 3.8+

• PyTorch 2.0+ with ROCm support

• AITER installed (see Installation)

• AMD GPU (gfx90a, gfx942, or gfx950)

Example Data

Some tutorials use sample data. Download with:

# Coming soon: test data downloader
bash scripts/download_test_data.sh

Jupyter Notebooks

Interactive notebooks are available in the examples/ directory:

# Install Jupyter
pip install jupyter

# Launch notebooks
cd examples
jupyter notebook

Running Examples

All tutorial code can be run directly:

# Clone repository
git clone https://github.com/ROCm/aiter.git
cd aiter

# Run tutorial script
python examples/basic_usage.py

Community Examples

Check out community-contributed examples:

• Llama 2 inference - Optimized inference with AITER

• Mixtral 8x7B - MoE model acceleration

• GPT-style models - Training and inference

Contributing Tutorials

We welcome tutorial contributions! See ../contributing for guidelines.

Tips for following tutorials:

1. Start with basics - Don’t skip the fundamentals

2. Run the code - Type it out, don’t just copy-paste

3. Experiment - Modify parameters and observe changes

4. Profile - Use ROCm profiler to understand performance

5. Ask questions - Open issues or discussions on GitHub