MORI Documentation

MORI (Modular RDMA Interface) is a bottom-up, modular, and composable framework for building high-performance communication applications with a strong focus on RDMA + GPU integration.

Getting Started

• Install MORI
  • Installation
  • Optional FlyDSL integration
  • Next steps
• Quickstart
  • Runnable checks
  • API sketches
  • MORI-EP: Dispatch and Combine
  • MORI-IO: Point-to-Point Transfers
  • MORI Shmem: Symmetric Memory
  • MORI-IR: Device Bitcode for GPU Kernels
  • Profiling with MORI-VIZ
  • Next Steps
• Choosing an EP API and backend
  • EPv2 configuration and lifecycle
  • Validation before tuning

User Guides

• MORI-EP User Guide
  • Table of Contents
  • Quick Reference
  • 1. Kernel Types
  • 2. Configuration
  • 3. Operator API
  • 4. Standard MoE Compatibility (DeepEP)
  • 5. Initialization
  • 6. Launch Configuration
  • 7. Complete Example
  • 8. Environment Variables
  • 9. Benchmarking
  • 10. Tuning
  • 11. Profiling with MORI-VIZ
  • 12. Framework Integration
  • Build Options
  • Source Files
• MORI Shmem Guide
  • Table of Contents
  • Quick Reference
  • 1. Concepts
  • 2. Initialization
  • 3. Query APIs
  • 4. Memory Management
  • 5. P2P Address Translation
  • 6. Synchronization
  • 7. HIP Module Init (Triton Integration)
  • 8. Initialization Flags
  • Environment Variables
  • Rail-only connections
  • Source Files
• MORI CCO Guide
  • Table of Contents
  • Quick Reference
  • 1. Concepts
  • 2. Initialization
  • 3. Memory and Windows
  • 4. Device Communicator
  • 5. Host Barrier
  • 6. Device-Side Programming
  • 7. Examples
  • 8. C++ API
  • 9. Environment Variables
• MORI-IR Guide
  • Table of Contents
  • Architecture
  • 1. Host-Side Python API
  • 2. Device Bitcode
  • 3. Device Functions
  • 4. Integration Example: Triton
  • 5. Integration Example: Raw Bitcode (no framework)
  • 6. Bitcode JIT Compilation
  • 7. Examples
  • 8. Testing
  • Known Limitations
• MORI-IO Introduction
  • Table of Contents
  • Design & Concepts
  • Workflow
  • Architecture
  • Python API Quick Reference
  • Example: Basic Read/Write
  • Environment & Configuration
  • Profiling MORI-IO with ROCTX Markers
  • Source Files
• MORI UMBP Single-Node Smoke Test
  • Overview
  • Prerequisites
  • Configuration Inputs
  • Launch Steps
  • Core Dumps
  • Troubleshooting
• MORI-VIZ (Visualizer)
  • Table of Contents
  • Overview
  • Instrumentation
  • Build & Code Generation
  • Python Analysis
  • Best Practices

Benchmarks

• MORI-EP Benchmark
  • Table of Contents
  • Intra-node
  • Inter-node
• MORI GEMM + All-Reduce Benchmark
  • Table of Contents
  • Running the benchmark
  • Headline
  • Where the time goes
  • The fp8 wire
  • Model-level evaluation
  • Negative results
  • Reproducing the end-to-end numbers
• MORI-IO Benchmark
  • Table of Contents
  • Which benchmark to use
  • C++ Benchmark (default, nixlbench-matching)
  • Python Benchmark
  • Benchmark Arguments
  • Results: Thor2 RDMA Read
  • Results: Thor2 RDMA Write
  • Results: CX7 RDMA (Batch Size = 1)
  • Benchmark Tuning Guide
• MORI UMBP PD Disaggregation Benchmark
  • Table of Contents
  • Prerequisites
  • Topology
  • Configuration
  • Step 1 — Start Docker on all nodes
  • Step 2 — Start Grafana and Prometheus
  • Step 3 — Build mori on all nodes
  • Step 4 — Create launcher scripts
  • Step 5 — Kill stale processes and launch prefill
  • Step 6 — Wait for ALL prefills ready
  • Step 7 — Launch decode and benchmark
  • Step 8 — Monitor decode and benchmark
  • Step 9 — Show results
  • Teardown
  • Environment Variable Reference
  • Troubleshooting
• UMBP Distributed Tier-Management Benchmark
  • Build
  • Run, generate, and replay
  • JSON backend and tier policy
  • Synthetic profiles
  • Trace contract
  • Results
• RDMA Bandwidth Utilization in Dispatch/Combine Kernels
  • Overview
  • Kernel Execution Path
  • What RDMA Actually Transfers
  • Theoretical Peak vs. Observed Throughput
  • How to Check RDMA Bandwidth from the Trace
  • Key Tuning Parameters That Affect RDMA Utilization
  • Sanity Checks from Hardware Counters (optional)
  • Summary

Developer Guides

• Mori IR Integration
• MORI JIT Compilation Framework
• MORI JIT v2

API Reference

• Communication APIs
  • MORI-EP (Expert Parallelism)
  • MORI Shmem (Symmetric Memory)
  • MORI-IO (Point-to-Point I/O)
  • MORI-IR (Device Bitcode Integration)
• MORI-VIZ Kernel Profiler
  • Requirements
  • Usage
  • export_to_perfetto
  • C++ Profiler APIs
• UMBP Master Client
  • Starting the Master Server
  • UMBPTierType
  • UMBPExternalKvNodeMatch
  • UMBPMasterClient
  • Cache Hit Tracking
  • UMBPExternalKvHitCountEntry
  • Usage Examples
  • End-to-End Example

Components

Component	Description
MORI-EP	Intra and inter-node dispatch/combine kernels for MoE Expert Parallelism
MORI-IO	Point-to-point communication library for KVCache transfer via RDMA/XGMI
MORI-CCL	Collective communication, including SDMA implementations; availability depends on the installed bindings
MORI-SHMEM	OpenSHMEM-style symmetric memory APIs for GPU memory and RDMA
MORI-CCO	Collective Communication Object: explicit-handle GPU communication (LSA / GDA / SDMA)
MORI-UMBP	Unified memory and bandwidth pool with tiered storage and distributed KV access
MORI-VIZ	Warp-level kernel profiler with Perfetto integration

Supported Hardware

GPUs: MI308X, MI300X, MI325X, MI355X (MI450X under development)

NICs: AMD Pollara (AINIC), Mellanox ConnectX-7, Broadcom Thor2 (Volcano under development)

Documentation versions

The page title identifies an exact release tag or a development source commit. Development documentation can describe features that are not in the latest stable package. Check the release notes and the backend restrictions before choosing a deployment configuration.

Quick Links

• GitHub: https://github.com/ROCm/mori

• Issues: https://github.com/ROCm/mori/issues

Indices and tables

• Index

• Search Page