A fast and efficient LLM inference backend on AMD iGPU

HIP EP compiles your ONNX graph through an MLIR pipeline and executes it on AMD GPUs with hipBLASLt and custom HIP kernels.

Models
20+
model architectures supported
Context
128K
maximum supported context, in tokens
Hardware
3
Ryzen AI processor series: Max, 400, 300
License
Open source
compiler, runtime and kernels in one repository

Model showcase

Models that already run on your GPU Measured on release v0.5.3

Qwen

LLM · Dense

Qwen2.5 14B Instruct

Prefill speed, at a prompt of Prefill 2K
1032 tok/s
Decode throughput, Decode
28.82 tok/s

LLM · Dense

Qwen2.5-Coder 14B Instruct

Prefill speed, at a prompt of Prefill 2K
1032 tok/s
Decode throughput, Decode
28.82 tok/s

VLM · Dense

Qwen3.5 9B

Prefill speed, at a prompt of Prefill 16K
1164 tok/s
Decode throughput, Decode
41.73 tok/s

VLM · MoE

Qwen3.5 35B-A3B

Prefill speed, at a prompt of Prefill 16K
1879 tok/s
Decode throughput, Decode
69.65 tok/s

VLM · MoE

Qwen3.5 122B-A10B

Prefill speed, at a prompt of Prefill 16K
430 tok/s
Decode throughput, Decode
24.17 tok/s

VLM · Dense

Qwen3.6 27B

Prefill speed, at a prompt of Prefill 16K
448 tok/s
Decode throughput, Decode
13.96 tok/s

VLM · MoE

Qwen3.6 35B-A3B

Prefill speed, at a prompt of Prefill 16K
1841 tok/s
Decode throughput, Decode
69.57 tok/s

VLM · Dense

Qwen3.8 27B

Prefill speed, at a prompt of Prefill 16K
463 tok/s
Decode throughput, Decode
14.96 tok/s

VLM · Dense

Qwen3.8 Flash-Next

Prefill speed, at a prompt of Prefill 16K
534 tok/s
Decode throughput, Decode
31.93 tok/s

Gemma

VLM · Dense

Gemma 3 4B IT

Prefill speed, at a prompt of Prefill 16K
2547 tok/s
Decode throughput, Decode
79.29 tok/s

VLM · Dense

Gemma 4 E4B IT

Prefill speed, at a prompt of Prefill 2K
1948 tok/s
Decode throughput, Decode
57.95 tok/s

VLM · Dense

Gemma 4 12B IT

Prefill speed, at a prompt of Prefill 2K
902 tok/s
Decode throughput, Decode
25.87 tok/s

VLM · MoE

Gemma 4 26B-A4B IT

Prefill speed, at a prompt of Prefill 2K
1227 tok/s
Decode throughput, Decode
59.73 tok/s

GPT-OSS

LLM · MoE

GPT-OSS 20B

Prefill speed, at a prompt of Prefill 2K
2307 tok/s
Decode throughput, Decode
90.41 tok/s

LLM · MoE

GPT-OSS 120B

Prefill speed, at a prompt of Prefill 2K
1061 tok/s
Decode throughput, Decode
48.40 tok/s

Laguna

LLM · MoE

Laguna XS 2.1

Prefill speed, at a prompt of Prefill 2K
2217 tok/s
Decode throughput, Decode
77.25 tok/s

LLM · MoE

Laguna S 2.1

Prefill speed, at a prompt of Prefill 2K
762 tok/s
Decode throughput, Decode
36.64 tok/s

Nemotron

LLM · Dense

Nemotron 3 Super 120B-A12B

Prefill speed, at a prompt of Prefill 16K
238 tok/s
Decode throughput, Decode
22.28 tok/s

Mistral

LLM · Dense

Mistral 7B Instruct v0.3

Prefill speed, at a prompt of Prefill 2K
1678 tok/s
Decode throughput, Decode
54.58 tok/s

VLM · MoE

Mistral Small 4 119B

Prefill speed, at a prompt of Prefill 16K
763 tok/s
Decode throughput, Decode
30.16 tok/s

Llama

LLM · Dense

Llama 3.1 8B

Prefill speed, at a prompt of Prefill 2K
1645 tok/s
Decode throughput, Decode
51.47 tok/s

DeepSeek

Phi

LLM · Dense

Phi-4 14B

Prefill speed, at a prompt of Prefill 128
1067 tok/s
Decode throughput, Decode
27.18 tok/s

Step

LLM · MoE

Step 3.5 Flash

Prefill speed, at a prompt of Prefill 2K
66 tok/s
Decode throughput, Decode
17.69 tok/s