Skip to main content
Ctrl+K

rocm-ernic 0.2.0

User Guide

  • Building and Installing
  • Architecture
  • Usage
  • The ionic Device
  • NVMe-oF Controller
  • S3 over RDMA Object Store
  • P4 Match-Action Engine
  • systemd Service and ernicctl
  • Prometheus Monitoring and Grafana Dashboard
  • Testing
  • Performance
  • NVMe-oF Performance
  • Performance trends

API Reference

  • API Reference
  • .rst

Performance trends

Contents

  • Send bandwidth over time
  • Send latency over time
  • NVMe-oF read bandwidth over time
  • S3-over-RDMA GET bandwidth over time
  • P4 fp16-to-fp8 conversion bandwidth over time

Performance trends#

Medians from the nightly runs, oldest at the left. Latest: e0414b65 on GitHub-hosted runners at 2026-10-28 04:36:34 UTC.

Each runner class is charted on its own axes and never joined into one line. A GitHub-hosted runner has noisy neighbours and no fixed CPU, so its figures are comparable with each other over time but not with the retired lab node’s. The README shields read the GitHub-hosted series.

Each panel carries one message size on its own scale. Bandwidth at 4 KiB and at 1 MiB differ by more than an order of magnitude, so a shared axis would flatten the small sizes onto the baseline and hide exactly the movement a regression shows up in. Bandwidth and latency are likewise never combined: they share no scale.

Send bandwidth over time#

Median bw_peak_GBs for send (higher is better).

Measured on GitHub-hosted runners:

Send bandwidth over time, one panel per message size 4 KiB 0 0.5 1 1.5 2 4 KiB — 0.91 GB/s — 7c740944 4 KiB — 0.79 GB/s — 2ca92d62 4 KiB — 1.09 GB/s — e715df52 4 KiB — 0.95 GB/s — 0b48aa1d 4 KiB — 0.59 GB/s — 0b48aa1d 4 KiB — 0.83 GB/s — 1e5932c6 4 KiB — 0.53 GB/s — 113e8656 4 KiB — 0.77 GB/s — bcccc890 4 KiB — 1.47 GB/s — 1ff21200 4 KiB — 0.95 GB/s — 14845579 4 KiB — 0.89 GB/s — 14845579 4 KiB — 1.25 GB/s — 14845579 4 KiB — 0.75 GB/s — 14845579 4 KiB — 0.81 GB/s — b4901b9c 4 KiB — 0.68 GB/s — b4901b9c 4 KiB — 0.69 GB/s — f847d97d 4 KiB — 1.35 GB/s — 76af59f3 4 KiB — 0.939025 GB/s — 76af59f3 4 KiB — 0.78 GB/s — 76af59f3 4 KiB — 0.8 GB/s — f7e3ecb1 4 KiB — 0.92 GB/s — f7e3ecb1 4 KiB — 1.01 GB/s — 65f3e2f0 4 KiB — 0.58 GB/s — 21f40e79 4 KiB — 1.59928 GB/s — e912e840 4 KiB — 0.68 GB/s — e912e840 4 KiB — 0.87 GB/s — e912e840 4 KiB — 1.42 GB/s — e912e840 4 KiB — 0.69 GB/s — bcb5d089 4 KiB — 1.59 GB/s — 73961c0b 4 KiB — 1.59 GB/s — 7b08b273 4 KiB — 1.5 GB/s — 2f1cbb8b 4 KiB — 1 GB/s — 8f8876d9 4 KiB — 0.74 GB/s — fdbdf7e3 4 KiB — 1.15 GB/s — 4479fc5d 4 KiB — 0.81 GB/s — dcc9ac9f 4 KiB — 1.39 GB/s — 151379b0 4 KiB — 1.49 GB/s — 95cb7235 4 KiB — 0.86 GB/s — b901e7e6 4 KiB — 1 GB/s — c9bef647 4 KiB — 1.65 GB/s — 9da810e2 4 KiB — 1.23 GB/s — 2f0a80cc 4 KiB — 1.49 GB/s — e0414b65 1.49 7c740944 e0414b65 64 KiB 0 5 10 15 20 64 KiB — 7.54 GB/s — 7c740944 64 KiB — 6.95 GB/s — 2ca92d62 64 KiB — 7.39 GB/s — e715df52 64 KiB — 8.02 GB/s — 0b48aa1d 64 KiB — 4.55 GB/s — 0b48aa1d 64 KiB — 7.01 GB/s — 1e5932c6 64 KiB — 4.9 GB/s — 113e8656 64 KiB — 7.1 GB/s — bcccc890 64 KiB — 13.75 GB/s — 1ff21200 64 KiB — 7.54 GB/s — 14845579 64 KiB — 7.51 GB/s — 14845579 64 KiB — 8.03 GB/s — 14845579 64 KiB — 5.65 GB/s — 14845579 64 KiB — 6.87 GB/s — b4901b9c 64 KiB — 6.05 GB/s — b4901b9c 64 KiB — 9.44 GB/s — f847d97d 64 KiB — 8.69 GB/s — 76af59f3 64 KiB — 9.57 GB/s — 76af59f3 64 KiB — 5.91 GB/s — 76af59f3 64 KiB — 7.43 GB/s — f7e3ecb1 64 KiB — 6.58 GB/s — f7e3ecb1 64 KiB — 9.21 GB/s — 65f3e2f0 64 KiB — 6.09 GB/s — 21f40e79 64 KiB — 13.2 GB/s — e912e840 64 KiB — 7.04 GB/s — e912e840 64 KiB — 7.44 GB/s — e912e840 64 KiB — 11.95 GB/s — e912e840 64 KiB — 7.02 GB/s — bcb5d089 64 KiB — 13.47 GB/s — 73961c0b 64 KiB — 11.07 GB/s — 7b08b273 64 KiB — 14 GB/s — 2f1cbb8b 64 KiB — 8.68 GB/s — 8f8876d9 64 KiB — 7.24 GB/s — fdbdf7e3 64 KiB — 10.12 GB/s — 4479fc5d 64 KiB — 7.53 GB/s — dcc9ac9f 64 KiB — 9.83 GB/s — 151379b0 64 KiB — 11.42 GB/s — 95cb7235 64 KiB — 6.4 GB/s — b901e7e6 64 KiB — 5.53 GB/s — c9bef647 64 KiB — 11.44 GB/s — 9da810e2 64 KiB — 8.11 GB/s — 2f0a80cc 64 KiB — 13.01 GB/s — e0414b65 13.01 7c740944 e0414b65 1 MiB 0 10 20 30 40 1 MiB — 16.06 GB/s — 7c740944 1 MiB — 15.45 GB/s — 2ca92d62 1 MiB — 18.36 GB/s — e715df52 1 MiB — 16.76 GB/s — 0b48aa1d 1 MiB — 16.61 GB/s — 0b48aa1d 1 MiB — 13.85 GB/s — 1e5932c6 1 MiB — 16.79 GB/s — 113e8656 1 MiB — 15.53 GB/s — bcccc890 1 MiB — 29.5 GB/s — 1ff21200 1 MiB — 16.74 GB/s — 14845579 1 MiB — 16.64 GB/s — 14845579 1 MiB — 20.03 GB/s — 14845579 1 MiB — 14.59 GB/s — 14845579 1 MiB — 17.62 GB/s — b4901b9c 1 MiB — 16.39 GB/s — b4901b9c 1 MiB — 21.03 GB/s — f847d97d 1 MiB — 20.18 GB/s — 76af59f3 1 MiB — 19.52 GB/s — 76af59f3 1 MiB — 15.15 GB/s — 76af59f3 1 MiB — 15.94 GB/s — f7e3ecb1 1 MiB — 14.79 GB/s — f7e3ecb1 1 MiB — 11.21 GB/s — 65f3e2f0 1 MiB — 10.28 GB/s — 21f40e79 1 MiB — 28.32 GB/s — e912e840 1 MiB — 14.5 GB/s — e912e840 1 MiB — 15.17 GB/s — e912e840 1 MiB — 23.81 GB/s — e912e840 1 MiB — 15.31 GB/s — bcb5d089 1 MiB — 21.73 GB/s — 73961c0b 1 MiB — 24.39 GB/s — 7b08b273 1 MiB — 24.35 GB/s — 2f1cbb8b 1 MiB — 12.39 GB/s — 8f8876d9 1 MiB — 14.38 GB/s — fdbdf7e3 1 MiB — 22.92 GB/s — 4479fc5d 1 MiB — 17.55 GB/s — dcc9ac9f 1 MiB — 14.45 GB/s — 151379b0 1 MiB — 24.34 GB/s — 95cb7235 1 MiB — 17.42 GB/s — b901e7e6 1 MiB — 13.3 GB/s — c9bef647 1 MiB — 23.72 GB/s — 9da810e2 1 MiB — 21.07 GB/s — 2f0a80cc 1 MiB — 24.79 GB/s — e0414b65 24.79 7c740944 e0414b65 GB/s, oldest run left to newest right

Last 10 runs, in GB/s:

Run

4 KiB

64 KiB

1 MiB

4479fc5d

1.15

10.12

22.92

dcc9ac9f

0.81

7.53

17.55

151379b0

1.39

9.83

14.45

95cb7235

1.49

11.42

24.34

b901e7e6

0.86

6.4

17.42

c9bef647

1

5.53

13.3

a969a3bb

–

–

–

9da810e2

1.65

11.44

23.72

2f0a80cc

1.23

8.11

21.07

e0414b65

1.49

13.01

24.79

Measured on the retired self-hosted lab node:

Send bandwidth over time, one panel per message size 4 KiB 0 0.125 0.25 0.375 0.5 4 KiB — 0.36 GB/s — 56b302de 4 KiB — 0.32 GB/s — 9a44f18e 4 KiB — 0.39 GB/s — 5e0516af 0.39 56b302de 5e0516af 64 KiB 0 0.75 1.5 2.2 3 64 KiB — 2.56 GB/s — 56b302de 64 KiB — 2.4 GB/s — 9a44f18e 64 KiB — 2.45 GB/s — 5e0516af 2.45 56b302de 5e0516af 1 MiB 0 2 4 6 8 1 MiB — 6.02 GB/s — 56b302de 1 MiB — 4.46 GB/s — 9a44f18e 1 MiB — 4.72 GB/s — 5e0516af 4.72 56b302de 5e0516af GB/s, oldest run left to newest right

Last 3 runs, in GB/s:

Run

4 KiB

64 KiB

1 MiB

56b302de

0.36

2.56

6.02

9a44f18e

0.32

2.4

4.46

5e0516af

0.39

2.45

4.72

Send latency over time#

Median lat_typical_us for send (lower is better).

Measured on GitHub-hosted runners:

Send latency over time, one panel per message size 4 KiB 0 125 250 375 500 4 KiB — 377.66 µs — 7c740944 4 KiB — 371.55 µs — 2ca92d62 4 KiB — 286.73 µs — e715df52 4 KiB — 306.54 µs — 0b48aa1d 4 KiB — 301.32 µs — 0b48aa1d 4 KiB — 348.54 µs — 1e5932c6 4 KiB — 377.33 µs — 113e8656 4 KiB — 401.77 µs — bcccc890 4 KiB — 320.08 µs — 1ff21200 4 KiB — 384.04 µs — 14845579 4 KiB — 379.71 µs — 14845579 4 KiB — 338.71 µs — 14845579 4 KiB — 389.6 µs — 14845579 4 KiB — 383.64 µs — b4901b9c 4 KiB — 399.32 µs — b4901b9c 4 KiB — 268.88 µs — f847d97d 4 KiB — 275.92 µs — 76af59f3 4 KiB — 339.82 µs — 76af59f3 4 KiB — 345.93 µs — 76af59f3 4 KiB — 353.7 µs — f7e3ecb1 4 KiB — 377.63 µs — f7e3ecb1 4 KiB — 271.23 µs — 65f3e2f0 4 KiB — 355.65 µs — 21f40e79 4 KiB — 336.16 µs — e912e840 4 KiB — 355.82 µs — e912e840 4 KiB — 360.52 µs — e912e840 4 KiB — 273.67 µs — e912e840 4 KiB — 416.58 µs — bcb5d089 4 KiB — 262.36 µs — 73961c0b 4 KiB — 257.03 µs — 7b08b273 4 KiB — 258.81 µs — 2f1cbb8b 4 KiB — 301.59 µs — 8f8876d9 4 KiB — 326.54 µs — fdbdf7e3 4 KiB — 326.37 µs — 4479fc5d 4 KiB — 335.1 µs — dcc9ac9f 4 KiB — 266.95 µs — 151379b0 4 KiB — 256.37 µs — 95cb7235 4 KiB — 346.94 µs — b901e7e6 4 KiB — 362.29 µs — c9bef647 4 KiB — 254.58 µs — 9da810e2 4 KiB — 347.34 µs — 2f0a80cc 4 KiB — 329.68 µs — e0414b65 329.68 7c740944 e0414b65 64 KiB 0 125 250 375 500 64 KiB — 367.26 µs — 7c740944 64 KiB — 442.82 µs — 2ca92d62 64 KiB — 290.74 µs — e715df52 64 KiB — 359 µs — 0b48aa1d 64 KiB — 338.99 µs — 0b48aa1d 64 KiB — 379.24 µs — 1e5932c6 64 KiB — 365.09 µs — 113e8656 64 KiB — 365.16 µs — bcccc890 64 KiB — 275.09 µs — 1ff21200 64 KiB — 385.74 µs — 14845579 64 KiB — 343.93 µs — 14845579 64 KiB — 390.71 µs — 14845579 64 KiB — 371.52 µs — 14845579 64 KiB — 362.41 µs — b4901b9c 64 KiB — 404.27 µs — b4901b9c 64 KiB — 313.69 µs — f847d97d 64 KiB — 283.65 µs — 76af59f3 64 KiB — 309.21 µs — 76af59f3 64 KiB — 363.77 µs — 76af59f3 64 KiB — 383.54 µs — f7e3ecb1 64 KiB — 350.41 µs — f7e3ecb1 64 KiB — 334.81 µs — 65f3e2f0 64 KiB — 389.24 µs — 21f40e79 64 KiB — 262.59 µs — e912e840 64 KiB — 378.43 µs — e912e840 64 KiB — 372.47 µs — e912e840 64 KiB — 273.74 µs — e912e840 64 KiB — 435.98 µs — bcb5d089 64 KiB — 306.96 µs — 73961c0b 64 KiB — 346 µs — 7b08b273 64 KiB — 387.74 µs — 2f1cbb8b 64 KiB — 299 µs — 8f8876d9 64 KiB — 386.66 µs — fdbdf7e3 64 KiB — 308.8 µs — 4479fc5d 64 KiB — 361.95 µs — dcc9ac9f 64 KiB — 285.23 µs — 151379b0 64 KiB — 343.49 µs — 95cb7235 64 KiB — 376.11 µs — b901e7e6 64 KiB — 391.12 µs — c9bef647 64 KiB — 259.91 µs — 9da810e2 64 KiB — 273.77 µs — 2f0a80cc 64 KiB — 327.93 µs — e0414b65 327.93 7c740944 e0414b65 1 MiB 0 375 750 1125 1500 1 MiB — 930.71 µs — 7c740944 1 MiB — 1026.74 µs — 2ca92d62 1 MiB — 663.11 µs — e715df52 1 MiB — 851.36 µs — 0b48aa1d 1 MiB — 802.29 µs — 0b48aa1d 1 MiB — 990.01 µs — 1e5932c6 1 MiB — 903.47 µs — 113e8656 1 MiB — 942.8 µs — bcccc890 1 MiB — 556.59 µs — 1ff21200 1 MiB — 930.52 µs — 14845579 1 MiB — 855.24 µs — 14845579 1 MiB — 689.34 µs — 14845579 1 MiB — 999.51 µs — 14845579 1 MiB — 912.17 µs — b4901b9c 1 MiB — 992.23 µs — b4901b9c 1 MiB — 696.25 µs — f847d97d 1 MiB — 875.87 µs — 76af59f3 1 MiB — 789.3 µs — 76af59f3 1 MiB — 911.54 µs — 76af59f3 1 MiB — 864.44 µs — f7e3ecb1 1 MiB — 924.29 µs — f7e3ecb1 1 MiB — 1181.5 µs — 65f3e2f0 1 MiB — 909.48 µs — 21f40e79 1 MiB — 546.04 µs — e912e840 1 MiB — 952.01 µs — e912e840 1 MiB — 954.08 µs — e912e840 1 MiB — 620.38 µs — e912e840 1 MiB — 1002.36 µs — bcb5d089 1 MiB — 826.47 µs — 73961c0b 1 MiB — 674.78 µs — 7b08b273 1 MiB — 673.12 µs — 2f1cbb8b 1 MiB — 1109.08 µs — 8f8876d9 1 MiB — 923.62 µs — fdbdf7e3 1 MiB — 718.05 µs — 4479fc5d 1 MiB — 942.58 µs — dcc9ac9f 1 MiB — 838.47 µs — 151379b0 1 MiB — 651.16 µs — 95cb7235 1 MiB — 920.28 µs — b901e7e6 1 MiB — 1020.02 µs — c9bef647 1 MiB — 513.93 µs — 9da810e2 1 MiB — 594.8 µs — 2f0a80cc 1 MiB — 538.22 µs — e0414b65 538.22 7c740944 e0414b65 µs, oldest run left to newest right

Last 10 runs, in µs:

Run

4 KiB

64 KiB

1 MiB

4479fc5d

326.37

308.8

718.05

dcc9ac9f

335.1

361.95

942.58

151379b0

266.95

285.23

838.47

95cb7235

256.37

343.49

651.16

b901e7e6

346.94

376.11

920.28

c9bef647

362.29

391.12

1020.02

a969a3bb

–

–

–

9da810e2

254.58

259.91

513.93

2f0a80cc

347.34

273.77

594.8

e0414b65

329.68

327.93

538.22

Measured on the retired self-hosted lab node:

Send latency over time, one panel per message size 4 KiB 0 100 200 300 400 4 KiB — 321.46 µs — 56b302de 4 KiB — 244.11 µs — 9a44f18e 4 KiB — 242.48 µs — 5e0516af 242.48 56b302de 5e0516af 64 KiB 0 100 200 300 400 64 KiB — 325.81 µs — 56b302de 64 KiB — 328.34 µs — 9a44f18e 64 KiB — 247.64 µs — 5e0516af 247.64 56b302de 5e0516af 1 MiB 0 150 300 450 600 1 MiB — 468.44 µs — 56b302de 1 MiB — 474.77 µs — 9a44f18e 1 MiB — 393.74 µs — 5e0516af 393.74 56b302de 5e0516af µs, oldest run left to newest right

Last 3 runs, in µs:

Run

4 KiB

64 KiB

1 MiB

56b302de

321.46

325.81

468.44

9a44f18e

244.11

328.34

474.77

5e0516af

242.48

247.64

393.74

NVMe-oF read bandwidth over time#

Median bw_avg_GBs for nvmeof (higher is better).

Measured on GitHub-hosted runners:

NVMe-oF read bandwidth over time, one panel per message size 4 KiB 0 0.038 0.075 0.113 0.15 4 KiB — 0.055195 GB/s — b66fdca9 4 KiB — 0.058755 GB/s — 7b6fa94a 4 KiB — 0.102948 GB/s — 7c740944 4 KiB — 0.086831 GB/s — 2ca92d62 4 KiB — 0.088529 GB/s — e715df52 4 KiB — 0.066281 GB/s — 0b48aa1d 4 KiB — 0.057417 GB/s — 0b48aa1d 4 KiB — 0.087054 GB/s — 0b48aa1d 4 KiB — 0.059156 GB/s — 1e5932c6 4 KiB — 0.063423 GB/s — 113e8656 4 KiB — 0.067731 GB/s — bcccc890 4 KiB — 0.060107 GB/s — 1ff21200 4 KiB — 0.057634 GB/s — 14845579 4 KiB — 0.101116 GB/s — 14845579 4 KiB — 0.086886 GB/s — 14845579 4 KiB — 0.067732 GB/s — 14845579 4 KiB — 0.06908 GB/s — b4901b9c 4 KiB — 0.098636 GB/s — b4901b9c 4 KiB — 0.059841 GB/s — f847d97d 4 KiB — 0.06439 GB/s — 76af59f3 4 KiB — 0.065498 GB/s — 76af59f3 4 KiB — 0.062091 GB/s — 76af59f3 4 KiB — 0.064499 GB/s — f7e3ecb1 4 KiB — 0.081664 GB/s — f7e3ecb1 4 KiB — 0.058396 GB/s — 65f3e2f0 4 KiB — 0.090241 GB/s — 21f40e79 4 KiB — 0.055678 GB/s — e912e840 4 KiB — 0.10748 GB/s — e912e840 4 KiB — 0.071709 GB/s — e912e840 4 KiB — 0.072811 GB/s — e912e840 4 KiB — 0.059753 GB/s — bcb5d089 4 KiB — 0.078136 GB/s — 73961c0b 4 KiB — 0.094428 GB/s — 7b08b273 4 KiB — 0.10627 GB/s — 2f1cbb8b 4 KiB — 0.060945 GB/s — 8f8876d9 4 KiB — 0.064688 GB/s — fdbdf7e3 4 KiB — 0.104818 GB/s — 4479fc5d 4 KiB — 0.035783 GB/s — dcc9ac9f 4 KiB — 0.112564 GB/s — 151379b0 4 KiB — 0.060744 GB/s — 95cb7235 4 KiB — 0.094514 GB/s — b901e7e6 4 KiB — 0.037943 GB/s — c9bef647 4 KiB — 0.034286 GB/s — a969a3bb 4 KiB — 0.087094 GB/s — 9da810e2 4 KiB — 0.088004 GB/s — 2f0a80cc 4 KiB — 0.071585 GB/s — e0414b65 0.071585 b66fdca9 e0414b65 64 KiB 0 0.5 1 1.5 2 64 KiB — 0.828823 GB/s — b66fdca9 64 KiB — 0.937654 GB/s — 7b6fa94a 64 KiB — 1.60176 GB/s — 7c740944 64 KiB — 1.31773 GB/s — 2ca92d62 64 KiB — 1.28944 GB/s — e715df52 64 KiB — 0.85107 GB/s — 0b48aa1d 64 KiB — 0.891161 GB/s — 0b48aa1d 64 KiB — 1.43795 GB/s — 0b48aa1d 64 KiB — 0.764613 GB/s — 1e5932c6 64 KiB — 0.800211 GB/s — 113e8656 64 KiB — 0.833989 GB/s — bcccc890 64 KiB — 0.79379 GB/s — 1ff21200 64 KiB — 0.879635 GB/s — 14845579 64 KiB — 1.49693 GB/s — 14845579 64 KiB — 1.32469 GB/s — 14845579 64 KiB — 0.99477 GB/s — 14845579 64 KiB — 0.847138 GB/s — b4901b9c 64 KiB — 1.47476 GB/s — b4901b9c 64 KiB — 0.927104 GB/s — f847d97d 64 KiB — 0.942458 GB/s — 76af59f3 64 KiB — 0.969352 GB/s — 76af59f3 64 KiB — 0.82765 GB/s — 76af59f3 64 KiB — 0.899732 GB/s — f7e3ecb1 64 KiB — 1.12972 GB/s — f7e3ecb1 64 KiB — 0.92882 GB/s — 65f3e2f0 64 KiB — 1.32793 GB/s — 21f40e79 64 KiB — 0.908251 GB/s — e912e840 64 KiB — 1.75355 GB/s — e912e840 64 KiB — 1.03409 GB/s — e912e840 64 KiB — 1.18399 GB/s — e912e840 64 KiB — 0.754218 GB/s — bcb5d089 64 KiB — 1.30441 GB/s — 73961c0b 64 KiB — 1.36824 GB/s — 7b08b273 64 KiB — 1.64799 GB/s — 2f1cbb8b 64 KiB — 0.868211 GB/s — 8f8876d9 64 KiB — 1.01999 GB/s — fdbdf7e3 64 KiB — 1.65297 GB/s — 4479fc5d 64 KiB — 0.619257 GB/s — dcc9ac9f 64 KiB — 0.932681 GB/s — 151379b0 64 KiB — 0.736472 GB/s — 95cb7235 64 KiB — 1.10373 GB/s — b901e7e6 64 KiB — 0.457612 GB/s — c9bef647 64 KiB — 0.503941 GB/s — a969a3bb 64 KiB — 1.33646 GB/s — 9da810e2 64 KiB — 0.979161 GB/s — 2f0a80cc 64 KiB — 0.603212 GB/s — e0414b65 0.603212 b66fdca9 e0414b65 1 MiB 0 1 2 3 4 1 MiB — 1.42307 GB/s — b66fdca9 1 MiB — 1.80337 GB/s — 7b6fa94a 1 MiB — 2.82049 GB/s — 7c740944 1 MiB — 1.93243 GB/s — 2ca92d62 1 MiB — 1.91219 GB/s — e715df52 1 MiB — 1.46981 GB/s — 0b48aa1d 1 MiB — 1.28665 GB/s — 0b48aa1d 1 MiB — 2.6433 GB/s — 0b48aa1d 1 MiB — 1.3423 GB/s — 1e5932c6 1 MiB — 1.47457 GB/s — 113e8656 1 MiB — 1.36652 GB/s — bcccc890 1 MiB — 1.47982 GB/s — 1ff21200 1 MiB — 1.77292 GB/s — 14845579 1 MiB — 3.03169 GB/s — 14845579 1 MiB — 1.83653 GB/s — 14845579 1 MiB — 1.69867 GB/s — 14845579 1 MiB — 1.49716 GB/s — b4901b9c 1 MiB — 2.6998 GB/s — b4901b9c 1 MiB — 1.56679 GB/s — f847d97d 1 MiB — 1.31101 GB/s — 76af59f3 1 MiB — 1.57585 GB/s — 76af59f3 1 MiB — 1.76967 GB/s — 76af59f3 1 MiB — 1.65339 GB/s — f7e3ecb1 1 MiB — 1.4474 GB/s — f7e3ecb1 1 MiB — 1.60305 GB/s — 65f3e2f0 1 MiB — 1.85275 GB/s — 21f40e79 1 MiB — 1.56642 GB/s — e912e840 1 MiB — 3.04698 GB/s — e912e840 1 MiB — 1.46082 GB/s — e912e840 1 MiB — 1.45395 GB/s — e912e840 1 MiB — 1.6261 GB/s — bcb5d089 1 MiB — 2.44423 GB/s — 73961c0b 1 MiB — 2.77744 GB/s — 7b08b273 1 MiB — 3.3809 GB/s — 2f1cbb8b 1 MiB — 1.79028 GB/s — 8f8876d9 1 MiB — 2.12703 GB/s — fdbdf7e3 1 MiB — 3.13766 GB/s — 4479fc5d 1 MiB — 1.89117 GB/s — dcc9ac9f 1 MiB — 3.35846 GB/s — 151379b0 1 MiB — 2.02672 GB/s — 95cb7235 1 MiB — 3.33257 GB/s — b901e7e6 1 MiB — 1.91172 GB/s — c9bef647 1 MiB — 1.28226 GB/s — a969a3bb 1 MiB — 1.22671 GB/s — 9da810e2 1 MiB — 1.35417 GB/s — 2f0a80cc 1 MiB — 2.12094 GB/s — e0414b65 2.12094 b66fdca9 e0414b65 GB/s, oldest run left to newest right

Last 10 runs, in GB/s:

Run

4 KiB

64 KiB

1 MiB

4479fc5d

0.104818

1.65297

3.13766

dcc9ac9f

0.035783

0.619257

1.89117

151379b0

0.112564

0.932681

3.35846

95cb7235

0.060744

0.736472

2.02672

b901e7e6

0.094514

1.10373

3.33257

c9bef647

0.037943

0.457612

1.91172

a969a3bb

0.034286

0.503941

1.28226

9da810e2

0.087094

1.33646

1.22671

2f0a80cc

0.088004

0.979161

1.35417

e0414b65

0.071585

0.603212

2.12094

S3-over-RDMA GET bandwidth over time#

Median bw_avg_GBs for s3 (higher is better).

Measured on GitHub-hosted runners:

S3-over-RDMA GET bandwidth over time, one panel per message size 4 KiB 0 0.003 0.005 0.007 0.01 4 KiB — 0.006039 GB/s — e715df52 4 KiB — 0.005941 GB/s — 0b48aa1d 4 KiB — 0.005359 GB/s — 0b48aa1d 4 KiB — 0.005512 GB/s — 0b48aa1d 4 KiB — 0.005994 GB/s — 1e5932c6 4 KiB — 0.005477 GB/s — 113e8656 4 KiB — 0.008293 GB/s — bcccc890 4 KiB — 0.005817 GB/s — 1ff21200 4 KiB — 0.005605 GB/s — 14845579 4 KiB — 0.00535 GB/s — 14845579 4 KiB — 0.007165 GB/s — 14845579 4 KiB — 0.005104 GB/s — 14845579 4 KiB — 0.005756 GB/s — b4901b9c 4 KiB — 0.002424 GB/s — b4901b9c 4 KiB — 0.005135 GB/s — f847d97d 4 KiB — 0.00538 GB/s — 76af59f3 4 KiB — 0.006538 GB/s — 76af59f3 4 KiB — 0.005117 GB/s — 76af59f3 4 KiB — 0.00663 GB/s — f7e3ecb1 4 KiB — 0.006336 GB/s — f7e3ecb1 4 KiB — 0.004645 GB/s — 65f3e2f0 4 KiB — 0.008593 GB/s — 21f40e79 4 KiB — 0.00559 GB/s — e912e840 4 KiB — 0.005796 GB/s — e912e840 4 KiB — 0.008158 GB/s — e912e840 4 KiB — 0.005795 GB/s — e912e840 4 KiB — 0.005358 GB/s — bcb5d089 4 KiB — 0.007768 GB/s — 73961c0b 4 KiB — 0.006014 GB/s — 7b08b273 4 KiB — 0.005207 GB/s — 8f8876d9 4 KiB — 0.005488 GB/s — fdbdf7e3 4 KiB — 0.005372 GB/s — 4479fc5d 4 KiB — 0.00545 GB/s — dcc9ac9f 4 KiB — 0.003812 GB/s — 151379b0 4 KiB — 0.007331 GB/s — 95cb7235 4 KiB — 0.007785 GB/s — b901e7e6 4 KiB — 0.008456 GB/s — c9bef647 4 KiB — 0.00553 GB/s — a969a3bb 4 KiB — 0.005324 GB/s — 9da810e2 4 KiB — 0.004907 GB/s — 2f0a80cc 4 KiB — 0.004922 GB/s — e0414b65 0.004922 e715df52 e0414b65 64 KiB 0 0.05 0.1 0.15 0.2 64 KiB — 0.100311 GB/s — e715df52 64 KiB — 0.089183 GB/s — 0b48aa1d 64 KiB — 0.084159 GB/s — 0b48aa1d 64 KiB — 0.089893 GB/s — 0b48aa1d 64 KiB — 0.095586 GB/s — 1e5932c6 64 KiB — 0.081948 GB/s — 113e8656 64 KiB — 0.132483 GB/s — bcccc890 64 KiB — 0.096441 GB/s — 1ff21200 64 KiB — 0.093426 GB/s — 14845579 64 KiB — 0.086312 GB/s — 14845579 64 KiB — 0.10714 GB/s — 14845579 64 KiB — 0.0862 GB/s — 14845579 64 KiB — 0.08659 GB/s — b4901b9c 64 KiB — 0.024142 GB/s — b4901b9c 64 KiB — 0.081798 GB/s — f847d97d 64 KiB — 0.091504 GB/s — 76af59f3 64 KiB — 0.11249 GB/s — 76af59f3 64 KiB — 0.092989 GB/s — 76af59f3 64 KiB — 0.11345 GB/s — f7e3ecb1 64 KiB — 0.098102 GB/s — f7e3ecb1 64 KiB — 0.075031 GB/s — 65f3e2f0 64 KiB — 0.134062 GB/s — 21f40e79 64 KiB — 0.084988 GB/s — e912e840 64 KiB — 0.070341 GB/s — e912e840 64 KiB — 0.129858 GB/s — e912e840 64 KiB — 0.092296 GB/s — e912e840 64 KiB — 0.093245 GB/s — bcb5d089 64 KiB — 0.1276 GB/s — 73961c0b 64 KiB — 0.09719 GB/s — 7b08b273 64 KiB — 0.082667 GB/s — 8f8876d9 64 KiB — 0.083969 GB/s — fdbdf7e3 64 KiB — 0.081574 GB/s — 4479fc5d 64 KiB — 0.093053 GB/s — dcc9ac9f 64 KiB — 0.081243 GB/s — 151379b0 64 KiB — 0.116682 GB/s — 95cb7235 64 KiB — 0.107186 GB/s — b901e7e6 64 KiB — 0.119994 GB/s — c9bef647 64 KiB — 0.085895 GB/s — a969a3bb 64 KiB — 0.084193 GB/s — 9da810e2 64 KiB — 0.079662 GB/s — 2f0a80cc 64 KiB — 0.085865 GB/s — e0414b65 0.085865 e715df52 e0414b65 1 MiB 0 0.625 1.2 1.9 2.5 1 MiB — 1.31572 GB/s — e715df52 1 MiB — 1.26371 GB/s — 0b48aa1d 1 MiB — 1.20345 GB/s — 0b48aa1d 1 MiB — 1.27119 GB/s — 0b48aa1d 1 MiB — 1.29512 GB/s — 1e5932c6 1 MiB — 1.27721 GB/s — 113e8656 1 MiB — 1.86829 GB/s — bcccc890 1 MiB — 1.10772 GB/s — 1ff21200 1 MiB — 1.27849 GB/s — 14845579 1 MiB — 1.22764 GB/s — 14845579 1 MiB — 1.60662 GB/s — 14845579 1 MiB — 1.25983 GB/s — 14845579 1 MiB — 1.27219 GB/s — b4901b9c 1 MiB — 0.799994 GB/s — b4901b9c 1 MiB — 1.18717 GB/s — f847d97d 1 MiB — 1.09365 GB/s — 76af59f3 1 MiB — 1.59065 GB/s — 76af59f3 1 MiB — 1.10522 GB/s — 76af59f3 1 MiB — 1.70216 GB/s — f7e3ecb1 1 MiB — 1.29286 GB/s — f7e3ecb1 1 MiB — 0.698547 GB/s — 65f3e2f0 1 MiB — 2.04993 GB/s — 21f40e79 1 MiB — 1.21768 GB/s — e912e840 1 MiB — 1.25729 GB/s — e912e840 1 MiB — 1.98461 GB/s — e912e840 1 MiB — 1.42608 GB/s — e912e840 1 MiB — 1.17955 GB/s — bcb5d089 1 MiB — 1.90672 GB/s — 73961c0b 1 MiB — 1.29093 GB/s — 7b08b273 1 MiB — 1.16135 GB/s — 8f8876d9 1 MiB — 1.21934 GB/s — fdbdf7e3 1 MiB — 1.15589 GB/s — 4479fc5d 1 MiB — 1.2696 GB/s — dcc9ac9f 1 MiB — 0.733097 GB/s — 151379b0 1 MiB — 1.52898 GB/s — 95cb7235 1 MiB — 1.8163 GB/s — b901e7e6 1 MiB — 1.95431 GB/s — c9bef647 1 MiB — 1.17389 GB/s — a969a3bb 1 MiB — 1.25818 GB/s — 9da810e2 1 MiB — 1.23548 GB/s — 2f0a80cc 1 MiB — 1.21928 GB/s — e0414b65 1.21928 e715df52 e0414b65 GB/s, oldest run left to newest right

Last 10 runs, in GB/s:

Run

4 KiB

64 KiB

1 MiB

4479fc5d

0.005372

0.081574

1.15589

dcc9ac9f

0.00545

0.093053

1.2696

151379b0

0.003812

0.081243

0.733097

95cb7235

0.007331

0.116682

1.52898

b901e7e6

0.007785

0.107186

1.8163

c9bef647

0.008456

0.119994

1.95431

a969a3bb

0.00553

0.085895

1.17389

9da810e2

0.005324

0.084193

1.25818

2f0a80cc

0.004907

0.079662

1.23548

e0414b65

0.004922

0.085865

1.21928

P4 fp16-to-fp8 conversion bandwidth over time#

Median bw_avg_GBs for p4cvt (higher is better).

Measured on GitHub-hosted runners:

P4 fp16-to-fp8 conversion bandwidth over time, one panel per message size 4 KiB 0 0.005 0.01 0.015 0.02 4 KiB — 0.013483 GB/s — c9bef647 4 KiB — 0.015136 GB/s — a969a3bb 4 KiB — 0.009988 GB/s — 9da810e2 4 KiB — 0.014568 GB/s — 2f0a80cc 4 KiB — 0.013607 GB/s — e0414b65 0.013607 c9bef647 e0414b65 64 KiB 0 0.013 0.025 0.038 0.05 64 KiB — 0.026705 GB/s — c9bef647 64 KiB — 0.038104 GB/s — a969a3bb 64 KiB — 0.018601 GB/s — 9da810e2 64 KiB — 0.037291 GB/s — 2f0a80cc 64 KiB — 0.026102 GB/s — e0414b65 0.026102 c9bef647 e0414b65 GB/s, oldest run left to newest right

Last 10 runs, in GB/s:

Run

4 KiB

64 KiB

4479fc5d

–

–

dcc9ac9f

–

–

151379b0

–

–

95cb7235

–

–

b901e7e6

–

–

c9bef647

0.013483

0.026705

a969a3bb

0.015136

0.038104

9da810e2

0.009988

0.018601

2f0a80cc

0.014568

0.037291

e0414b65

0.013607

0.026102

previous

NVMe-oF Performance

next

API Reference

Contents
  • Send bandwidth over time
  • Send latency over time
  • NVMe-oF read bandwidth over time
  • S3-over-RDMA GET bandwidth over time
  • P4 fp16-to-fp8 conversion bandwidth over time

By Advanced Micro Devices, Inc.

© Copyright 2025-2026 Advanced Micro Devices, Inc. All rights reserved..