Case error

Run2026-09-17T112050-35210422045Commit73409f79a56ac10ae5374b419e5737221daadc34Date2026-09-17 11:20:50 UTCTargetgfx950Reciperecipes/sanitizers/daily-consan-gemm.yamlExecutionerrorFindings0Containerrocm/pytorch:rocm10.0_ubuntu26.04_py3.14_pytorch_release_2.13.0@sha256:3174cb7061d94c427da96c0edef4adea28046fa3f3b2ff3948dc4e995665ff8crocjitsu bundle6e6574b59b8a1283c95e85cb93dd0e4eeab378f2
Reasonconsan_output_parse_error: reader 109123484335968 atomic site count mismatch

Reproduce

aorta sweep run --recipe recipes/sanitizers/daily-consan-gemm.yaml

Set ROCJITSU_PREBUILT (unpacked rocjitsu bundle supplying rj_waitcheck and the ConSan hook) before running. aorta sets HSA_TOOLS_LIB, HSA_TOOLS_DISABLE_REGISTER, RJ_CONSAN_MODE, RJ_CONSAN_POLICY for you from the recipe's policy block.

Rebuild the fixtures

fixtures/isa/consan_gemm_f32.hsaco

one representative heavy f32 SS GEMM code object

export PATH="$(python -c "from aorta.instrumentation.rocm_paths import resolve_rocm_roots; print(resolve_rocm_roots().llvm_bin_dir)"):${PATH}"
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH:+${LD_LIBRARY_PATH}:}$(python -c "import os; from aorta.instrumentation.rocm_paths import resolve_rocm_roots as r; x = r(); print(os.pathsep.join(dict.fromkeys([str(x.core_lib_dir), str(x.lib_dir)])))")"
mkdir -p recipes/sanitizers/fixtures/isa
python scripts/sanitizers/prepare_gemm_isa.py --csv recipes/sanitizers/fixtures/gemm_shapes_unique.csv --out recipes/sanitizers/fixtures/isa --top-n 0 --consan-object recipes/sanitizers/fixtures/isa/consan_gemm_f32.hsaco

Extracted from the shipped Tensile libraries, never compiled from a .hip source.

fixtures/bin/consan_gemm_load

a host repro binary built from fixtures/kernels/consan_load.hip

export PATH="$(python -c "from aorta.instrumentation.rocm_paths import resolve_rocm_roots; print(resolve_rocm_roots().llvm_bin_dir)"):${PATH}"
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH:+${LD_LIBRARY_PATH}:}$(python -c "import os; from aorta.instrumentation.rocm_paths import resolve_rocm_roots as r; x = r(); print(os.pathsep.join(dict.fromkeys([str(x.core_lib_dir), str(x.lib_dir)])))")"
mkdir -p recipes/sanitizers/fixtures/bin
hipcc --offload-arch=gfx950 -DOBJECT="\"$(pwd)/recipes/sanitizers/fixtures/isa/consan_gemm_f32.hsaco\"" recipes/sanitizers/fixtures/kernels/consan_load.hip -o recipes/sanitizers/fixtures/bin/consan_gemm_load

The -DOBJECT define is required: it is how the binary learns which code object to load.

The exact invocations the nightly used are in .github/workflows/sanitizers-nightly.yml.

Artifacts not published

PathSHA-256
fixtures/isa/consan_gemm_f32.hsaco57c5d8efa448315ddc5f49170757ebeec1eb397468676ff158d19c71572b1c55
fixtures/bin/consan_gemm_load6e4ff7c53fcf007e3c6604ec6093510cbff56ae83337fbb1a6057aa245be39ad

Recorded digests

KeyValue
code_object:consan_gemm_f32.hsaco57c5d8efa448315ddc5f49170757ebeec1eb397468676ff158d19c71572b1c55
command/workspace/aorta/recipes/sanitizers/fixtures/bin/consan_gemm_load
command_sha2566e4ff7c53fcf007e3c6604ec6093510cbff56ae83337fbb1a6057aa245be39ad
hook/workspace/aorta/.sanitizer-nightly/rocjitsu-prebuilt/lib/librocjitsu_dbi_hooks.so
hook_sha25635e793caddd1966afc7071334ba7484b5c28c73a9fd9203249267337373b92da
selected_identity_sha256acd50ca2bb450f4fb8144de0b24d11a502038583bf296d9e36f32091c5fbe079
selected_kernelgemm_f32_ss

Files

Every file published for this case. Its logs, recipe copy and inputs were pruned for falling outside the nightly's log-retention window. It is not the whole recipe: 2 required inputs are CI-built and excluded by design, listed under artifacts not published with their SHA-256s.

FileSize
REPRODUCE.md4.7 KB
env.json4.9 KB
sanitizer_report.json2.6 KB