CCO device API → FlyDSL and Triton integration
Lets FlyDSL and Triton kernels call the CCO GDA/LSA/SDMA device API through
extern-C wrappers compiled to bitcode and linked into the kernel. The C++
device implementation (ccoGda<P, Mode> / ccoSdma) is reused unchanged.
The layers (top → bottom)
CCO_DEVICE_FUNCTIONS (one scalar ABI table)
├─ FlyDSL: cco.DevComm(h).gda(0).put(...)
│ └─ link_extern(ffi(...), cov6 bitcode)
└─ Triton: mori.ir.triton.cco.gda_put_iw_inc(...)
└─ core.extern_elementwise(..., cov5 bitcode)
└─ llvm.call @cco_gda_put__iw__inc
└─ ccoGda<P, Mode>::put<...>() / ccoSdma::put<...>()
What to look at where
Question |
File |
|---|---|
What device ops exist / the C++ wrapper |
|
The shared ABI metadata |
|
FlyDSL FFI objects |
|
Triton extern wrappers |
|
OO API ( |
|
Enums ( |
|
|
|
Buffer load/store with a cache policy that holds on every FlyDSL version |
|
u32 signal-slot accessors (peer flags, rank-private epochs) |
|
How the |
|
How the |
|
How to prebuild the |
|
Runnable examples |
|
Key design points
Scalar handles. FlyDSL FFI is scalar-only, so device objects cross as
uint64intptrs (ccoDevComm*,ccoWindow_t); signal id/value cross asint32/uint64. Struct layout lives only in C++.No dispatch overhead (monomorphization). Each template axis (
Coop×ThreadMode×RemoteAction× doorbell mode) is compiled into a distinctextern "C"symbol, name-mangled with a tag (cco_gda_put__iw__inc__deferred= indep warp + signal-inc on a DEFERRED handle). These are compile-time constants (Python enums), sohandles.pypicks the symbol by name at trace time and the kernel emits one direct call to one instantiation. The team rides in a constantGdaFlagsword that folds once the wrapper is inlined. Passing a runtime DSL value for any axis is aTypeError(it can’t select a symbol).always_inline(CCO_DEV) then inlines the thin forwarder into the kernel.ThreadMode is selectable (
ThreadMode.INDEPENDENT/AGGREGATE); the data path carries it as part of its(ThreadMode,Coop)tag (it/iw/ib/at).AGGREGATEis only valid withCoopScope.THREAD(cco coalesces the warp’s lanes itself — enforced in both the wrapper andhandles.py).Concrete status returns. Side-effect wrappers return
int32status 0. FlyDSL ignores it; Triton needs a concrete element return type forextern_elementwise, especially when an argument is a block value.Provider fixed at build time via
CCO_GDA_BUILD_PROVIDER(NIC macro) — oneccoGda<P, Mode>per NIC, same as the C++ kernels / shmem bitcode.OO handles (
DevComm/Window/Gda) are method-carryingfx.structs (cco_struct): build once, reuse across control flow.Gda.ctxandGda.doorbellareConstexprso the handle carries a single IR value; an op that posts to or flushes a QP takesctx=to use another context for that call, which may be a runtime value.
LSA vs GDA — different models
LSA (intra-node P2P): cco only exposes
cco_lsa_ptr→ the peer’s load/store-accessible VA. The kernel operates on that pointer directly (buffer_load/buffer_store); cco does NOT move the data. See examples 04/05.GDA (RDMA): opaque network ops, so they ARE exposed as device ops —
gda.put / put_value / get / signal / wait_signal / flush / flush_peer / flush_async / wait.team(Team.WORLD, orTeam.GDAfor a peer numbered in the GDA team, i.e. the node index on a RAIL connection) rides in a constantGdaFlagsword.Doorbell mode belongs to the handle,
dc.gda(ctx, doorbell=GdaDoorbell.X), and is part of the symbol name.IMMEDIATE(default) posts ring for themselves.DEFERREDput / put_value / get post without ringing; the next ring that reaches them (an IMMEDIATE post or signal after them, a flush or flush_async) releases them. Worth it when one warp posts many WQEs to a QP and releases them once.CALLER_ORDEREDposts also skip the ready mark, and only a CALLER_ORDERED flush releases them, for callers that already order every post before the flush (contract onGdaDoorbell); such a handle has nosignal, so put with asignal_op.QPs are shared by every handle on one
ctx(taken modulogda_context_count), whatever its mode. IMMEDIATE and DEFERRED mix freely; while CALLER_ORDERED posts are pending on a QP, only CALLER_ORDERED handles may post to or flush it. Nothing checks this, so give such traffic its own contexts.flush_async/wait:flush_asyncreturns a request forwait, so work can go between them. Callwaitwith the same coop (withBLOCKonly thread 0 holds the request).Per-call
ctx=picks the QP context (waitneeds none: the request names the QP). It may differ per lane only with coopTHREADon signal, flush_async, and put / put_value / get inThreadMode.INDEPENDENT;AGGREGATE,flushandflush_peerneed onectx(and peer) across the warp.
See examples 03/06.
SDMA (intra-node copy engine):
sdma.put / get(window + offset) andput_ptr / get_ptr(resolved addresses) mirrorccoSdma’s, with its template arguments as compile-time keywords (coop,local_signal,remote_signal,aggregate+commit,signal_per_copy,same_queue). A signal bumpssignal_slot(<sender lsa rank>, qid)– ours forlocal_signal, the peer’s forremote_signal– whichwait_signalpolls;quiet / quiet_queuedrain regardless. See example 07.
Build & run
Set up the environment with the deploy-mori skill (.claude/skills/deploy-mori),
then install MORI and the example runtime deps:
pip install . # builds + co-locates all libmori_*.so
pip install mpi4py "flydsl==0.4.0" # mpi4py: bootstrap; flydsl: the device bindings
After pip install . no PYTHONPATH / LD_LIBRARY_PATH / MORI_CCO_BC is
needed — the libs are co-located in site-packages/mori/ (RUNPATH $ORIGIN) and
the device bitcode is JIT-compiled on first use (cached in ~/.mori/jit, per
arch+NIC+cov). The only run-time env var is the RDMA interface:
export MORI_SOCKET_IFNAME=<iface> # e.g. enp159s0np0
export MORI_CCO_GDA_CONN=full # required for GDA on a single node
mpirun -n 2 python examples/cco/python/03_flydsl_put/main.py
See examples/cco/README.md for the full
run guide (Python + C++), including the two-node (crossnode) setup. To skip
JIT, override with MORI_CCO_BC=/path/to/libmori_cco_device.bc or prebuild via
tools/build_cco_bitcode.sh.
Triton uses code object version 5 and passes CCO state explicitly:
from mori.ir.triton import cco
kernel[(1,)](
dc.ptr,
win.handle,
extern_libs=cco.get_extern_libs(),
)
No shmem install_hook() is used. Run the complete example with:
torchrun --standalone --nproc_per_node=2 \
examples/cco/ir/test_triton_cco.py --transport lsa
Adding a new device op (the path)
Add the
extern "C"wrapper incco_device_wrapper.cpp, as a call to the public device API (ccoGda/ccoSdma/ …), not its internals. Template axes cross the scalar FFI one of two ways, both with no runtime cost:a tag in the symbol name (
cco_<op>__<tag>), one symbol per value: the GDA ops via theCCO_TC_LIST/CCO_COOP_LISTX-macros and their doorbell mode (__deferred/__caller_ordered), the SDMA coop scope;a
flagsargument the wrapper dispatches on (SdmaWithFlagsforcco_sdma_{put,get}_ptr__<coop>,GdaWithTeamfor the team bit of the GDA ops that take a peer). The wrapper is inlined, so a compile-time constantflagsfolds to one instantiation; the Python handles always pass one. Prefer this for an axis with many combinations. SDMA wrappers go inside#if BUILD_CCO_SDMA. JIT re-compiles automatically — the cache key includes the wrapper source and the cco headers.
Add the scalar signature once to
CCO_DEVICE_FUNCTIONSinops.py. Give an SDMA op afamilystarting withsdma_: the bitcode tests use it to skip the op when BUILD_CCO_SDMA is off (the#ifabove is what drops the symbol). Triton exports the entry and FlyDSL binds it (_bindings.ALL) automatically.Expose it as a method in both
flydsl/handles.pyandmori/ir/triton/cco/handles.py, with the keywords of the C++ template arguments, and add the method name totest_triton_facades_match_flydsl_handle_names.
tests/python/cco/test_triton_bindings.py checks that the cov5 bitcode defines
exactly the registered symbols (no more, no fewer) with the registered argument
and return types, and that FlyDSL binds the same set. It does not check step 3
beyond the method names.
Examples
Example |
Shows |
|---|---|
|
GDA put + signal/wait (single-node FULL + 2-node CROSSNODE) |
|
LSA direct peer-pointer store in the kernel |
|
LSA custom all-reduce: peer pointers + device signal barrier |
|
GDA template matrix: (thread_mode,coop) {indep×thread/warp/block, aggr×thread} × signal {inc,add}; deferred and caller-ordered puts + flush_async/wait per team |
|
Triton DevComm queries plus LSA, SDMA, and GDA |