Fused Communication and Compute Operations#

Operations that fuse communication with compute in a single kernel, accessible via the ops property on the Iris instance (e.g. ctx.ops.matmul_all_reduce(...)). The compute side is currently GEMM, supplied by tritonBLAS; the communication side is an Iris collective. Fusing them means the collective overlaps the matmul instead of running as a separate kernel after it.

matmul_all_reduce#

OpsNamespace.matmul_all_reduce(output_tensor, A, B, bias=None, async_op=False, config=None, workspace=None)[source]#

Fused matrix multiplication and all-reduce.

Computes: output = all_reduce(A @ B + bias)

Parameters:
  • output_tensor – Output tensor (M, N)

  • A – Input matrix A (M, K)

  • B – Input matrix B (K, N)

  • bias – Optional bias vector (M,) or (N,)

  • async_op – If False, performs barrier at end

  • config – Optional FusedConfig for tuning

  • workspace – Optional pre-allocated workspace

Returns:

Updated workspace object

Return type:

workspace

Example

>>> output = shmem.zeros((M, N), dtype=torch.float16)
>>> shmem.ops.matmul_all_reduce(output, A, B)

all_gather_matmul#

OpsNamespace.all_gather_matmul(output_tensor, A_sharded, B, bias=None, async_op=False, config=None, workspace=None)[source]#

Fused all-gather and matrix multiplication.

Computes: output = all_gather(A_sharded) @ B + bias

Parameters:
  • output_tensor – Output tensor (M, N)

  • A_sharded – Sharded input matrix (M, K_local)

  • B – Input matrix B (K, N) where K = K_local * world_size

  • bias – Optional bias vector (M,) or (N,)

  • async_op – If False, performs barrier at end

  • config – Optional FusedConfig for tuning

  • workspace – Optional pre-allocated workspace

Returns:

Updated workspace object

Return type:

workspace

Example

>>> K_local = K // world_size
>>> A_sharded = shmem.randn((M, K_local), dtype=torch.float16)
>>> output = shmem.zeros((M, N), dtype=torch.float16)
>>> shmem.ops.all_gather_matmul(output, A_sharded, B)

matmul_all_gather#

OpsNamespace.matmul_all_gather(output_tensor, A, B, bias=None, async_op=False, config=None, workspace=None)[source]#

Fused matrix multiplication and all-gather.

Computes: output = all_gather(A @ B + bias) along M dimension

Parameters:
  • output_tensor – Output tensor (M*world_size, N)

  • A – Input matrix A (M, K)

  • B – Input matrix B (K, N)

  • bias – Optional bias vector (M,) or (N,)

  • async_op – If False, performs barrier at end

  • config – Optional FusedConfig for tuning

  • workspace – Optional pre-allocated workspace

Returns:

Updated workspace object

Return type:

workspace

Example

>>> M_local = M // world_size
>>> A = shmem.randn((M_local, K), dtype=torch.float16)
>>> output = shmem.zeros((M, N), dtype=torch.float16)
>>> shmem.ops.matmul_all_gather(output, A, B)

matmul_reduce_scatter#

Rank r gets rows [r * M / W, (r + 1) * M / W) of the sum of A @ B over the W ranks, like torch.distributed.reduce_scatter_tensor, in an (M / W, N) output. FusedConfig(engine="sdma") moves the partial results with the SDMA copy engines instead of compute units; the context must then be created with engines=("sdma",). Pass the returned workspace back in on later calls. The other ops support only engine="cu" (the default).

OpsNamespace.matmul_reduce_scatter(output_tensor, A, B, async_op=False, config=None, workspace=None)[source]#

Fused matrix multiplication and reduce-scatter.

Computes A @ B on every rank and gives rank r rows [r * M / W, (r + 1) * M / W) of their sum over the W ranks, like torch.distributed.reduce_scatter_tensor of A @ B. M must be a multiple of W.

Parameters:
  • output_tensor – Output tensor (M / world_size, N)

  • A – Input matrix A (M, K)

  • B – Input matrix B (K, N)

  • async_op – If False, performs barrier at end

  • config – Optional FusedConfig for tuning; engine picks compute units or SDMA

  • workspace – Optional workspace from an earlier call (reuse it: a new one synchronizes ranks)

Returns:

Updated workspace object

Return type:

workspace

Example

>>> output = shmem.zeros((M // world_size, N), dtype=torch.float16)
>>> workspace = shmem.ops.matmul_reduce_scatter(output, A, B)
>>> shmem.ops.matmul_reduce_scatter(output, A, B, workspace=workspace)
>>> # Move the partials with the SDMA copy engines (context created with engines=("sdma",))
>>> config = FusedConfig(engine="sdma", block_size_m=256, block_size_n=256)
>>> workspace = shmem.ops.matmul_reduce_scatter(output, A, B, config=config)