Tech preview

Loom¶

Loom is a language and compiler for native programs on CPUs, GPUs, and NPUs. The SDK includes a runtime and loom-serve, an inference engine that hosts multiple models and modalities together. You can use the full stack, embed parts of it, or take the compiler's output into your own runtime.

Get the SDK Run a model

Programs can be written in a subset of C++ or an isomorphic, agent-friendly Loom IR. The compiler provides structured diagnostics and reports for authoring and tuning, and emits ELF, HSACO, SPIR-V, WebAssembly, and other artifacts.

Those programs can be bundled into model packages: .loom source or .loombc bytecode, weights, and a JSON descriptor. Model-specific code lives in those files. You can load a new model, edit an implementation, or share your changes without restarting the server.

It runs on Windows and Linux; native execution needs only the OS and hardware drivers.

This project has been engineered with the assistance of agents.

Quickstart¶

With an activated Python 3.12+ environment, try loom-fun's Qwen2.5 0.5B Instruct on an AMD GPU. Fetch the model and start the server, then send a request from another terminal. These commands use a POSIX shell.

Terminal 1 Start the server

pip install 'hrx-loom[hf]'
loom-fetch huggingface:loom-fun/bartowski-Qwen2.5-0.5B-Instruct-loom

...
Qwen/Qwen2.5-0.5B-Instruct/model.json

Use the printed path, or just its directory:

loom-serve --model=Qwen/Qwen2.5-0.5B-Instruct --device=amdgpu

...
{"event":"ready","address":"127.0.0.1:8080","models":1}

Terminal 2 Send a request

curl -fsS localhost:8080/v1/chat/completions \
  --json '{"model":"Qwen/Qwen2.5-0.5B-Instruct",
           "messages":[{"role":"user","content":"Hi!"}]}'

Example response, excerpted

"message": {
  "role": "assistant",
  "content": "Hello! How can I help you today?"
}

Leave the server running while you use it. The reply is in choices[0].message.content. The fetch and serve guide covers model selection, startup status, and other requests. Each model package describes the hardware and configuration it supports.

A composition of models¶

Say you're building a digital assistant. Start with speech recognition, a small language model, and speech synthesis. Add image understanding so it can see what you're working on, embeddings so it can search your documents, and an image model when you want it to draw something. A larger language model can take on deeper research.

Install the SDK, start loom-serve, and have your agent register the models you want to use. Connect your application through the OpenAI-compatible endpoints. Adding another capability means registering another model package with the running server.

Your application Interface, tools, agent behavior, and orchestration

Other clients Your devices, friends, and family

HTTP requests

loom-serve HTTP endpoints · telemetry · serving policy

Embeddable C API (in development)

Loom services Models · scheduling · resource management

Each model brings its programs, weights, and descriptor.

Speech recognition
Conversation LLM
Speech synthesis
Embeddings
Image understanding
Image generation
Research LLM
+ Another model

Shared scheduling and execution Compatible batches · on-demand weights · retained state · managed memory

x86 CPU
AMD GPU
AMD XDNA NPU
Vulkan GPU

Several model packages, one hosting system. Each model's programs determine which targets it supports.

Model weights, state, and workspace can share a managed memory budget. Weights can load on demand, useful state can remain between requests, and idle memory can be reclaimed as the workload changes. Each modality batches compatible requests. Several clients using a model share its weights and can share execution; other computers, friends, or family can use the same service.

You can leave residency to the configured policies or control it from the application. HTTP endpoints let you drain a model, resume it, keep its weights resident, or trim idle backing while the server keeps running.

From model source to execution¶

A model's structure, tensor layouts, parameter handling, and numerical implementations live in its source. The compiler works with programs, types, memory, and target instructions. The hardware abstraction handles execution and data movement. Loom's services host text, image, speech, and other workloads; loom-serve exposes them over HTTP. The model supplies the computation behind those services.

That source covers more than individual kernels. Command programs describe kernel launches, their buffers, and their dependencies. Model programs prepare parameters and drive execution. Functions and templates can be shared across those programs and their kernels. An author can change how a layer is computed and how its work is launched in the same source package.

When model dimensions, data layouts, workload shapes, or hardware features become known, Loom can specialize the program around them. The author chooses what remains configurable and what becomes concrete. This can happen ahead of time, when a model loads, or when a new workload needs a different specialization. The same Qwen27B source files already run through two quite different paths: Loom emits AMD GPU instructions for native execution, or SPIR-V for a Vulkan driver to compile and run.

C++ subset Loom authoring API

Import / export

Loom IR .loom text / .loombc bytecode

Reusable libraries Programs, kernels, and motifs

Loom compiler Link · specialize · optimize Programs · commands · kernels

Code generation

Specialization inputs Shapes · layouts · target facts

x86 CPU: ELF
AMD GPU: HSACO
AMD XDNA NPU: Tile and array programs
Vulkan: SPIR-V
WebAssembly: Wasm modules

AOT and JIT use the same source, linking, and specialization model. Use the artifacts with Loom's runtime or your own.

This also makes experimentation part of normal use. A running server can compile and select a compatible kernel or command-program variant while keeping the model's weights and state resident. An author can test a change, inspect its structured compile report, and compare it against the original on the same workload. The improvement is something they can share as source files.

The small distribution follows from these boundaries. Model behavior lives in model packages, specialization produces the code needed for a particular use, and execution machinery is shared. A new model can add capabilities without adding another runtime and its dependency tree. That matters even when the weights occupy 100 GB: the software around them still has to be deployed, updated, trusted, and kept compatible.

Work that carries forward¶

The loom-fun project gives an early example of what people can build on this foundation. Over one weekend, a community contributor used a Claude subscription and the released SDK to port fifty models. They published the model packages, source, validation results, and performance comparisons on Strix Halo, along with the skills used to do the work. Those skills give the next person a starting point too.

Today, a practical way to begin is to find a working model similar to the one you want, point your agent at its source and the loom-fun skills, and adapt it. The existing model supplies a complete example of preparation, execution, serving, and qualification. The result is another package that people can run, inspect, and change with the same SDK.

We're building out hrx-loom-library so that more of this work becomes assembling components. The aim is reusable service behavior, techniques such as speculative decoding and cohort batching, common kernels for common targets, and the math underneath them: quantization, reductions, matrix operations, and other recurring pieces. Authors should be able to reuse a component, fork it, or replace the part they want to explore, then concentrate optimization where their application needs it.

A model port can then contribute more than one supported model. A better decoder, a new quantization implementation, or a useful scheduling technique can become something the next author builds on. The source, checks, and measurements preserve what was learned while leaving room to try something different.

The goal is to make models programmable components of an application. A model brings its weights and the programs that implement it, assembled from pieces you can reuse or change. The model owns its implementation. Loom provides the platform.

From CPUs to multi-GPU servers¶

Deployments span CPU applications and integrated GPUs. A development demonstration has also served full DeepSeek across eight MI300X GPUs. These use the same compiler and runtime foundation, with model programs and deployment configuration describing how work is placed. x86 execution covers AMD and Intel CPUs; Vulkan provides a path to other vendors' GPUs alongside the native AMD GPU backend. AMD XDNA NPUs already run embedding models and can work alongside GPU workloads.

Work can also span different kinds of processors. Our image-generation pipeline experiments run on the GPU, the NPU, or a mixture of both from the same Loom source files. The Quake rendering demonstration uses CPU preparation, NPU rasterization, and GPU consumption, with Loom compiling the NPU and GPU programs. These are general compute tools that can support quite different applications.

Multi-node execution is under development. A pipeline-parallel DeepSeek demonstration has already run across two Strix Halo all-in-one machines; that work has not yet landed in the main project.

A tech preview you can build on¶

Loom is an early experimental project. We're building out the reusable library and refactoring model hosting into an embeddable C API, with HTTP and serving policy layered above it. More scheduling and request handling will move into Loom programs so that authors can experiment with those behaviors in libraries and model packages.

Experiments in model behavior belong in those programs and libraries. Changes to the core should support new hardware, programming models, and shared capabilities that help those implementations.

Start by running a model or authoring a model package. The integration guides cover using the compiler and its artifacts in your own software, and the programming guide explains the language. The compiler and runtime are useful independently, and taking the ideas elsewhere is welcome too.