Fetch and serve a model¶
loom-fetch installs the files for one model preset. Pass the resulting
model.json to loom-serve to run it. The card connects the model's programs,
parameters, tokenizer, and service contract; a separate server configuration is
optional.
To share your own programs and model cards, the Hugging Face publishing guide provides a small copyable repository, explicit metadata refresh, and an end-to-end smoke test.
This walkthrough transcribes a short English recording with
Moonshine Streaming Tiny.
It uses the original checkpoint and the Loom library's AMD GPU implementation.
Start with the SDK installed, including
the hf extra, and a supported AMD GPU. For a CPU-only example without learned
weights, use the arithmetic embedding walkthrough.
Inspect the available choices¶
The example pins a published library commit so the card, lock, programs, and sample recording come from the same revision. The following commands use a POSIX shell; the Windows acquisition command follows below.
model_url=https://raw.githubusercontent.com/ROCm/hrx-loom-library/4317508bf5adb251a284ea8dc4e5a24a11bd1c17/model/moonshine-ai/moonshine-streaming-tiny
loom-fetch "$model_url/model.json" --list
The choices include the original parameter variant, transcription service,
and streaming preset. Inspect the same facts as JSON, optionally with jq:
Listing reads the card without downloading weights or programs. Presets combine
a variant, service, program set, and canonical execution settings. Here,
streaming selects the original weights, source programs, and four pending
requests. The preset name describes the model's execution configuration; it
does not select a file format.
Fetch the selected files¶
Fetch reports its current phase, file counts, and available byte counts on
stderr. Known byte totals include a compact bar, percentage, and estimated time
remaining once progress establishes a rate. These updates also appear in
redirected logs and with --no-input; they require no terminal.
For example, a large file can produce these status lines (illustrative counts):
loom-fetch Download [###.........] 25% 1.0 GiB / 4.0 GiB 34.1 MiB/s ETA 1m 30s received 512.0 MiB (17.1 MiB/s) model.safetensors 2/3 files
loom-fetch Verify [############] 100% 4.0 GiB / 4.0 GiB model.safetensors 2/3 files
loom-fetch Complete 3/3 files
stdout contains only the installed card path, or the final JSON result with
--json:
loom-fetch "$model_url/model.json" --preset=streaming --output=./moonshine --no-input --json >installed.json 2>fetch.log
Use --quiet to suppress progress while retaining errors and access guidance.
Download progress describes reconstructed file bytes. Hugging Face's network
transfer bytes appear on the same row as received; these may differ from file
bytes because of cached or compressed chunks. ETA uses the file's observed
average progress, excluding bytes already present at resume, not the network
counter. It estimates this file stage, not the remaining installation. Before
progress establishes a rate, it reads ETA estimating; unknown file totals have
no ETA. Verification reports bytes checked, including cache hits. Ongoing
updates are limited to one line every thirty seconds, while new phases and final
completion appear promptly. A complete installation is reported only after its
model card has been published.
On Windows PowerShell, the same acquisition is:
$modelUrl = 'https://raw.githubusercontent.com/ROCm/hrx-loom-library/4317508bf5adb251a284ea8dc4e5a24a11bd1c17/model/moonshine-ai/moonshine-streaming-tiny'
loom-fetch "$modelUrl/model.json" --preset=streaming --output=./moonshine
Fetch reads the adjacent model.json.lock, resolves the selected dependencies,
and verifies identities within the lock's declared scope before publishing the
installed card. An asset-scoped lock protects weights, tokenizers, and other
assets while leaving programs editable. A complete publication lock additionally
checks program bytes. The output
contains local program and asset paths and retains the chosen preset. Supply a
new output directory for another selection; a changed selection never silently
replaces an installed model.
The default --materialize=auto copies programs and their auxiliary files so
you can edit them independently. It shares cached weights and other assets;
two installations that differ only in execution settings can share the same
parameter files. --materialize=copy makes every file independent when moving
an installation or using a filesystem that cannot share them.
--materialize=share also shares programs with their source or cache, so edits
can affect other installations. Keep the backing cache and local source files
available while using shared files.
With an asset-scoped lock, editing a program requires no lock update, and a
repeat fetch into the same installation preserves that edit. Git can version
and share the model's source files. Program origin aliases, such as @library,
do not require per-file hashes in an asset-scoped lock. --offline reuses cached
assets and the last fetched source bytes for a new installation, without
network requests. The model format reference
describes lock updates and complete publication coverage.
Hugging Face gating is handled during acquisition. If access is required,
fetch displays the repository's access URL and authentication guidance. Use
hf auth login when credentials are required. Complete
the publisher's approval process in your browser, then recheck from the prompt;
approval can be delayed. Fetch never accepts terms on your behalf. In scripts,
--no-input fails with the required action instead of waiting for a terminal.
--wait-for-access explicitly enables automatic rechecking for unattended use.
Resolve publication and access failures¶
A card URL, publisher approval, and downloader credentials are separate inputs. If a GitHub card or its adjacent lock returns HTTP 404, check that both files are published at the selected path and revision. A local checkout or unmerged branch can contain a model that is not on the published branch. The pinned example above keeps the card, lock, programs, and recording together; changing only the card's revision does not update the rest of that example.
For a Hugging Face access request, approval belongs to the account downloading the files. Browser approval with one account does not authorize another account's credential. In the environment performing acquisition, these commands identify the effective account and establish a saved login:
An HF_TOKEN environment variable overrides the saved login. With --json,
the fetch access diagnostic reports the credential source, not its value. After
changing an environment override, restart fetch so it sees the new environment.
A container or agent sandbox may intentionally have no access to the host's
saved login.
Acquisition can instead run in an authorized host shell with
--materialize=copy, followed by exposing that self-contained installation to
the server. Serving local files requires no Hugging Face credential.
Without --wait-for-access, --no-input reports an unmet access requirement and
exits unsuccessfully. It does not accept terms for the user. --wait-for-access
is the explicit waiting policy, not a remedy for a missing published file. The
publisher's gated-model instructions
explain approval, and the authentication guide
describes credential setup.
Start the server¶
Leave this terminal running. The server loads only local files; it does not
contact Hugging Face or interpret the acquisition lock. Preparation, linking,
and JIT compilation run on workers. A JSON ready message reports that the
HTTP listener is available. Model preparation progresses separately:
/v1/models lists prepared models. If this model does not appear, inspect
/v1/status: its loom.registration progresses to ready, or loom.error
explains a preparation failure. An incompatible SDK/model contract is reported
here instead of becoming an inference result.
Use curl.exe in Windows PowerShell to select curl rather than a shell alias.
The request's model name is the card's ID,
moonshine-ai/moonshine-streaming-tiny. Fetch has already retained streaming,
so no --preset or separate serve.json is needed here.
Transcribe a recording¶
In another terminal, download the small reference recording and submit it:
curl --fail --location https://raw.githubusercontent.com/ROCm/hrx-loom-library/4317508bf5adb251a284ea8dc4e5a24a11bd1c17/model/moonshine-ai/moonshine-streaming-tiny/test/short.pcm --output=short.pcm
curl --fail-with-body -H 'Expect:' http://127.0.0.1:8080/v1/audio/transcriptions -F model=moonshine-ai/moonshine-streaming-tiny -F 'file=@short.pcm;headers="Content-Type: audio/pcm;rate=16000;channels=1"'
On Windows PowerShell, download the same recording and preserve the multipart
header's quotes with the --% stop-parsing token:
curl.exe --fail --location https://raw.githubusercontent.com/ROCm/hrx-loom-library/4317508bf5adb251a284ea8dc4e5a24a11bd1c17/model/moonshine-ai/moonshine-streaming-tiny/test/short.pcm --output=short.pcm
curl.exe --% --fail-with-body -H "Expect:" http://127.0.0.1:8080/v1/audio/transcriptions -F model=moonshine-ai/moonshine-streaming-tiny -F "file=@short.pcm;headers=\"Content-Type: audio/pcm;rate=16000;channels=1\""
The response contains:
Replace short.pcm with your recording to transcribe other audio. This raw
format is signed little-endian PCM16, mono, at 16 kHz. Renaming a WAV or MP3 file
does not convert it to PCM. The audio transcription reference
describes encoded uploads, supported content types, and response options. This
Moonshine preset supports English recordings up to 40.96 seconds and greedy
decoding; it does not provide speaker diarization or word alignment.
Ctrl-C in the server terminal stops admission and joins accepted work before exiting. The fetched model remains available for the next run.
Discover other models and services¶
A release SDK can name a compatible library catalog. With that association,
loom-fetch --list shows its models and bare loom-fetch opens the terminal
picker. A development SDK requires a card URL or explicit catalog:
loom-fetch --catalog=github:ROCm/hrx-loom-library@4317508bf5adb251a284ea8dc4e5a24a11bd1c17 --list --json
This snapshot includes Moonshine speech recognition, Pocket speech synthesis,
Qwen3.8 Flash-Next text generation, Qwen3.8-27B text and vision, Qwen3 Embedding,
and Krea image generation. Pocket's streaming preset uses Alba, Marius, and
Anna; streaming-noncommercial adds Jean.
Each catalog entry identifies a runnable package with published programs and asset sources. The model map records each package's supported devices and presets.
Acquisition and startup are the same across modalities. The selected service determines the request contract:
| Service | HTTP request |
|---|---|
| Text, including optional image input | Chat completions |
| Speech synthesis | Audio speech |
| Speech recognition | Audio transcription |
| Embeddings | Embeddings |
| Image generation | Image generation |
| Known-token sequences | Sequences |
| Finite decisions | SystemOne |
A program set can contain several Loom text or bytecode files and libraries.
The server links the selected providers together and specializes them for the
device. A publisher may offer source or bytecode sets; --programs
overrides a preset only when another compatible set is actually published.
The source checkout in this example publishes the source set. Its bytecode
choices require a separately built publication.
Reuse local assets and configure a deployment¶
After a successful fetch, --offline forbids all network requests and reuses
cached documents and payloads. An origin override can use an already provisioned
checkpoint directory while retaining lock verification:
loom-fetch "$model_url/model.json" --preset=streaming --offline --origin=checkpoint=/srv/checkpoints/moonshine --output=./moonshine-local
That directory must supply the paths relative to the named origin, including
model.safetensors and tokenizer.json. A missing file fails; the override
does not fall back to the network. --no-lock is an explicit development mode
for experiments with modified inputs. Publishers use --update-lock to record
the exact selected set, or --update-lock --all-variants for a complete
publication. Adding --programs=source locks all compatible parameter variants
of the source publication without acquiring generated bytecode. The model format guide
defines those authoring workflows and transformations in detail.
For a routing alias, device assignment, or different server admission settings, write a small serving document referencing the fetched card. The model remains reusable by other hosts; HTTP listeners and deployment policy belong to the server configuration.