Serving HTTP profiles¶

All TCP and Unix listeners expose the same catalog and HTTP endpoints. Inference requests select a registered deployment through the model field in JSON or multipart form data. Names come from model documents, not checkpoint filenames or an inferred architecture. An absent model returns 404.

The text, image, embedding, speech and transcription profiles use familiar OpenAI response shapes, but are not the entire OpenAI API. Each profile describes the fields it consumes and the options delegated to the model. Chat requests accept additional fields so clients can send metadata and model-specific extensions. Acceptance alone does not establish that an option affects generation. The server has no built-in authentication or TLS termination.

Request framing¶

Request bodies accept either Content-Length or HTTP/1.1 Transfer-Encoding: chunked. Supplying both rejects. Chunked transfer is decoded incrementally into the same bounded request body; the complete request is validated before routing. --request_body_bytes bounds decoded body bytes, independently of bounded HTTP/chunk framing and multipart metadata. This permits upload clients that do not know the body length in advance; it does not turn an inference endpoint into a continuing microphone-input session.

Multipart fields retain their names, ordering and binary contents through request ownership. Consumers use those parsed fields without interpreting MIME boundaries again. Part order does not control routing. Unsupported transfer encodings, ambiguous framing and malformed/truncated multipart bodies reject. Expect: 100-continue is unsupported; upload examples explicitly clear that header.

Discovery and administration¶

Method and path	Input or result
`GET /healthz`, `GET /readyz`	Server availability: `{"status":"ready"}`.
`GET /v1/models`	Prepared deployments not being removed, with input/output modalities and a `loom` capability object.
`GET /v1/status`	Every registration, progress/errors, admission, scheduler state and shared server resources.
`POST /v1/models/register`	Complete model document, optionally with uploaded source files; HTTP 202 accepts registration intent.
`POST /v1/models/drain`, `POST /v1/models/undrain`	`{"model":"name"}`; HTTP 200 acknowledges the gate change.
`POST /v1/models/unregister`	`{"model":"name","mode":"drain"}`; mode may instead be `"abort"`; HTTP 202 accepts removal.
`POST /v1/models/activate`, `POST /v1/models/deactivate`	`{"model":"name","component":"primary"}`; HTTP 202 accepts a residency operation on a healthy drained registration.
`POST /v1/models/pin`, `POST /v1/models/unpin`	Same component selector; HTTP 200 changes parameter eviction policy.
`GET /v1/models/variants`	Native executable inventory or one candidate's retained compilation diagnostics.
`POST /v1/models/variants`	Stage native providers against a ready base without selecting them.
`POST /v1/models/variants/select`	Select a ready candidate with expected-current protection.
`POST /v1/models/variants/remove`	Remove an unused terminal candidate.
`POST /v1/pools/trim`	`{"pool":"interactive","target_bytes":0}` selects an explicit pool; HTTP 200 schedules eligible reclamation and returns that pool's current resource accounting.
`POST /v1/devices/trim`	`{"device":"gpu"}` prunes unused compiled generations on that device without trimming any pool. Live model catalogs retain their compiled products.
`GET /metrics`	Prometheus text counters and ownership gauges.
`GET /logz`, `GET /logz/stream`	Bounded JSONL polling or SSE with independent cursors.
`GET /v1/requests`, `GET /v1/sessions`	Bounded live owner inspection; sessions require a retained-text service.

Prepared but drained or weight-evicted models remain discoverable. created is the registration timestamp; owned_by is "local". loom.kind, optional loom.endpoint, and loom.tokenize describe the deployment role, HTTP inference route, and loaded tokenizer. Native-only activity models omit loom.endpoint; HTTP inference requests to them return 404. Management and tokenizer utilities remain available according to their normal capabilities. Text also reports configured context_length and limits.max_output_tokens, not training context or total cache-pool capacity. loom.input_tokens:true advertises exact prepared-chat input counting. Capabilities do not establish learned quality or support for every optional field.

Prepared text models declare reasoning explicitly. Models with an effort selector also publish supported_reasoning_levels and default_reasoning_level, the policy used when a request omits reasoning controls. For example, a Qwen policy can advertise:

{
  "reasoning": true,
  "supported_reasoning_levels": ["none", "low", "medium", "xhigh"],
  "default_reasoning_level": "none"
}

These are fields of each /v1/models row, not server-wide settings. An absent declaration means unknown; reasoning:false means the model does not support reasoning. Accepted aliases need not appear as distinct levels. Discovery reflects source policy without inferring capabilities from the model name.

Residency controls default to component:"primary". A text deployment with an image-input component also accepts component:"image"; an image-generation deployment's normal computation is still primary. An undeclared component returns 404; an invalid selector returns 400. Drain and unregister apply to the whole registration and do not accept a component selector.

Operation specifies asynchronous control completion, conflict and overload behavior. Observation specifies model selection, cursors, SSE resume and inspection pagination.

Native executable variants¶

A prepared service supporting native updates reports loom.variants:{"selected":12,"count":2} in model discovery and status. Text, sequence, image, embedding and decision services enable this capability through nonzero variant_capacity. Disabled or custom services without the capability return 501. A registration must have a healthy prepared service; drained admission is permitted, removal is not.

GET /v1/models/variants requires X-Loom-Model: scout and an empty body. It returns object:"list", model, selected, count, offset, data and next_offset. Optional X-Loom-Offset selects a zero-based index; each page contains at most 64 entries in creation order. This is live inventory, not a stable snapshot across concurrent removals. Instead, X-Loom-Variant: 13 selects one full record, including compilation diagnostics. The variant and offset headers are mutually exclusive.

Staging accepts:

{"model":"scout","base":12,"base_directory":"/srv/models/scout/candidate","sources":["config.loom","advance.loombc"],"wait":true}

base identifies a ready choice in this registration. It asserts that the new providers preserve its resource contract; it does not merge in the base's providers. sources supplies the complete native provider set. With a JSON request, relative paths require an absolute base_directory; otherwise paths must be absolute. Files are read by server-side compilation workers. A multipart request instead supplies an immutable source package, described below. prepare, control and other model fields reject.

Omitted or true wait completes the HTTP request after JIT with 200 for ready or 422 for failed compilation. False returns 202 after accepting construction. Both responses contain selected and variant. A variant record contains object:"variant", id, base, phase (compiling, ready or failed) and users (explicit consumer references). Terminal failures retain an error with code/message. Detailed records also include available compilation evidence. Its stage_duration_sum_ns sums overlapping stage intervals, as defined in preparation timing. Staging never changes the preferred choice, including when JIT fails.

Selection accepts {"model":"scout","variant":13,"expected":12}. The request completes at the service's safe execution boundary and returns {"selected":13}. A stale expected ID, pending selection, compiling/failed candidate or candidate from another registration rejects without switching. Selection failure leaves the previous preference intact. Query inventory after an error or interrupted connection to inspect the actual preference.

Execution replacement preserves preparation/control VM programs, stage declarations and specialization values, parameter placement, and retained state. It can replace command programs and their reachable kernels, or direct-dispatch implementations including resident pipelines. It does not rerun weight preparation. Text selects between retired epochs; embedding and decision services select outside their active cohort; image services finish the whole image and CPU encoder before selection. Prepared but unsubmitted work may use the new execution because its request/result layout is unchanged. Image command part counts and transient requirements may change without relinking the control VM. Changing preserved model contracts requires a new registration.

Removal accepts {"model":"scout","variant":12} and returns {"removed":12}. Compiling or consumer-retained entries cannot be removed. Missing entries return 404; conflicts return 409; exhausted pool or management-observation credit returns 429. Unknown, duplicate or wrongly typed fields return 400.

IDs never recycle within the server process, including after unregister/register. They are not persistent identifiers across restarts. Compilation results remain until explicit removal or model retirement. Disconnect withdraws the reader, not accepted compilation or selection. Unregister withdraws waiting compilation readers with 410 while still joining the compiler's actual model ownership.

Uploaded source packages¶

Registration and native staging also accept multipart/form-data: exactly one request part contains the same JSON as the corresponding endpoint, and repeated file parts carry programs and metadata. Each file's filename is a case-sensitive UTF-8 path inside the package, not a server filesystem destination. The request filename defaults to request.json; its directory is the base for relative source references.

For a local candidate.json containing {"model":"scout","base":12,"sources":["config.loom","advance.loombc"]}:

curl --fail-with-body -H 'Expect:' \
  http://127.0.0.1:8080/v1/models/variants \
  -F 'request=@candidate.json;type=application/json;filename=model/candidate.json' \
  -F 'file=@config.loom;filename=model/config.loom' \
  -F 'file=@advance.loombc;filename=model/advance.loombc'

Package filenames use / separators and contain no empty, . or .. components, absolute paths, backslashes, drive prefixes or control characters. Authored relative references may use .. to reach a shared provider inside the package, but cannot escape its root. Missing members fail without falling back to server-local files. Explicit absolute source paths still name server-local providers, so an upload may combine changed files with existing local inputs. Filenames are literal UTF-8, not URL-decoded.

Parts may arrive in any order; duplicates and malformed/truncated bodies reject before registration or staging. File bytes are retained in memory independently of the request connection, including binary .loombc. Native staging releases its source ownership after constructing independent executables and copied diagnostics. Registration retains the package until the model retires, including when admission:"drained" defers preparation. These are code packages, not checkpoint transfers or a persistent package catalog.

For POST /v1/models/register, the request part is the complete model document. Relative prepare, control, native providers, tokenizer metadata and source-declared weight-preparation policies resolve inside the package. An image or finite model's relative asset directory also applies within that namespace when loading metadata. Absolute locators still explicitly select server-local files.

Parameter checkpoint bytes always use the server filesystem, independently of the package namespace. An absolute base_directory in the model document binds relative checkpoint paths there; it does not redirect relative code or metadata away from the package. For example, checkpoint:"." or assets:"." can let a model read uploaded tokenizer metadata while its source-declared weight files resolve under base_directory:"/srv/checkpoints/model".

The normal --request_body_bytes limit covers the complete multipart body; there is no implicit upload exception. Both content-length and chunked framing are accepted. The empty Expect header above suppresses an unsupported 100-continue handshake. This endpoint loads executable code and has the same trusted-network boundary as model registration.

Chat completions¶

POST /v1/chat/completions accepts a text deployment and a message history:

{
  "model": "scout",
  "messages": [{"role":"user","content":"What is 2+2?"}],
  "stream": true,
  "stream_options": {"include_usage": true},
  "max_tokens": 16,
  "temperature": 0
}

Omitted, false or null stream selects one chat.completion JSON response with the complete assistant message, finish reason and usage. stream:true selects incremental chat.completion.chunk SSE events. stream_options.include_usage adds a final usage-only chunk when streaming; null or omission leaves it off. stream_options has no effect on a nonstreaming response, and additional members such as include_obfuscation are ignored. Both modes use the same generation and retained-session path.

max_completion_tokens sets the output limit. If it is absent or null, max_tokens applies; if both are absent or null, the deployment default applies. When both have values, max_completion_tokens takes precedence. The selected limit must be an integer from 1 through 2147483647 and fit the deployment's context and output allowance. The limit counts selected tokens, including terminal EOS when generated, so completion-token usage can exceed the number of visibly decoded tokens. Input and reserved output must fit the deployment's context. Overflow rejects with HTTP 400 instead of silently trimming history or reducing the output budget. The combined-budget diagnostic reports input tokens, max_tokens, required stored tokens, and capacity; the last selected output token does not need another stored position. A source input check may reject earlier, before the full input count is available.

temperature, top_p, top_k and seed are numeric request fields interpreted by the model's source policy. Omission uses source-defined deployment defaults; unsupported values reject before admission. The server carries the validated policy to the model program without reading logits back to the host. Supported sampling algorithms and value ranges belong to the model package.

Sampling is separate from thinking: the boolean enable_thinking, string reasoning_effort, and model_options object are passed to source chat policy, which validates their meaning and renders the model's framing. Supported effort levels, aliases, and omitted-control defaults belong to the model package; unsupported or contradictory controls reject before admission. Reasoning-aware models return reasoning_content separately from answer content, in both SSE deltas and complete messages.

preserve_thinking and chat_template_kwargs also pass to source policy. The model package defines which controls it consumes, their types, and how top-level and nested spellings combine. Native admission interprets only transport and resource controls; other request fields remain intact for the model program.

Request compatibility¶

Additional members in request, message, supported content-part, image/video descriptor, and tool-call history objects do not cause rejection. The complete original request remains available to the model program. Required values, supported content types, and the types and bounds of consumed fields are still checked: an unknown content type is an error rather than a silently omitted part of the prompt. Duplicate native fields reject, including a null followed by another value.

Optional native controls accept null as omission. This includes tools, output limits, stream_options, media_io_kwargs, image detail, and optional message fields such as reasoning_content and tool_calls. Source-owned sampling and template options use the model package's defaults and validation rules.

Request fields	Behavior
`model`, `messages`, `stream`, output limits, `stream_options.include_usage`	Native routing, input preparation, response framing, and resource limits.
`tools`, `tool_choice`, `parallel_tool_calls`, template and sampling options	Passed to the model program; supported behavior belongs to its package.
`stop`, `presence_penalty`, `frequency_penalty`, `response_format`	Passed to the model program. Native acceptance does not promise stop-string matching, penalties, or constrained output. A policy that does not consume them ignores them.
`n`, `logprobs`, `top_logprobs`	Accepted and passed through. The native chat response currently contains one choice and `logprobs:null`.
`user`, `metadata`, `store`, and other unconsumed extensions	Accepted without a native effect; available to model policy. `store` does not create a retrievable completion archive or control session/prefix caching.

Clients can send familiar defaults such as n:1, stop:null, logprobs:false, zero penalties, and response_format:{"type":"text"} without a native unsupported-field error. Model documentation specifies which non-default options affect generation. Model configuration and administration endpoints retain their own documented schemas.

Tools and output¶

tools accepts up to 4096 entries. The catalog, tool_choice (including "required" and named-function objects), parameter schemas, and metadata such as strict pass through to the model program. They are not a native execution allowlist or a promise of constrained generation. Model-specific prompting and sampling determine how choices affect generation. The server returns generated calls without checking their names against the catalog or rejecting them based on tool_choice. Null or omitted tools supplies an empty catalog. The source policy owns tool grammar, safe output boundaries and canonical history. Generated calls become structured tool_calls; the server never runs the tools. Argument strings are opaque at the HTTP boundary: unknown functions, missing or extra arguments, duplicate members, and even invalid argument JSON can reach the client. The harness validates arguments and permissions before execution, then returns either a result or a validation error as a role:"tool" message in the next full conversation history. A bad read({"q":"file"}) call can therefore receive a missing-path error and be corrected by the model without retrying an earlier successful image-generation call.

For model programs selecting the optional XML codec, schema type hints recover JSON values from untyped parameter text. Unknown parameters remain literal strings; duplicate parameters remain duplicate JSON members in their original order. No required-field, range, enum, or permission checks run in that codec. The model-authoring contract describes its representation rules.

Generation stopped by max_tokens returns finish_reason:"length", including when complete tool calls precede the limit. An unfinished final XML call is omitted rather than repaired or presented as executable. Completed calls, visible text, reasoning, and token usage remain available; the client decides how to continue. EOS returns tool_calls when calls were generated, or stop otherwise. History with an omitted suffix cannot reuse different retained device state.

An output translation failure returns an error object with "type":"loom_output_error": HTTP 500 for a nonstreaming completion, or an SSE error followed by [DONE] after streaming has begun. This is distinct from loom_request_error, which describes a rejected request. Both carry a human message and a code naming the native failure, such as INVALID_ARGUMENT, OUT_OF_RANGE, or UNAVAILABLE. Admission and checkpoint failures distinguish temporary unavailability (503) from internal failure (500). Broken model framing, including an unterminated XML parameter at EOS, is a translation error; its diagnostic identifies the call ordinal and byte offset. A failed tool-call list publishes no partial structured calls. Earlier successful tool results remain the client's conversation history; resuming an unfinished turn preserves those results and any created artifacts rather than repeating their side effects. The server does not automatically retry generation or execute a repaired call.

Image input¶

An image-capable text deployment reports both text and image input modalities. Message content may be an ordered array of text and image_url parts. A text part contains type:"text" and text; an image part contains type:"image_url" and an image_url object with url and optional detail. Detail is auto by default, or low or high, with interpretation owned by the model's source policy.

Image URLs must be base64 data URLs. The server does not fetch network URLs or read image paths from the request. Image-enabled builds always accept static PAM (image/x-portable-arbitrarymap) without third-party codecs. PNG, JPEG, GIF, and WebP are enabled by default and can be omitted individually without removing image input or other serving modalities. PAM supports standard black-and-white, grayscale, and RGB tuples, with optional straight alpha and sample maxima up to 65535, normalized to RGB8 or RGBA8. Animated or multi-image inputs, MIME/signature mismatches, and images exceeding source-declared byte, dimension, or pixel limits reject. The normal HTTP request-body limit still applies.

Multiple images and concurrent requests can share projection cohorts. Features remain device-resident through language input; a disconnected peer does not change the surviving images' association. Full conversation history remains required for retained reuse. Image identity includes the compressed bytes, ordered placement, detail and resolved source metadata, so reuse cannot cross a change in those semantics. A complete retained media prefix skips pixel decoding and projection; it does not require a separate image-feature cache.

Models with temporal source policy accept the vLLM pre-extracted JPEG sequence convention through video_url. A request can interleave video, images, and text:

{
  "model": "your-vision-model",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "video_url", "video_url": {
        "url": "data:video/jpeg;base64,<first-JPEG-base64>,<second-JPEG-base64>"
      }},
      {"type": "text", "text": "Describe how the scene changes."}
    ]
  }],
  "media_io_kwargs": {"video": {
    "fps": 100,
    "frames_indices": [0, 75],
    "num_frames": -1,
    "do_sample_frames": false
  }}
}

Each comma-separated payload is a complete JPEG image. Frame timestamps are frames_indices[i] / fps, giving 0 and 0.75 seconds in this example. fps defaults to 1 and must be positive and finite; omitted indices default to consecutive integers starting at zero. Explicit indices are nonnegative, nondecreasing integers with exactly one entry per supplied frame. Repeated indices are valid. Timing metadata applies to every video in the request.

This subset consumes client-selected frames. Set num_frames: -1 to consume all supplied frames; with the field omitted, clips are limited to 32 frames, matching vLLM's default without silently discarding additional frames. Positive num_frames and do_sample_frames: true are unsupported. Optional total_num_frames must be positive and at least the supplied frame count; duration must be finite, nonnegative seconds. These describe the original clip and do not resample the selected sequence.

Each frame counts toward the source-declared image input limit and uses the existing compressed-byte and geometry bounds. The model policy owns temporal grouping, spatial resizing, timestamp tokens, and positional encoding. Remote URLs, MP4/WebM containers, and other media-I/O options are rejected. Frame input uses the JPEG codec and requires no video-container decoder.

Retained conversations¶

Untagged requests can reuse a matching completed conversation still resident in an idle execution row. The server selects the longest reusable canonical history, including exact ordered image identity, and processes only the new suffix. This works without configuring extra cache or checkpoint capacity. Active requests and named sessions are not candidates for this automatic row selection. Reuse does not assign a persistent session identity to the client.

X-Loom-Session: SESSION requests best-effort reuse of a named history. Requests still supply full history. On an ordinary cache miss the server can replay that history; a session name is not a durable chat database. Streaming and JSON turns can alternate without changing the continuation contract.

Usage reports reused tokens in prompt_tokens_details.cached_tokens. Edited history, source-ineligible completions, and evicted state replay normally. An idle row advances in place; retaining an older branch point requires an explicit checkpoint. session_cache_capacity retains additional named histories beyond execution rows, while prefix_cache_capacity captures shared intermediate input prefixes. Both remain separately configured limits.

Prefix caching retains exact state at natural prefill boundaries. An unused cache slot can also retain a later boundary near the prompt's end, reducing replay when a client drops generated reasoning or changes a later message. This extension never evicts another prefix. New branches and memory pressure can still reclaim entries, and a one-slot cache retains its first matching ancestor. Reuse depends on exact token and media identity, not message labels; cached_tokens reports the prefix actually used.

X-Loom-Checkpoint: NAME requests an explicit immutable endpoint for the destination session. It is not best effort: an unknown name returns 404 and a history mismatch returns 400 before replacing the destination continuation. Session and checkpoint names are scoped to the selected deployment, so equal names on two models do not share state.

Named checkpoints¶

The deployment needs nonzero checkpoint_capacity. Control requests carry X-Loom-Model: scout; this selector may be omitted only when exactly one deployment is registered. Names contain 1–64 ASCII letters, digits, underscores, dots or hyphens. They are identifiers, not authentication tokens.

Request with empty body	Effect
`POST /v1/checkpoints/NAME` with `X-Loom-Session: SESSION`	Pin an idle completed session; HTTP 201 reports its consumed token position.
`POST /v1/checkpoints/NAME/suspend`	With elastic backing, park the endpoint in host memory; HTTP 200 reports `host_snapshot_bytes`.
`DELETE /v1/checkpoints/NAME`	Release the explicit pin; HTTP 200 after accepted release.

Absent sessions or checkpoint names return 404; busy sessions, duplicate names or queued borrowers preventing deletion return 409. Exhausted pin capacity returns 503. Fixed backing rejects suspension with 400.

Deletion remains available while inference admission is drained, including when inference requests are waiting. It uses the normal bounded checkpoint control slot. An unprepared registration returns 404 without preparing the model. Pinning and suspension wait for inference admission to reopen.

Pin completion publishes an immutable consumed frontier and the exact pending token. The next completion still sends full history extending the endpoint's canonical transcript. Another destination session forks that endpoint; the same destination rewinds it. This supports keeping a tool-output summary while discarding the verbose tool text from subsequent context.

Suspension moves the endpoint's placement without resetting live branches. Its host image is process-local, not a file or durable disk cache. Resuming warms it into new pool IDs. On unified-memory hardware, host storage does not create additional physical RAM. Explicit pins stay protected until released; automatic idle caches may yield under pressure.

Accepted pin/suspend work survives client disconnect through actual readiness. Deleting a pin does not invalidate active branches that already own independent references. An impossible reservation or refused restore returns overload without silently replacing the destination's continuation.

Known-token sequences¶

POST /v1/sequences accepts a deployment with kind:"sequence". The caller supplies exact known IDs; the server does not tokenize text, select an extra token, or advance beyond those IDs. Each successful request streams SSE.

curl -N http://localhost:8080/v1/sequences \
  -H 'Content-Type: application/json' \
  -d '{"model":"encoder","position":0,"token_ids":[11,27,42],"options":{}}'

An initial request omits session and has position:0. The attached event returns a server-issued handle. Each chunk contains that handle, its consumed position and next_position, exact token_ids, and opaque source bytes in data with encoding:"base64". Chunk boundaries follow actual execution cohorts, not the HTTP request's token count. A final done event publishes the completed position. For example:

event: attached
data: {"session":"0123456789abcdef0123456789abcdef","position":0}

event: chunk
data: {"session":"0123456789abcdef0123456789abcdef","position":0,"next_position":3,"token_ids":[11,27,42],"encoding":"base64","data":"AQIDBA=="}

event: done
data: {"session":"0123456789abcdef0123456789abcdef","position":3}

The bytes above are illustrative; their shape, dtype, and interpretation belong to the model package. Append with the returned session, the exact completed position, and only the new token_ids. options is an optional source-defined object. Unknown, duplicate, or wrongly typed fields reject; IDs are nonnegative int32 values and the array must be nonempty. Context and shared pool limits apply before admission. X-Loom-Session is not accepted on this endpoint: identity is the session body field.

A stale position or concurrent use of the same handle returns 409. A handle that has been evicted, unregistered, or invalidated returns 404. Handles survive execution-row reuse while retained in the model's session cache; they are not durable across registration replacement or server restart.

The existing checkpoint controls can pin a completed sequence, using its handle as X-Loom-Session. To fork, send X-Loom-Checkpoint: NAME, omit session, and supply the checkpoint's consumed position with the new IDs. The response receives a new handle. Sequence checkpoints have no implicit pending prediction.

Completion means local device retirement and output acceptance, not peer acknowledgement. A disconnected in-flight append invalidates its handle after accepted work retires; an independent pinned checkpoint remains usable. A transport failure after the final response was accepted can be ambiguous to the caller. The endpoint is not an exactly-once retry protocol: recover from a known checkpoint or replay the full known history under a new handle.

Output credit is bounded per active row. A slow reader removes only its row from new cohorts; other credited sequences can continue. Bytes are captured before reusable model scratch is overwritten and downloaded asynchronously for HTTP encoding.

Speech synthesis¶

POST /v1/audio/speech accepts a speech deployment:

{"model":"speaker","input":"Good day.","voice":"alba","response_format":"pcm"}

The response streams signed little-endian PCM16 at 24 kHz, mono, with Content-Type: audio/pcm;rate=24000;channels=1. Completion closes the response. Network reads need not coincide with waveform blocks; consumers retain any partial sample between reads. Disconnection suppresses further generation while already accepted work retires.

Requests have independent voice, progress and output credit. Compatible next operations may share numerical batches, including requests arriving after generation has begun. A stalled reader excludes its own request from further work while peers continue. Batch boundaries are internal execution boundaries; they do not add a multiplexing protocol to the client's PCM stream. The streaming walkthrough shows complete requests and playback, and the source contract explains how packages define compatible work.

model, nonempty UTF-8 input, voice and explicit response_format: "pcm" are required. voice is a package-supported identifier or an object containing an id string. Optional instructions, speed and seed reach source request preparation, which rejects controls it cannot implement. Speed defaults to 1 and accepts 0.25–4. The seed extension accepts an unsigned 64-bit integer or decimal string and defaults to zero.

stream_format may be omitted or set to "audio". Encoded formats, SSE and an omitted response format return an input error. Uploaded reference recordings use the separate voice-registration endpoint below. File-based players require an encoded-audio profile rather than raw PCM.

Voice registration¶

POST /v1/audio/voices registers a reference recording with a speech ABI 2 model. Its multipart fields are model, a nonempty UTF-8 id, and file. The file accepts the same WAV, MP3, FLAC, and raw PCM inputs as audio transcription, through the same native parser and decoder. The model determines which recording lengths and geometry it can use.

curl --fail-with-body -H 'Expect:' \
  http://127.0.0.1:8080/v1/audio/voices \
  -F model=speaker -F id=narrator \
  -F 'file=@reference.wav;type=audio/wav'

A successful response is {"id":"narrator"}. It arrives only after CPU audio preparation and learned conditioning have completed; subsequent speech requests can then select "voice":"narrator". Registration decodes the original geometry once, and the model's CPU Loom program owns mixing, resampling, and trimming. Synthesis reuses the resulting conditioning. The server does not write a prepared recording or weight archive to disk.

IDs are local to the loaded model. A duplicate ID returns 409, exhausted voice or admission capacity returns 503, and an unavailable ID returns 404. A service without the upload ABI returns 501. Disconnecting an unfinished upload cancels publication, while accepted CPU and device work finish retiring safely.

Remove a voice with:

curl --fail-with-body http://127.0.0.1:8080/v1/audio/voices/remove \
  -H 'Content-Type: application/json' \
  -d '{"model":"speaker","id":"narrator"}'

Removal hides the ID from new requests. Already admitted requests retain their conditioning until completion, so its slot becomes reusable only after those readers retire. Uploaded voices live until removal or model unloading. A model card can name original recordings to register on every load.

Audio transcription¶

POST /v1/audio/transcriptions accepts a recognition deployment and a completed recording as multipart/form-data. Builds with audio codecs accept WAV, MP3, and FLAC, converting their sample rate and channel layout to the model's input geometry:

curl --fail-with-body -H 'Expect:' \
  http://127.0.0.1:8080/v1/audio/transcriptions \
  -F model=recognizer \
  -F 'file=@recording.wav;type=audio/wav'

The default response is {"text":"Recognized speech."}. response_format=text returns UTF-8 plain text. Independent finalized segments appear in input-time order, separated by newlines; segment identities break equal-time ties.

Field	Contract
`model`	Required registered recognition deployment.
`file`	Required nonempty recording in an enabled format, or raw PCM with explicit geometry; the filename never names a server path.
`language`	Optional UTF-8 language hint interpreted by the package.
`prompt`	Optional UTF-8 context interpreted by the package.
`temperature`	Finite number from 0 through 1, default 0; source policy may restrict it.
`response_format`	`json` by default, or `text`.
`stream`	Omitted or `false` for this finite-upload profile.

Container bytes select the decoder, independently of the filename or MIME label. audio/wav, audio/mpeg, and audio/flac are conventional labels; application/octet-stream also works for a supported container. A disabled codec or unsupported container returns an error. Direct WebM/Opus and MP4/AAC uploads are not accepted; clients must convert them to an enabled format.

Raw PCM16LE and F32LE work in every build. The file part declares audio/pcm with a positive sample rate and the interleaved channel count required by the selected model. The server converts the declared rate to the model's rate. For a 24 kHz mono recording sent to a 16 kHz mono recognizer:

curl --fail-with-body -H 'Expect:' \
  http://127.0.0.1:8080/v1/audio/transcriptions \
  -F model=recognizer \
  -F 'file=@recording.pcm;headers="Content-Type: audio/pcm;rate=24000;channels=1"'

These bytes contain whole PCM16 frames without a WAV header. Add format=f32le to the media type for interleaved little-endian F32 samples; every sample must be finite and between -1 and 1. format=s16le is the default. Missing or duplicate geometry, nonpositive rates, mismatched channel counts, and partial frames reject. Raw PCM uses bandlimited rate conversion with continuous filter history across worker chunks; it does not mix channels. Equal rates preserve each sample. PCM input, its rate conversion, and PCM speech output require no audio codec dependency.

The request-body byte limit and source-defined input-frame limit apply independently. The converted raw PCM frame count is checked before model admission; encoded recordings are checked as decoded frames reach recognition. An overlong recording rejects without truncation or a partial success response. Decoding and conversion stage a bounded number of frames on the shared CPU executor, retaining each recording's codec and resampling history across blocks. Raw conversion preserves the input duration: N frames produce ceil(N * model_rate / input_rate) frames, with filter lookahead flushed only at the recording's end. Conversion storage is bounded per request; a rate ratio that exceeds that budget rejects without reducing filter quality.

Uploads may share numerical batches while retaining independent sample clocks, model histories, and transcript state. A disconnect cancels that request; upload bytes, decoder state, and model storage remain owned until accepted work retires. Concurrent requests continue through their own input and output credit.

Unknown or duplicate fields, unsupported response formats and stream=true reject. The finite profile does not emit provisional hypotheses, word timing or speaker labels. The recognition source contract preserves revisions and optional alignment/diarization evidence for typed consumers. A chunked HTTP upload still completes before recognition begins; it is not an appendable microphone session.

Embeddings¶

POST /v1/embeddings accepts:

{"model":"vectors","input":["a","bbb"],"dimensions":2,"encoding_format":"float"}

input may be one string, an ordered string batch, a token-ID sequence or a batch of token-ID sequences. encoding_format is "float" by default or "base64"; base64 encodes little-endian IEEE754 F32 values. dimensions requests a source-supported width, not arbitrary host truncation. Optional user is a string accepted as client metadata. Unknown fields reject.

The response is an object:"list" with ordered embedding objects, original per-request indices and token usage. Several requests can share one dispatch cohort without sharing response identities. Source programs own tokenization, pooling, projection, normalization and capacity. Cancellation suppresses that client's response without invalidating peers in the cohort.

Image generation¶

POST /v1/images/generations generates one image at the requested size:

{"model":"illustrator","prompt":"a red fox in the snow","size":"1536x1024","seed":"0","response_format":"b64_json"}

The response has one data[0].b64_json PNG. Only n:1 and response_format:"b64_json" are supported. Optional size is WIDTHxHEIGHT; omitting it uses the deployment's configured width and height. The model's source validates supported geometry. Sizes are exact, never silently rounded. A new shape can compile asynchronously on demand while sharing resident weights; later requests reuse its code within the deployment's specialization-cache limits. Larger images require more work and memory, and a first request can include compilation time. Request sizes do not change the deployment default. seed defaults to zero and accepts an unsigned integer or decimal string.

Optional loras selects adapters registered with the deployment, with an independent strength for each:

{"model":"illustrator","prompt":"a red fox in the snow","loras":[{"id":"watercolor","scale":0.3},{"id":"ink","scale":0.7}]}

This example requires registered watercolor and ink adapters. Each id is a nonempty string, not a path or URL to load. scale is a finite F32 number, defaults to one, and may be negative or zero. Omitting loras, or supplying an empty list, selects the base model. The list preserves order and repeated IDs, including a separate strength for each occurrence. Model request preparation resolves those names against the deployment's registrations. Selections affect only that request, including when it shares a batch with other requests.

The Krea 2 Turbo model guide shows a complete deployment using original adapter checkpoints. Krea rejects unregistered names even at zero strength, shares prepared factors across requests, and removes registered zero-strength occurrences before capture. Floating-point accumulation follows selection order, so deterministic replay uses the same ordered list.

This endpoint is text-to-image; it does not accept reference-image uploads or imply image-editing support. Unknown fields reject.

Finite decisions¶

POST /v1/systemone accepts a decision deployment, string/object/array state, and a nonempty map of typed questions:

{
  "model": "judge",
  "state": "The customer cannot log in.",
  "questions": {
    "route": {
      "type": "choice",
      "instructions": "Choose a team.",
      "criteria": {"support":"Account access","billing":"Payments"}
    },
    "urgency": {"type":"score","instructions":"Rate urgency.","criteria":["Low","Medium","High"]},
    "needs_reply": {"type":"noul","instructions":"Is a reply needed?"}
  }
}

Each question requires a type and instructions. Instructions accept a string, object or array. Choice has 1–255 named options and returns the winning ID plus all probabilities. Score has 2–10 ordered levels and returns an expected zero-based score, probabilities and a legend retaining typed criteria. Noul returns the probability of true. Question and option identities are preserved. The native bound is 1024 questions; source capacity can be smaller.

The source produces normalized F32 vectors. The host validates and encodes them; it does not silently normalize or truncate malformed outputs. Usage reports source input tokens and zero output tokens: this is finite evaluation, not autoregressive generation. Schema/source-input failures return 422. Strict JSON is required; comments, trailing commas, malformed Unicode, duplicate decoded IDs, unknown fields and image-input extensions reject.

Count a chat before generating¶

POST /v1/chat/completions/input_tokens accepts the same JSON body as chat completions and returns one {"input_tokens":N} response. It applies the deployment's source-defined chat template, tool catalog, thinking options, generation framing, and image-position accounting. It counts the complete fresh input, including positions that a later generation might reuse from cache.

With a Chat Completions body saved as request.json, use the same bytes for counting and generation:

curl http://localhost:8080/v1/chat/completions/input_tokens \
  -H 'Content-Type: application/json' --data-binary @request.json

The count runs on shared CPU workers without loading weights, reserving an execution row or KV capacity, projecting images, or changing retained history. It can progress while every generation row is occupied. It shares the model's source-policy invocation, so it waits for an existing CPU borrower to retire; it does not create another VM or block a worker waiting for model capacity. The normal model routing and drain policy still applies.

The request's generation fields remain validated, but stream:true still returns one JSON count and no output budget is reserved. Source input is bounded by the deployment's context capacity; an oversized input rejects instead of returning a truncated count. A successful count is not a promise that input plus the requested completion budget will fit, or that execution memory is available. Omit X-Loom-Session and X-Loom-Checkpoint: this stateless endpoint rejects them rather than selecting cached history. The count queue uses pending_capacity; a full queue returns 503. Utility requests appear in request inspection and outcome counters, but never increment inference-token counters.

Raw tokenization¶

POST /tokenize uses the selected deployment's loaded tokenizer:

{"model":"scout","content":"Hello!","add_special":false,"parse_special":true}

It returns {"tokens":[...]}. The shown special-token options are defaults. POST /detokenize accepts {"model":"scout","tokens":[...]} and returns {"content":"..."}, preserving special tokens. Neither operation applies chat templates or model-specific prompt framing. Empty text and arrays are valid.

These CPU utilities do not activate weights, occupy execution rows or wait for an inference epoch. They still obey model admission and retain their tokenizer borrow through actual task completion. An unknown model or absent tokenizer returns 404. Unknown, duplicate, wrongly typed or invalid-token fields return 400. A tokenizer execution failure returns 500 for that request.

Both utilities accept complete long-context inputs without a separate token or decoded-text quota. The server's configurable --request_body_bytes transport limit applies to their JSON requests; exceeding it returns 413. Allocation or tokenizer failures never return a truncated success. with_pieces:false is accepted; true rejects because this decoder contract does not expose raw token bytes. prompt, add_special_tokens and chat-tokenization fields are not aliases for this profile.