Author a model package for loom-serve

A serving package combines a model's numerical programs with its input, parameter and state contracts. The server supplies shared workers, JIT, device queues, memory ownership, request scheduling and HTTP encoding. Adding a model that fits an existing modality changes the package, not the server's C code or executable.

The first-output walkthrough runs a complete arithmetic package before weights or neural accuracy enter the picture. That is a useful starting point for a port: changing its math preserves a working path through registration, execution, output and retirement.

Select the execution contract

The important distinction is the shape of state and output, not the model name:

Contract	Source responsibilities	Starting package
Retained text	Token framing, paged state geometry, mixed prompt/decode math, progress, sampling and chat policy.	Tiny text model
Staged image	Prompt preparation, parameter domains, numerical stages, invocation continuation and final RGBA8.	Image model
Finite embeddings	Input preparation, cohort cost/compatibility, pooling/projection and final vectors.	Embedding model
Finite decisions	Structured question framing and compact normalized answer distributions.	Decision model
Streaming speech	Voice conditioning, independent request state, compatible next-work batches and continuous waveform blocks.	Speech model
Continuing recognition	Independent input clocks, compatible next-work batches and revisable transcript segments.	Recognition model

These packages use actual source VM programs, native kernels and production HTTP services, not canned native replies. They establish the serving contract, not learned quality or competitive neural performance. Each learned port needs its own numerical oracle and representative workload.

Image-conditioned text composes retained text with an optional finite image projector, described below. Audio input or an arbitrary workflow graph needs its own input/output ownership contract beyond these HTTP profiles, not an embedding vector with overloaded fields. The HTTP reference defines the existing boundary.

Keep the package complete and relocatable

A typical package contains:

model.json           deployment, source roles, assets and capacities
prepare.loom         cold declarations and parameter interpretation
control.loom         warm request and execution policy
config.loom          model/target specialization inputs
kernel.loom          numerical providers and reusable helpers
tokenizer.json       tokenizer asset, when needed
weights.safetensors  checkpoint selected by preparation policy

These filenames are conventions, not hardcoded server paths. The model document names the actual files. GGUF and safetensors inputs follow the selected preparation policy; a file extension or model name does not convert an arbitrary checkpoint. Its keys, encodings and layouts must agree with the authored source.

All program roles and numerical providers accept .loom or .loombc. Bytecode is portable IR and still JITs for the selected device. References in a file document resolve from its directory, including references to shared motif libraries elsewhere. JSON-only HTTP registration supplies its absolute server-local base. A multipart source package instead carries programs and metadata in a separate logical namespace while parameter files stay server-local. The live development workflow shows upload, native selection and retained-state comparisons.

definition.sources supplies shared VM and native providers. Roots and referenced symbols determine dependency closure through the normal Loom linker. Reusable helpers take SSA values or template arguments; model-specific configuration belongs at program/kernel boundaries. This keeps a motif usable without importing another model's global configuration namespace.

Preparation and control VM files each link against that catalog as primary inputs. Ordinary function declarations and template applications resolve to exported library definitions; library-private helpers stay private. This lets preparation, control and numerical commands share layout calculations without duplicating them across files.

Explicit func.decl import("module") declarations remain capability imports, bound to native modules when the VM program loads. The cold program names numerical roots; the warm program invokes the resulting prepared stages. Shared source helpers do not turn device kernels into direct VM calls.

Reuse Unicode properties

Host text programs can use the string module's codepoint.* operations instead of embedding Unicode tables in each model. They are available in loom-serve, loom-test, and loom-benchmark VM execution. Each operation accepts one Unicode codepoint; full case mapping handles expansions such as ß → SS:

func.decl import("string", "codepoint.to_upper") @unicode.to_upper(%codepoint: i32) -> (i32, i32, i32, i32)

func.def public @uppercase(%codepoint: i32) -> (i32, i32, i32, i32) {
  %count, %first, %second, %third = func.call @unicode.to_upper(%codepoint) : (i32) -> (i32, i32, i32, i32)
  func.return %count, %first, %second, %third : i32, i32, i32, i32
}

codepoint.to_upper returns a count and three codepoint slots; codepoint.to_lower returns a count and two slots. Unused slots are zero. Both use locale-independent Unicode 15.1 mappings, preserve unmapped input, and allocate no buffers. codepoint.is_decimal, codepoint.is_digit, and codepoint.is_whitespace each take one i32 codepoint and return i32 zero or one. Digits include decimal digits and characters such as ²; whitespace means Unicode White_Space. Model-specific spacing, punctuation, and capitalization policies remain in the authored program.

Separate preparation from execution

The cold prepare VM program receives modality inputs and the source-owned options object. It validates architecture constraints and declares stages, configuration and checkpoint domains. Shared preparation capabilities include:

Import in module prepare	Meaning
stage(root, tag)	Declare a command-program root and a source-defined selection tag.
dispatch(root, tag)	Declare a buffer-only native kernel root with its compiled launch configuration.
config(stage, key, value)	Bind a public Loom configuration value in textual syntax.
config_i64(stage, key, value)	Bind a signed integer configuration value.
workspace_limit(stage, bytes)	Bound command workspace, including compatible live replacements.
memory_forecast(root)	Select a finite component's host control entry for complete-request memory estimates.
source(directory, path)	Declare the first shard of a checkpoint and return its source ordinal.
source_append(source, directory, path)	Append a shard to that checkpoint's ordered index.
source_scope(scope)	Select a model card's named, typed parameter files as one shared checkpoint source.
parameter(stage, binding, input, source, policy)	Bind a checkpoint to one policy-local input of a reflected fixed parameter binding.

A split checkpoint uses source for its first file and source_append for each additional shard. The files populate one index in declaration order. The same source can supply several resident bindings or stages without repeating the file list. Sources identify bytes, independently of resident placement or preparation policy; a source declaration by itself loads no parameter payload. Cold preparation opens each declared source once and shares its index and imported-file cache across consuming bindings. Request choices retain the cold prefix and index new sources on the preparation worker; warm execution does not reopen checkpoint paths. A missing or invalid checkpoint fails preparation before its parameter placement or activation.

Each fixed binding declares its policy inputs once, densely from zero. A policy can combine independent checkpoints containing the same keys: its weights imports use those local input ordinals, not model-wide source ordinals. Stages naming the same ordered file lists and policy share a resident domain. The loader accepts explicit GGUF, safetensors, and other supported parameter files without inferring filenames.

Weight policies can combine checkpoint ranges with values computed during preparation. The import weights.input(source, key, key_offset, key_length, offset, length, alignment) captures a checkpoint range; weights.input_buffer(data, alignment) captures an immutable copy of a nonempty readable VM buffer. Both append bytes to the current tensor's input recipe and return their aligned offset within it. Computed bytes survive policy teardown and are reused if device parameters are reactivated; they do not create a file or another checkpoint visible through weights.count or weights.parameter.

A policy that declares inputs without a preparation command loads those bytes directly into the final tensor. Its ranges must cover the complete tensor contiguously, with no alignment gaps, and the policy returns the tensor's reflected byte length. This supports checkpoint slices, concatenated ranges from multiple sources, and computed host data without a staging buffer or a copy kernel. A policy declaring neither inputs nor a command returns zero and loads the canonical key from source zero.

When the policy declares a preparation command, input offsets address its staging buffer, and alignment gaps are allowed. The command receives the final tensor followed by staging, and the policy returns the tensor's reflected byte length. Without captured inputs, a command transforms the canonical source-zero bytes in place. Only commands with captured inputs contribute to staging capacity.

Both weights.parameter(source, key, key_offset, key_length) and weights.input resolve the named byte range of key during the call. A policy can build names in reusable scratch storage and pass only the used range. The captured entry and returned metadata refer to the retained checkpoint index; subsequent writes to the key buffer cannot change them.

Request preparation can declare additional sources after the immutable cold source prefix, or bind that prefix to new stages. It cannot append shards to a cold source. Each request choice owns its declarations independently; selecting another choice does not change the checkpoint identity of accepted work.

The exact signatures and ownership are in the preparation contract. Declarations copy their strings; opaque returned state must own its storage or retain its inputs. Preparation is not a place to submit device work.

The server resolves these declarations, compiles through its shared compiler task queue, and retains the selected stages for the model. Configuration can specialize stage shapes or layouts without giving every session its own program. Compatible requests share the prepared code and parameters. Warm work does not invoke a compiler subprocess or redo unchanged model preparation.

The warm control VM imports the runner-local runner.execute_N and runner.feedback capabilities. execute_N submits a prepared stage with its buffers; feedback transfers a bounded final result into a registered host slot. Both enqueue work without waiting. The native model owner schedules completion on work/feedback timelines and preserves buffers until their actual last use. The runner-module contract defines arity, buffer types and lifetime. This is a coarse serving capability, not a source program's ownership of the entire device or process scheduler.

Independent work can use explicit dependencies. runner.fence(work) returns a hal.fence for a previously accepted work value. It captures that producer, not later submissions or the separate feedback branch. runner.join(ready) orders subsequent model work after the supplied fence without waiting on the CPU or synchronizing other models. A null fence represents no external work. Completion describes readiness; storage ownership still extends through the last accepted consumer, including when a dependency fails.

Measure device work

Control programs can bracket prepared stages with queue-clock captures:

(frequency_hz, valid_bits) = runner.timestamp_clock()
runner.timestamp(feedback_buffer, begin_byte_offset)
runner.execute_N(stage, ...)
finished = runner.timestamp(feedback_buffer, end_byte_offset)
runner.feedback(slot, feedback_buffer, byte_offset, byte_length)

Each capture writes a raw 64-bit tick to an 8-byte-aligned, transfer-writable destination. Its return value is an accepted work timepoint, not the captured tick. Captures follow preceding model work, and subsequent work follows the capture. They enqueue without a host wait or a device-wide barrier. Returning the samples in the existing feedback transfer needs no separate readback.

timestamp_clock reports the dispatch queue family's clock. A zero frequency or width means that no timestamp clock is advertised; unsupported captures return an error. Within that family, subtract samples modulo 2^valid_bits and divide by frequency_hz to obtain elapsed seconds. A captured zero is valid, and successful completion establishes that the sample was written. Each queued epoch owns separate sample slots until its readers retire; a later capture must not overwrite feedback that is still in flight.

Read checkpoint bytes on demand

Large lookup tables and paged experts can remain in checkpoint files instead of occupying a resident parameter binding. The warm parameters capability reads demanded byte ranges from the same sources declared during preparation:

parameters.gather(source, key, wait, descriptors, descriptor_offset, count, target)
    -> hal.fence

source is the selected invocation's source ordinal, and key is a host byte buffer naming an entry in that checkpoint. Independent sources may use the same key. descriptors and target are hal.buffer references. Starting at descriptor_offset, each 24-byte record contains three little-endian u64 values: the entry-relative byte offset, destination-buffer byte offset, and byte length. Destination ranges are disjoint; zero-length records require no payload transfer. Hashing, deduplication, packing, and cache placement remain model-program decisions, not tensor interpretations in the server.

A control program can enqueue independent reads before joining either one:

producer = runner.execute_N(descriptor_stage, ...)
produced = runner.fence(producer)
first = parameters.gather(source_a, key_a, produced, ranges_a, 0, count_a, target_a)
second = parameters.gather(source_b, key_b, produced, ranges_b, 0, count_b, target_b)
runner.execute_N(independent_stage, ...)
runner.join(first)
runner.execute_N(first_consumer, ...)
runner.join(second)
runner.execute_N(second_consumer, ...)

The supplied fence covers descriptor production and destination availability, including any earlier readers of reused storage. The returned fence orders payload consumers and descriptor reuse. An empty gather preserves its incoming dependency without touching storage. Changing a request's source choice or removing its model does not redirect or release accepted reads: each read retains its original provider and buffer captures until actual retirement. Failed readiness is not permission to recycle those buffers.

This implementation asynchronously snapshots the descriptors to host memory, then submits the ranges through the parameter provider and HAL file queues. The VM call never waits for completion or synchronizes unrelated models. It requires queue download and deferred host-call support. Checkpoint opening uses the configured parameter-file mode; direct I/O requires appropriately aligned file ranges and destinations and does not silently fall back to buffered reads. A snapshot costs 24 bytes per descriptor plus one copied key; checkpoint payloads are not materialized in a separate model-owned host array.

This is host-VM-initiated asynchronous I/O, not a persistent GPU request service. A kernel that requests data while continuing other work needs a device-side progress and credit protocol beyond this call. The shared source identity and byte-range contract do not impose a model-wide barrier on that separate execution mechanism.

Tokenize request inputs

Request preparation can use the input capability to encode text and decode token spans with the deployment's tokenizer. encode(text, flags, capacity) returns a zero-padded little-endian I32 ID buffer and live token count; it can stop at the requested capacity. decode(ids, flags, byte_capacity) consumes the complete ID buffer and returns a zero-padded UTF-8 buffer and live byte length. An insufficient decoding capacity fails instead of returning partial text. Pass only live IDs, excluding encoding padding. Each call starts fresh tokenizer state, including leading-space handling. Encoding flags are ADD_SPECIAL_TOKENS = 4 and NO_SPECIAL_TOKEN_MATCHING = 8; decoding supports SKIP_SPECIAL_TOKENS = 1, with zero retaining special tokens. The source owns text normalization, sentence boundaries, and capacity admission.

Trace one finite request end to end

The embedding package is the smallest complete example. Its cold entry is prepare(asset_root, options). It returns a batch-unit limit, upload/output/ result capacities, an optional tokenizer asset, a source name and opaque state. Its control program exports:

prepare_request(state, input) -> (request_state, units, compatibility_tag)
prepare_batch(state, stage_tags, batch: ptr<finite.batch>)
    -> (stage_index, upload_payload, result_plan)
run(state, stage_index, input_buffer, output_buffer, batch: ptr<finite.batch>)
collect_batch(state, batch: ptr<finite.batch>)

The HTTP parser provides indexed protocol records, not a source-private JSON guess. prepare_request performs model framing and validation. units is a positive additive bound on real work/storage; it need not mean tokens. Equal compatibility tags mean requests can share a cohort. Each request retains its source buffer across epochs; the model shares one serialized control instance. Returned buffers retain any aliased input.

The native scheduler starts with the oldest prepared request and fills the cohort from compatible waiting requests. It does not wait for hypothetical arrivals. The finite capability exposes the entire selected cohort to one preparation or collection call:

Import	Contract
batch_count(batch) -> i64	Number of selected members.
batch_state(batch, ordinal) -> buffer	Retained request state at its original cohort ordinal.
batch_pack(batch) -> (ranges, packed)	Explicitly copy states into an upload directory and payload.
batch_origin(batch, ordinal) -> i64	Reserved cohort origin in source-defined storage units.
batch_buffer(batch, ordinal, root) -> hal.buffer	Stable device root whose capture protects the cohort until queue retirement. Submission only.
batch_release(batch, ordinal)	Relinquish a completed cohort's storage during collection while its logical request continues.
batch_offer(batch, ordinal, units, tag)	Publish the next work offer during successful collection. Zero units finishes the request.
batch_finish(batch)	Finish every member of a one-shot cohort during collection.

The borrowed batch is valid only during that call. Cancellation preserves its original ordinals until collection retires. batch_pack produces request-order [offset, length] I64 ranges; the packed buffer contains the same directory followed by the states. A one-shot program can upload it directly. A continuing program can instead read each state and construct a compact control packet. Batch preparation selects a stage and maps physical output back to requests.

After the accepted device frontier succeeds, collect_batch commits progress and publishes each member's next offer. It never runs for failed readiness. Unfinished members rejoin behind new arrivals, so compatible late requests can join the next epoch. Each member can publish an ordered prefix of its remaining result rows. Collection finishes a member exactly when that prefix exhausts its remaining rows. Earlier rows retain their independent output storage while the source releases and reuses numerical cohort scratch. Empty output alone does not indicate completion. A one-shot source calls finite.batch_finish from collection and uses this same lifecycle.

Request-state buffers are host control storage. Device intermediates need an explicit producer-owned lifetime across epochs: native generators own that storage and its retirement. Compiler workspace inside a source run invocation is scoped to that invocation.

Finite prepare_batch can also declare stages with the prepare capability. It uses the same stage, dispatch, config, config_i64, and workspace_limit imports as cold preparation. Returned ordinals follow the cold stages and are valid for that invocation. This lets a package specialize an actual image geometry or cohort extent without enumerating a size catalog at registration. A request can also declare sources and parameter bindings; omitting parameter declarations preserves the cold layout. New source ordinals follow the immutable cold prefix and belong to that exact request choice.

The server reuses an exact declaration set within its selected source generation or compiles it asynchronously before parameter admission. Other models keep running on the shared workers. Cancellation joins accepted compilation before releasing its source and invocation, and cached choices are bounded by the deployment's specialization_cache. Neither specialization nor image-to-text composition requires downloading hidden features to the host.

The result plan describes physical output bytes separately from the live result prefix, then each request's rows and each row's offset/width. Results retain their device storage independently of the producing model. A logical request spanning cohorts keeps its rows in the original buffers and publishes one final result in input order. Device consumers import the producer's readiness and read those rows without a consolidation copy or host roundtrip. HTTP consumers explicitly download and retain compact snapshots until final encoding; device padding and activation storage need not be downloaded. Projection and normalization happen in the numerical program. Decision encoding similarly consumes final probabilities, not hidden activations.

The example deliberately reverses physical output rows. Correct client indices therefore prove that result mapping works rather than merely agreeing with a convenient device layout. Its independent arithmetic oracle also makes ragged packing and cohort association observable. See the finite-source contract for exact record layouts and ownership.

Retained text and staged images

Text uses one prepared model and cached stage set. Sessions are rows and storage references, not independent VM processes. Its source declares page/recurrent geometry and consumes packed spans that can mix prompt tokens, decode anchors and speculative verifier work. Token capacity and independently advancing span capacity are separate. The source reports completed progress; native ownership commits it after readiness.

Resident capacity grows independently of those epoch shapes. Source control exports resize_state(control_state, resident_capacity), returning replacement control state followed by the same five storage records used at preparation. Existing private views, recurrent slots, and KV planes retain their byte offsets; only the resident directory, its tables, and append-only storage grow. The native owner joins accepted model work before growth. A task evaluates source layout and allocates host session records, then the server owner submits native growth and publishes the replacement after its queue frontier completes. Other models remain schedulable throughout preparation and the asynchronous device wait. Programs and weights remain loaded. Virtual backing remaps the same physical slabs; fixed backing copies retained bytes at this cold boundary and temporarily requires both allocations. Unused virtual capacity does not acquire physical memory until request admission needs it.

Shared prefixes and explicit checkpoints keep immutable endpoint references. A branch needs a private recurrent writer without copying the entire KV prefix. The tiny text package exercises a partial-page frontier, an unconsumed pending token, fork/rewind, cancellation and full-replay equivalence. Those boundaries matter more than getting one unretained prompt to emit plausible text. The text service links the storage, epoch and chat-policy declarations beside their owners.

Cold image preparation selects stages and parameter domains for default geometry. Warm source receives the requested height and width alongside prompt, seed, and decoded adapter ID/scale records. It resolves IDs against its registrations, validates geometry, and returns opaque request bytes with work and compatibility facts. Compatible requests form one cohort; source batch preparation selects a cold stage or declares stages through the same invocation-only prepare capability as finite cohorts. Exact declaration sets reuse code within the selected source generation; misses compile before parameter admission. Explicit checkpoint declarations select retained banks or prepare independent banks on demand; omitted declarations preserve the cold parameter layout. Stages within one cohort share a parameter binding map.

Image control then drives command parts to final RGBA8. It returns a continuation cursor; another invocation occurs only after the previous accepted frontiers retire. Compiler-reflected workspace remains live across barrier-aligned parts. RGBA8 output and compiler-aligned workspace have separate cohort-sized pool allocations. Final feedback retains the output after workspace retirement. Each request keeps a stable output slice even if a peer is cancelled. Neither storage nor pixel geometry belongs to the executable cache. Final output is row-major RGBA8: four bytes per pixel in R, G, B, A order. Models perform their final quantization on device; native PNG encoding preserves those bytes, including alpha, using each request's dimensions. The image source contract defines entry signatures, alignment and cancellation.

Declare reasoning support

The text control program exports text_chat_reasoning_v1(), returning three I32 values: whether reasoning is supported, a bitmap of selectable efforts, and the effort ordinal used when a request omits controls. Preparation calls this once and validates the declaration before publishing the model. The server copies the resulting facts into discovery; it does not use them to interpret requests or modify sampling.

Effort bits and ordinals are none=0, minimal=1, low=2, medium=3, high=4, xhigh=5, and max=6. Source advertises the model's distinct policies, not every accepted alias. A selectable ladder contains at least one reasoning level, and its default must be in the bitmap. Non-reasoning models return (0, 0, 0); reasoning models without an effort selector return (1, 0, 0). For example, (1, 45, 0) advertises none/low/medium/xhigh with reasoning off by default. The HTTP profile describes the discovery fields.

Tool-call translation

Text policy owns the generated grammar and canonical history. Its parse_tools(schemas: buffer, response: buffer, truncated: i64) -> buffer root returns function records with a name and an opaque argument string; generic chat handling assigns IDs and encodes the same records for HTTP and SSE. The server and model policy do not execute tools, enforce tool permissions, validate executable arguments, or repair calls. Those operations belong to the harness. Native admission passes model-specific request fields through without imposing a tool-choice or template-option vocabulary. Source policy consumes the fields needed for its own prompt, sampling, and output representation. render_tool must preserve the call data when formatting the next turn, and complete_text can authorize retained state reuse only when its canonical bytes match the generated history. truncated is one when generation reaches its token limit without EOS, and zero after EOS. A truncated response can retain complete calls while omitting an unfinished final call; its finish reason remains length, not tool_calls. The original response reaches complete_text, including any omitted suffix, so canonical history cannot authorize reuse of different generated bytes.

A source program can import tools.parse_xml(schemas: buffer, text: buffer, truncated: i64) -> buffer for the <tool_call><function=NAME><parameter=NAME>VALUE</parameter>…</function></tool_call> grammar. This is an optional codec, not a model-wide protocol requirement. It validates framing and UTF-8, preserves duplicate parameter members in order, and uses property schemas only to distinguish representations:

Hint	Argument representation
String, including string-capable unions	Literal text, even when it looks like a number or JSON.
Explicit nullable hint with null text	JSON null.
Non-string hint with a complete JSON value	Original JSON spelling, without numeric conversion or type enforcement.
Unknown property/tool, repeated tool name, opaque schema, or non-JSON text	Literal string.

Hints cover direct types, nullable type arrays, and anyOf/oneOf branches; they are not a JSON Schema evaluator or reference resolver. One framing newline on each side of a parameter is removed; other string whitespace, JSON-looking text, and XML entities remain literal. With truncated=1, an unfinished grammar prefix at the end is omitted, and preceding complete calls are retained. No closing syntax or argument data is invented. Malformed framing still fails the complete call list without publishing a prefix; with truncated=0, unfinished framing fails too. Missing required arguments, unknown names, and schema-invalid values are representable calls for the harness to handle. Other grammars can return argument strings directly, including malformed JSON, provided their source history formatter can retain those strings faithfully.

Known-token stages

A sequence deployment uses retained rows, packed epochs, block pools, and checkpoints without the generative chat contract. It consumes only supplied known IDs and publishes source-defined opaque output. It can implement an encoder or another retained stage; a source may also run a head and include its result bytes explicitly, without asking the server to sample or feed them back.

Preparation retains the text model's nine-result ABI. The terminal-token marker may be null. Control exports epoch, encode_epoch, publish_epoch, encode_state, resize_state, bind_request, request_defaults, prepare_request, and capture_epoch. Isolated step, tokenizer/chat roots, and drafting are not required. Every submitted span has selection/proposal flags zero, and completed progress contains no native output-token selection.

prepare_request(defaults, request, nonce, tokens) validates one complete sequence request before row reservation or response streaming. tokens is a read-only, reference-counted buffer of parsed little-endian nonnegative i32 IDs; its byte length is four times the token count. The source checks vocabulary and other model-specific input constraints here and returns its options buffer. An input error rejects only this request with its diagnostic; retained histories and other requests remain available. Validation covers the complete append, including tokens beyond the first scheduled epoch. This call receives no row state and must leave retained execution state unchanged. Source may retain the token buffer in its returned options. Generative text's request_options retains its three-argument contract; its source produces tokens in prepare_input.

Text and sequence plans are compact arrays of nine little-endian u32 fields per span. The first epoch's records occupy [0, first_count); optional continuation records immediately follow at first_count. A record's resident state ID is independent of its plan ordinal. The continuation-flags argument to publish_epoch is a buffer of little-endian u32 flags indexed by first-plan ordinal, not a bitmask indexed by state ID. Source programs size their tables and results from the requested resident and cohort capacities; neither coordinate has a built-in sixteen-entry limit.

publish_epoch receives the retained control-state buffer as its first argument, followed by the plan and completed feedback. It runs only after successful epoch completion and can update source-owned history for subsequent epochs, such as measured execution costs. The source owns this history's layout and carries it through control-state growth. Submitted device work uses its already-captured inputs; publication does not change those inputs or the completed feedback banks.

encode_epoch receives the selected first-epoch and continuation token capacities as first_packet_capacity and next_packet_capacity. These are the prepared shapes, independent of live token counts and whether the inputs contain media. The continuation capacity is zero when no continuation is submitted. Source can use these values to lay out packets and associate captured state with the prepared programs that consume it.

encode_epoch writes metadata and input packets, then returns (output_count, transfer_plan, byte_offset, byte_length). The plan view contains little-endian u64 records {source_binding, source_offset, target_binding, target_offset, length}. The source and target tables each bind metadata at ordinal zero and input at ordinal one. Source and destination offsets are independent: compact host data can populate a sparse device layout without transferring its holes. Only listed bytes are uploaded: preparing capacity for a large packed epoch does not make a one-token epoch transfer that entire capacity. Source initializes all bytes its numerical stages read. Nonempty records are ordered by target binding and offset, with no overlapping targets. Repeated source ranges are permitted. Records describe independent placements, not an ordered sequence of copies.

The plan names logical storage, not pointers, registrations, or completion handles. The execution owner binds physical resources and establishes readiness. The plan can reuse retained control state and is consumed during submission; packet bytes stay immutable until accepted transfers retire, including failure. Source reuse, destination visibility, and destination reuse after consumption are separate ownership boundaries. See the byte-placement contract for the encoding and validation rules. The same contract applies to generative text and media epochs. See epoch uploads for local HAL binding and payload ownership.

capture_epoch(control_state, shape_index, span_count, plan) returns a buffer of little-endian i64 triples: [span_ordinal, residual_byte_offset, byte_length]. Ranges are grouped in submitted span order, with at least one nonempty range per span. Multiple ranges concatenate in source order. The plan is checked and copied before row mutation; output bytes are copied after the producer and before the next epoch can overwrite the shared residual. See capture ownership for device views, readiness, and explicit host materialization.

Captured output owns its pooled storage and execution domain independently of the model, its weights, and retained rows. A downstream device consumer imports readiness and retains its view through queue retirement. HTTP uses a compact host snapshot and the sequence profile. Row IDs, request-relative input offsets, and packed activation offsets are different coordinates: the source must derive capture ranges from its actual packed layout, not treat a row or input offset as a physical output address.

Stream speech with independent requests

A speech package can share weights and numerical execution among clients while retaining each request's voice, cache history, RNG progression and playback position. Start with the speech walkthrough for a complete HTTP caller and a package whose output is independently checkable.

Cold preparation declares numerical stages and parameter domains, plus resident request capacity, additive batch capacity, input/state strides, conditioning storage, bounded output/feedback storage and waveform geometry. Those declarations separate persistent request storage from temporary batch geometry. A larger model can change its vocoder, diffusion schedule, cache layout or block size without teaching the HTTP service its architecture.

The warm source exports these operations:

Operation	Source-owned result
prepare_request	Validate text, voice, instructions, seed and speed; return retained CPU state and immutable request bytes.
prepare_work	Describe one request's next operation with compact work bytes, positive additive units and a compatibility tag, without advancing it.
prepare_batch	Pack a compatible cohort and retain its plan, using explicit resident-row and byte-range records.
generate	Execute the plan through the prepared runner stages, optionally requesting bounded feedback.
complete	Interpret retired feedback for one original batch ordinal; publish playback position, sample range, frame count and EOF.

The model declaration owns the exact signatures, scalar widths and buffer layouts. prepare_batch receives the prepared stage-tag catalog separately from the packed request work; the source chooses its numerical stage and packing representation.

Speech ABI 2 adds reusable uploaded voices. speech_prepare_v2 appends voice_capacity and voice_audio_capacity to the eleven scalar preparation results, followed by the existing four metadata buffers. The latter capacity bounds decoded original F32 audio bytes. Initial conditioning can populate a prefix of the declared conditioning buffer; the service copies it once.

speech_prepare_voice_v2(model, slot, samples, rate, channels) runs on a CPU worker with interleaved F32 audio at its original rate and channel count. The shared decoder has checked complete frames and finite normalized samples. The program owns model-specific mixing, resampling, trimming, and resource limits. It returns an opaque voice buffer, retained upload state, and immutable request bytes. Learned encoding then uses the ordinary prepare_work, prepare_batch, generate, and complete lifecycle. Upload completion publishes no audio; EOF publishes the voice only after accepted work retires.

speech_prepare_request_v2 receives that opaque voice buffer in place of a voice-ID string. The native service maps IDs to slots and captures each voice at admission, including queued requests. Removing an ID hides it immediately but preserves conditioning for captured readers. A slot has one preparation writer and is reused only after that writer and every reader retire. Programs need neither a name table nor a second readiness protocol. The other roots use the same speech_*_v2 suffix. Speech ABI 1 retains its source-defined voices and rejects uploads.

Consider a request A already decoding when B arrives with another voice and a longer prefix. B first runs its required conditioning and prefix work. At a retired boundary, the service queries ready requests again. If A and B offer equal compatibility tags and their total work fits, they can share the next batch while retaining distinct cache positions. If their current phases need different numerical work, they advance in separate cohorts. Compatibility is a fact established by the package's kernels; the service cannot infer it from two opaque generation calls or from matching output formats.

Five dynamic roots give numerical programs compact batch work, stable request rows, shared conditioning, stable stream rows and packed output. Changing batch membership changes the small work description and row mapping; it does not move entire request histories. Compiler-private workspace belongs to the current command. Persistent convolution tails, recurrent state, KV caches and solver history belong to the request's stream row or explicitly owned conditioning.

One accepted batch retains its original row mapping through execution and collection, even if a client cancels. complete runs for every original ordinal, including phases with no audio or readback. A zero-frame result is independent of EOF. Native serving checks the published range, converts samples to the wire format and copies each result into that request's bounded output storage before releasing shared feedback. A stalled client then prevents only its own next operation from becoming eligible.

Shared conditioning has a publication boundary too. A voice entry becomes reusable only after its producer retires, and its storage remains valid while accepted readers use it. Per-request RNG state must produce the same sequence regardless of neighboring requests or their scheduling. The model owns these numerical invariants; the service owns admission, output credit and retirement.

Qualify a port with staggered requests of different lengths and voices, changing batch membership, a stalled reader, cancellation during an accepted batch and reuse of the released row. Compare each surviving stream with isolated execution and compare the model's numerical stages with an independent reference. Batch occupancy alone establishes neither a throughput improvement nor a playback deadline.

Image-conditioned text

A text definition's optional image component implements the finite contract. It shares the deployment's device and numerical provider catalog, while owning separate preparation and parameter residency. Ordinary text and exact retained image-prefix hits do not prepare the projector. Explicit component warmup follows the same preparation boundary. Its parameter pin policy is independent of the primary language model's policy; deactivation does not discard retained text state.

The text control program declares image limits and receives ordered content records plus a message-local frame table. Each media part names a contiguous frame range; each frame supplies display geometry and a presentation timestamp. An ordinary image has one frame with timestamp zero. The source chooses projection groups within each part, renders their model framing, and returns an explicit group count, marker ranges, projected row counts, and opaque preprocessing metadata. Native code validates complete, ordered coverage of the inspected frames before committing the plan. Input preparation then maps those projections into logical input spans. The ordered-media contract defines these records and retained identity.

On a media-prefix miss, codecs realize packed RGB8/RGBA8 pixels without resizing or normalizing them. The projector receives an image request packet containing a plane directory, encoded/display geometry, orientation, source metadata, and pixels for every frame in the group. Its device program owns preprocessing and projection. Compatible images from the same request or different requests share finite cohorts; source work units and compatibility tags govern packing, not a native assumption of one image per invocation.

Projection identity includes every frame's compressed content and timestamp, its part placement and detail, the selected grouping, and source preprocessing metadata. Retained reuse compares the identity established when the plan was accepted. For example, a model that pads an odd final frame by using it twice must recompute that last projection when a second frame is appended. Earlier complete pairs remain eligible for prefix reuse when their preprocessing and prompt placement are unchanged.

Projected rows remain on the device. The language epoch imports producer readiness and gathers only the rows its current spans need. Source encode_epoch maps those rows and ordinary tokens into its input packet and owns the position information consumed by its numerical stages. The media epoch contract defines the host control records separately from device features. Projector weights and completed features have independent lifetimes; live requests retain features only until their language consumers finish.

With a finite serving memory limit, the projector's cold preparation selects a control entry with prepare.memory_forecast(root). The selected function has this signature:

forecast_memory(state, ranges, descriptors)
    -> (private_peak_bytes, retained_bytes, expanded_bytes)

The three inputs are buffers. The range table uses little-endian i64 offset/length pairs for descriptors containing image headers and source metadata, without decoded pixel planes. The three results are nonnegative i64 byte estimates for the complete logical request: the producer's private peak (parameters, activation staging, input and command workspace), all retained output roots, and gathered output rows with 16-byte row alignment. Retained output is excluded from the private peak and includes padded/shared roots; gather storage includes separate copies of aliased rows.

A logical text request can join one projector cohort shared with peers. Its other cohorts contain only its own projections. The retained-output estimate can therefore include private projection output plus one maximum cohort root, instead of multiplying every projection by the maximum peer count. Compatible frame pairs within a video continue to batch across the request's private cohorts.

These trusted source estimates let the server reject a request that cannot fit alone before pixel decoding or weight activation. It checks producer and language-consumer phases separately. The forecast does not reserve future capacity against concurrent requests; each phase still passes ordinary admission.

Recognize continuing audio

A recognition package accepts a continuing sequence of normalized F32 PCM frames and explicit end-of-input. Input packet boundaries do not pad audio or finalize an utterance. Each request retains its own sample clock, convolution history, encoder state, decoder cache and transcript revisions as batch membership changes. A source can also publish speech activity probabilities, either alone or together with transcripts. Native callers explicitly select supported output types. The recognition deployment role requires transcript support and exposes HTTP transcription. The activity role requires activity support and serves native streams without an HTTP inference endpoint. Both use the same preparation, batching, input credit, and retirement contracts; compound sources can share their weights and execution across different output selections.

The recognition model declaration owns entry signatures and binary layouts. Cold preparation declares stages, parameter domains, sample geometry, bounded input/event storage, recording limits, resident rows and additive batch capacity. Warm source operations are:

Operation	Source-owned result
prepare_request	Validate language, prompt and temperature; receive the selected output types and retain independent request state and immutable conditioning.
prepare_work	Inspect a bounded PCM snapshot with its absolute sample position and EOF flag; offer compatible numerical work or zero units while awaiting input.
prepare_batch	Pack ready work using explicit resident-row and byte-range records, retaining original cohort order.
generate	Submit one source-selected numerical phase without waiting for device completion.
complete	Acknowledge consumed input and publish a bounded ordered packet of typed events, independently of request completion.

Acknowledgement returns input credit only after actual execution retirement. Source code owns any PCM needed beyond prepare_work; the snapshot itself is borrowed only during that call. Every accepted cohort member is collected, including cancelled requests, before shared feedback or request rows can be reused. A stalled event consumer excludes only its own request from new work. Source-defined compatibility permits requests at different sample or token positions to share a batch when their next operations support it.

Cold preparation declares supported output types, maximum packet bytes, and maximum events per completion. A request selects a nonempty subset; an unrequested event is a source error. The completion buffer and its produced byte extent are separate, allowing the source to reuse output capacity across different counts. Native collection copies the packet into reusable request-owned storage before another source call can overwrite it. Events remain independently observable, even when several revise the same transcript segment. Consuming the entire packet returns that request's output credit. Final events can outlive model removal.

An activity event covers a positive interval of actual input and supplies a finite speech probability in [0,1]. When activity is requested, each completion's observations partition its acknowledged input prefix in order, without gaps or overlap. Source programs retain future context as unconsumed input until its observation is established; intermediate numerical phases can acknowledge zero frames. The declared input window includes that lookahead. This lets callers retain bounded pre-roll while the source chooses its own numerical work. Coverage is not an exact onset, offset, or word boundary. Turn thresholds and hangover belong to the caller's policy; source code owns numerical state and EOF handling.

Each transcript segment has a stable request-local identity, increasing revision, covered sample interval and replacement UTF-8 text. Finalizing a segment makes its text and metadata immutable. Request completion requires every published segment to be finalized. A source can additionally provide speaker labels and word alignment; an input-coverage interval alone does not establish where words were spoken. Consumers preserve these distinctions instead of converting a revisable hypothesis into an append-only text delta.

The finite HTTP adapter decodes enabled WAV, MP3, and FLAC containers into the same bounded F32 input used by raw PCM. Encoded input supports channel conversion and filtered resampling on shared CPU workers. Raw PCM also supports filtered resampling without an audio codec dependency, with matching channel counts and an exact rational sample clock; model feature extraction, numerical stages, and transcript policy remain source responsibilities. Each upload owns its codec history independently of source batch membership, and accepted decode work retains that state until retirement. Packet and worker boundaries preserve filter history. Explicit input EOF flushes lookahead without extending the recording's duration.

The adapter selects transcript output, consumes revisions, and returns the completed document. Typed consumers can apply revisions as they arrive and independently supply more PCM. Qualify a model with isolated reference transcripts, staggered arrivals, different recording lengths, a stalled reader, accepted-work cancellation and row reuse. Compound-output sources also preserve event ordering and earlier revisions within a single packet. Compare text and sample clocks across changing cohorts, including a final partial model frame.

Make parameters and temporaries visible

Command stages declare fixed checkpoint domains and preparation policies. Stages sharing a weight bank must agree on reflected placement. Preparation indexes checkpoint metadata; activation streams and transforms weight payloads into final storage. Warm requests reuse those plans. Deactivation can release reloadable backing without discarding compiled code or retained session state.

The embedding package's weighted variant and policy demonstrate adding a checkpoint-backed head without native model code. Comparing the checkpoint's actual shapes/types with the numerical oracle is part of the port; current key/extent checks do not establish model identity.

Private command workspace comes from compiler reflection and queue-ordered allocation. It is not a permanently resident per-session tensor arena. Under elastic backing it draws from the same physical pool and serving budget as parameters and model state. Stable workspace addresses can survive physical trim without retaining the pages. Prepared working-set peaks drive admission; the physical slab size determines the minimum commitment, not each small logical allocation's size. Escaping output remains live through its final device or transfer consumer; copied host feedback survives CPU encoding. A slow network reader receives copied response storage rather than retaining device workspace.

Finite direct-dispatch stages instead take input/output buffers and own their device-local working storage. The resident XDNA example uses this path. These direct stages have no checkpoint-domain binding in the current source contract; weighted direct programs need an explicit source-owned parameter-placement contract, not a guess from native argument reflection. The ordinary weighted command example and unweighted resident example prove different boundaries.

Establish correctness before throughput

Start with the selected package's deterministic output and replace one numerical stage with the intended model operation. Checks can compare functions directly with independent VM oracles; kernel checks specify an appropriate oracle rather than copying candidate output into a golden. Correctness workflows cover functions, cases and scenarios.

A complete port then establishes final outputs through the same server binary using an external package directory. The useful cases include ragged inputs, different request sizes, cancellation during accepted device work, a failed request followed by successful reuse, and unregister/re-register. Retained text also compares append/fork/rewind against full replay. A compilation-only success does not establish these ownership or numerical contracts.

Agent-driven kernel development and compile reports provide the optimization loop once correctness is measurable. Real model shapes, memory traffic and independent error bounds guide layout, traversal, fusion and specialization. Microkernel speed is one input; server observations and end-to-end useful throughput determine whether the change helps the workload.