UMBP Runtime-Tunable Environment Variables
Reference for the UMBP_* env vars consumed by the Mori UMBP stack at
runtime — both the in-process timing/retry knobs parsed by the C++ library
and the deployment knobs read by Python launcher scripts and the
SGLang/hicache integration layer. Not every UMBP_* variable the code
reads is listed yet.
See also:
design-master-control-plane.md— what each knob actually affects.src/umbp/include/umbp/common/env_time.h— parser helpers (GetEnvSeconds / GetEnvMilliseconds / GetEnvMicroseconds / GetEnvUint32).src/umbp/include/umbp/common/config.h::UMBPConfig::FromEnvironment— the env overlay applied to per-processUMBPConfigdefaults.
Resolution semantics
Unset or empty env value → default is used, no log.
Non-numeric value, negative number, trailing garbage (e.g.
"10abc"),uint32overflow, or a value below the parameter’smin_allowedthreshold → default is used, and one WARN per env name per process is emitted on stderr viaUMBP_LOG_WARN.Parsing uses
std::strtollwith base 10. This means:Leading whitespace is skipped (
" 123"parses as123).An explicit sign prefix is accepted (
"+123"OK;"-5"fails the non-negative check on all current params and falls back).Trailing whitespace or any non-digit suffix (
"123 ","0x10") is rejected, falling back to the default.
Every production call site caches the resolved value in a function-local
static const autoon first use (distributed/SPDK-proxy consumers).ClientRegistryConfig::FromEnvironment()/EvictionConfig::FromEnvironment()themselves do not cache, but theMasterServerConfigthey produce is built once at master startup, so the net effect is the same: env changes after first touch have no effect within the same process. To exercise a different value, fork a fresh binary.std::getenvand the logger are NOT async-signal-safe. First use must happen on a normal thread, not inside a signal handler.
When master starts, bin/master_main.cpp prints one line
[Master] Resolved timing: ... after MasterServerConfig::FromEnvironment()
so operators can audit the effective values.
Master / client registry
Read by the master process (bin/master_main.cpp via
MasterServerConfig::FromEnvironment()).
Env var |
Default |
Unit |
Description |
|---|---|---|---|
|
|
sec |
Registry entry TTL; client is evicted if no heartbeat arrives within |
|
|
sec |
Reaper wake-up period inside |
|
|
count |
Consecutive misses before a client is considered dead. |
|
|
sec |
|
|
|
sec |
Master-side read-lease length granted by |
|
|
count |
Recommended client heartbeat interval = |
|
|
ms |
Per-call gRPC deadline applied to outbound |
|
|
bytes |
Most bytes one eviction round frees from one |
|
|
ms |
How long a peer retries an eviction demotion that failed in a way that can clear (no space at the target, e.g. an SSD staging arena full of reads, or a copy/commit error) before it drops the evicted copy so the tier still frees. Counted from the first attempt. |
|
|
sec |
External KV hit-count entry TTL. A hash with no counted match for longer than this is removed from the hit index. |
|
|
sec |
External KV hit-count GC sweep interval. |
|
|
count |
Maximum hashes accepted by one |
|
|
enum |
Base RoutePut placement algorithm over eligible node entry pools. |
|
|
enum |
Node-affinity bias layered on top of the base algorithm. |
|
|
count |
Number of independently-locked, key-hashed shards backing the block-location index inside |
Peer / pool client
Read by each pool client process (typically an SGLang/vLLM worker
that has loaded libmori_umbp_pybinds.so, i.e. imported mori.umbp).
Env var |
Default |
Unit |
Description |
|---|---|---|---|
|
|
enum |
How a node with a master evicts. |
|
|
ms |
Read lease of a DRAM or HBM backend: how long a key’s pages stay protected from local eviction while a peer reads them over RDMA. Only needs to exceed one DRAM RDMA round trip (sub-ms), so 500 ms is ~100x margin. Read once per process; |
|
|
ms |
Read lease of an SSD backend. It has to cover an SSD read plus the RDMA, not just the RDMA, and it is also how long a staging page stays claimed, so it sets the backend’s read concurrency against its fixed staging arena: shortening it frees pages sooner and costs read coalescing. Read once per process; |
|
|
ms |
Overall deadline for retrying a peer-local or remote batch resolve that returned |
|
on |
bool |
Check-after-read for remote gets (#749). The peer’s read lease ( |
|
|
ms |
Deadline for |
|
|
sec |
|
|
|
ms |
Cadence at which the pool client’s |
|
|
port |
Port the standalone server serves |
|
|
ms |
Cadence of the periodic activity summary: one INFO line per window saying what this process offloaded, loaded, evicted, promoted and demoted, plus what is resident. Runs in every deployment — master, masterless, embedded — because it is the readout for the runs that have no Prometheus at all (a ctest, a one-shot benchmark, an sglang rank being bisected). |
|
|
count |
Number of unshipped |
|
empty |
path |
Records successful production PUT/GET calls in the benchmark trace format. Empty disables recording. |
|
|
id |
Logical client stream written into production trace events. |
|
|
seed |
Payload seed stored in the trace header for deterministic replay. |
|
|
count |
Minimum gRPC sync poller threads on the peer service. The default pool is small, and every peer RPC that stages SSD bytes holds one of those threads for the whole device read — on a loopback read the pool, not the drive, is the binding constraint. Same head-of-line-blocking fix as |
|
|
count |
Maximum gRPC sync poller threads on the peer service. Clamped up to |
|
|
bool |
Back the SSD staging arena with hugetlbfs pages. Every byte the SSD backend moves crosses that arena twice (device <-> arena, arena <-> wire), so its TLB behaviour is on the critical path in a way an ordinary buffer’s is not. Falls back to 4 KiB pages when no hugetlb pages are free — a node must still come up. Standalone server binary only; the SGLang path uses the pybind |
|
2 MiB |
bytes |
Hugepage size requested for the SSD staging arena. |
|
|
bytes |
Size of the registered host arena backing ranged (sub-object) multi-buffer I/O, and the entire opt-in for the feature: at |
|
|
bool |
Back both ranged scratch arenas with hugetlbfs pages. Each arena is ONE NIC memory region whose translation entries are its pages, and NICs cap entries per region, so hugepages let an arena grow far larger. Needs |
|
|
bytes |
Hugepage size for |
|
|
count |
How many shards a ranged scratch arena is cut into, counted over the whole arena (at most 64), each leased by one call at a time, so that many remote ranged calls stage concurrently; a shard is |
|
|
bool |
Lay the ranged scratch arenas out by NUMA node: each arena’s bytes are split evenly over the nodes that have GPUs, each part bound to its node and registered on its own, and a remote ranged call stages through the part on its caller buffer’s node (a GPU’s node for device memory). Its RDMA leg then runs on that node’s NICs and the copy into the GPU stays on its socket; total memory is unchanged. A node whose part cannot be allocated on it falls back to one arena per direction. |
|
|
enum |
The one medium this node serves when no backend policy is given: |
SPDK proxy
Read by the spdk_proxy daemon (for intervals it emits) and by the
pool client process via SpdkProxyTier (for stale / poll checks).
Env var |
Default |
Unit |
Description |
|---|---|---|---|
|
|
ms |
Threshold after which the SHM-header heartbeat is considered stale. Consumed by |
|
|
ms |
How often the proxy daemon’s |
|
|
sec |
Period of dead-channel reap + |
|
|
sec |
Sleep before detach when |
|
|
ms |
Yield step used by writeback / batch-drain busy waits. |
|
|
ms |
Max time |
|
|
ms |
Daemon self-exits after this much idle time with zero active sessions. |
|
|
ms |
Grace period before forcibly reaping an inactive tenant. |
|
|
bool |
Set non-zero to enable proxy write-back caching. |
|
|
bytes |
Per-tenant SHM data-region quota. |
|
— |
MB |
SPDK ring buffer size in MB. |
|
|
KB |
RAID strip size when constructing a SPDK RAID bdev across multiple NVMe controllers. |
UMBPConfig overlay
UMBPConfig() (and its alias UMBPConfig::FromEnvironment()) overlays
these on top of the struct defaults; fields the caller assigns afterwards
win. Set them before constructing the config — they are read once, there.
Env var |
Default |
Description |
|---|---|---|
|
4 GiB |
|
|
|
DRAM tier eviction watermarks. |
|
|
|
|
64 KiB |
Allocation granularity of the pool an embedded deployment gets when the config names no deployment at all. The pool is paged, so a value smaller than a page still occupies one: a caller storing KV pages should set this to the page’s exact byte size, and one storing small values wants it smaller. Ignored once |
|
|
POSIX backend root(s). Comma-separated for multi-drive, one directory per physical drive (e.g. |
|
32 GiB |
|
|
4 GiB |
|
|
|
|
|
|
Compute checksums on write, verify on read. Off isolates how much of the SSD path is integrity work rather than storage — the DRAM tier does no checksumming, so an unqualified comparison conflates the two. Records written with it off carry |
|
|
Worker threads for the CPU-bound phases inside one drive’s batch (checksum verify on read, checksum + record assembly on write). Matches the DRAM tier’s default so a DRAM-vs-SSD comparison is not silently 4 threads against 1. |
|
|
Worker threads for the multi-drive fan-out. |
|
|
Coalesce concurrent reads of the same key: the first requester touches the drive, the rest are served by memcpy from its buffer. Exists for MLA + TP, where every attention-TP rank GETs a byte-identical key and one page is otherwise read |
|
|
|
|
unset |
Diagnostic. Prints a stage-by-stage |
|
|
|
|
|
Forwarded to |
|
(empty) |
|
|
(empty) |
SPDK bdev name (e.g. |
|
|
SPDK reactor CPU mask. |
|
|
DPDK hugepage limit (MB). |
|
(empty) |
NVMe PCI BDF (e.g. |
|
|
SPDK NVMe controller name. |
|
|
Internal I/O worker threads for |
|
|
SHM segment name. |
|
|
Tenant id for this client. |
|
|
Per-tenant quota, |
|
|
Channel count. |
|
|
MB of SHM data region per channel. |
|
(auto) |
Path to the |
|
|
Auto-spawn the proxy daemon if not already running. |
|
|
Allow tenants to borrow capacity from the shared pool. |
|
|
Reserved shared bytes that cannot be borrowed. |
Deployment / launcher env vars
Not parsed by the C++ library directly. These are consumed by the
SGLang / hicache wrappers and src/umbp/scripts/run_umbp_single_node_hicache.sh
to construct the
UMBPDistributedConfig plumbed into the C++ side. Listed here so
operators can find them in one place.
Env var |
Description |
|---|---|
|
|
|
JSON backend/tier policy used by a distributed-backed standalone server. |
|
|
|
|
|
Path to the |
|
This node’s address as advertised to peers. Must be reachable from every other node. Required only with a master: with none, nothing registers and no peer can dial in, so |
|
|
|
IO engine port (single port, or comma-separated list for multi-engine deployments). |
|
Port |
|
|
Testing
tests/cpp/umbp/distributed/test_env_time.cppcovers the parser helpers (default / valid / empty / non-numeric / trailing garbage / negative / below-min / zero-when-allowed / uint32 overflow / multiple independent names).Business-path tests that require exercising multiple values of the same env within one test suite must
fork— the function-localstatic constcaches cannot be reset mid-process.CI environments that export any
UMBP_*globally must strip those variables before running UMBP test targets, otherwise the first test to touch a given name will freeze the CI-injected value for the entire process.