A disaggregated KV-cache inference stack for large language models on AMD Instinct GPUs. Combines vLLM with LMCache to offload KV tensors across GPU VRAM, CPU DRAM, local NVMe, and NFS-over-RDMA.
Full catalogue of every metric exposed by the AIC monitoring stack — node exporter, NVMe exporter, RDMA exporter, and the vLLM/LMCache inference exporters. Generated nightly from live scrape data.
Nightly KV-cache cliff benchmark results across all three storage arms: VRAM-only, NVMe (GDS), and NFS-over-RDMA. Tracks throughput and latency trends over time.