Research Map & Progress — Spring 2026

A semester closing report across two PhD problem lines — agent-kvcache and MAS_comm — plus a standalone line, inference_improvement, that supplies the cost model the first depends on.

agent-kvcache · inference_improvement · MAS_comm 6 GPU types · SWE-smith replay Every status traced to repo evidence Spring 2026

1. One-Page Snapshot

The semester ran two PhD problem lines (agent-kvcache, MAS_comm) and one standalone supporting line (inference_improvement), which supplies the per-step cost model agent-kvcache's scheduler consumes and is not a sub-direction of it.

3.3–4.7×
Continuum JCT vs FCFS, JPS≥3
2.95×
CPU-offload @VRAM pressure
301K
per-step rows; max-form rejected
~98%
prefix-cache hit (Continuum)

Honestly read, the semester splits into validated work (the four-policy scheduling campaign, the CPU-offloading study, the trace tooling, a paused-but-validated continuum sweep) and open work (the inference_improvement predictor — a real result on 301K rows but on an unmerged branch with stale tables and a failing self-test — and the MAS_comm KV-communication goal, which still trades accuracy for marginal speed). Two named items, a JitServe baseline and a SimAI study, do not yet exist and are reported as not-started.

Evidence discipline: every status and number here was checked against the repositories this term; maturity labels reflect what reproduces on disk, not what a commit claims.

2. The Research Map

The map below is the spine of this report. Solid lines are the project hierarchy (three research lines plus side lines); the orange dashed lines are the connections that matter — one topic feeding another, or two topics that are the same underlying problem in two forms. Node colour encodes maturity.

supplies cost model shared core: KV-cache reuse PhD Research Spring 2026 agent-kvcache primary line inference_improvement standalone · supplies cost MAS_comm main line side lines secondary scheduling campaign validated 3.3–4.7× JCT cpu-offloading validated 2.95× @pressure trace-viewer validated 36 traces math-modeling (SLO) Overleaf-only jitserve-baseline not started five-factor ceiling model 989 TFLOP/s host residual S(N) calibrated clock droop validated · #13 n=5 means L2-reuse law validated · #2,#6 4.59% MAPE cross-GPU transfer replayed 1.56× unseen CacheBlend HF port most mature MAS-integrated KV comm not validated 67% vs 92% MAS_comm_wt_vllm no GPU run vllm-continuum paused · validated survey · predictor deepdives active simai_pd_staging not started
validated in progress paused not started / unverified cross-topic relationship

3. How the Topics Connect

The sub-topics are not a flat checklist; they feed one another. Naming the connections explicitly is the point of this report.

4. Line 1 — agent-kvcache primary

This is the primary PhD line and carried most of the semester's effort. The summary below first frames the line as a whole, and the subsections that follow walk each sub-topic from its validated empirical core to its open modeling frontier.

Problem
A multi-turn agent recomputes its whole prompt after every tool call, because the engine evicts its KV cache during the idle gap.
Connects to
Consumes inference_improvement's cost model (§5) and the SLO model (§4.4); absorbs cpu-offloading as a policy; observed via trace-viewer; descended from vllm-continuum.
Progress
Scheduling, offloading, and tooling are validated; the SLO model lives only in Overleaf, fed by the separate inference_improvement line (§5).

4.1 Scheduling campaign validated

The semester's most solid result is a four-policy job-completion-time matrix on real agent trajectories, replayed token-by-token from recorded SWE-smith runs through a vLLM V1 logits processor, with per-step scheduler state captured. It spans four KV policies (FCFS, Continuum, SimpleCPUOffload, LMCache) × six GPU types × five concurrency levels.

A strength worth naming: two early claims were caught and retracted by adversarial re-runs — 'LMCache broken on vLLM 0.19' (a sockaddr_un 107-char IPC-path overflow) and a 'Continuum throttling failure' (a hardcoded pin TTL of 10.0s). The campaign's credibility comes from these corrections.

4.2 cpu-offloading study validated

The offloading sub-line answers one question cleanly: on a VRAM-pressured A100-40GB at JPS=10, SimpleCPUOffload cuts mean job time 169.4s→57.4s (2.95×), and a 128 GB DRAM pool captures 80–90% of Continuum's benefit with no custom scheduler. When HBM is not the bottleneck it is neutral, and an undersized 16 GB pool thrashes and is worse than nothing. Decision settled: offload only under VRAM pressure.

4.3 trace-viewer validated

A single-file 272 KB browser viewer plus a 971-line server, live and serving 36 traces, renders the full per-step scheduler signal (batch classification, KV/DRAM pressure, break reasons, preemption timeline, prefix-hit distribution) — so the scheduling claims are inspectable rather than asserted.

4.4 math-modeling (formal SLO design) Overleaf-only

The formal SLO model lives in an external Overleaf project with zero LaTeX in either checkout, so its objective and maturity cannot be assessed from the repositories and are treated as unverified. What is local is the calibrated latency model of §5 it would build on; the first task next term is to bring the Overleaf model under version control and connect the two.

5. Line 3 — inference_improvement standalone line · feeds §4

inference_improvement is a research line in its own right — a separate repository (203 commits this term) with its own published deepdive series. It does not sit under agent-kvcache; it feeds it: its product is the per-step cost an SLO-aware scheduler must have — predict a decode step's wall-clock duration before dispatching it, on an unprofiled GPU, from datasheet parameters. Every ordering in agent-kvcache's scheduler (§4.1) is only as good as this estimate.

Problem
A scheduler must price a decode step before running it, but a GPU's achieved efficiency is far below datasheet peak and shifts with clock, batch, and cache state.
Connects to
Feeds agent-kvcache's scheduler (§4.1); its formal counterpart is the SLO model in §4.4; published as deepdives #1/#2/#4/#6/#7/#8/#10/#11/#13.
Progress
The clock note and L2-reuse law are validated and published; the unified predictor is accurate on real traces but its committed tables are stale on an unmerged branch; the live serving demo is still PACE-queued.

The most interesting, least settled piece is a per-operator roofline model of one forward step. On 301,366 real per-step rows it statistically rejected the paper's monolithic $\max(T_c, T_m) + T_0$ form on all three datacenter GPUs, reframing the empirical 'additive fits better' as a mechanism: summing each operator's own compute-vs-memory maximum is what produces the additive behaviour.

$$\text{rejected:}\;\; t_{\text{step}}=\max(T_c,T_m)+T_0 \qquad\longrightarrow\qquad \text{lower error:}\;\; t_{\text{step}}\approx \sum_{\text{op}} \max\!\big(t^{\text{op}}_c,\,t^{\text{op}}_m\big)$$
Honest maturity: the result reproduces (max-form R² 0.51/0.52/0.17; the additive form has lower error on each), but on H200 both forms fit poorly (0.17 vs 0.43), so the additive advantage there is low-signal. The work sits on an unmerged branch: committed tables are stale (docs cite 491 cells at R²=0.955/MAPE=14.0; a live re-run gives a larger 12,850-cell set) and the self-test fails (asserts H100 N=32 decode in 20–60ms; predictor returns 13.3ms). The live numbers are better-behaved than the stale docs (an old 92.9% MAPE inflation now ~6.1%) — the issue is documentation staleness, not a broken model. Nothing here should reach a paper until the tables are regenerated and the branch merged.

The five-factor 989-TFLOP/s compute ceiling

The live predictor writes one step as compute + exposed host residual + sampling, with the compute branch summing a per-operator roofline. The achieved efficiency factorizes into a clock factor times a clock-free delivery product (memory separate). The peak is an FMA-count identity — H100 $132 \times 4 \times 512 \times 2 \times 1.83\,\text{GHz} = 989.4$ TFLOP/s — yet a real qkv projection tops ~681 TFLOP/s (0.69 of spec) under DVFS.

$$t_{\text{step}} = \max\!\big(T_{\text{blocks}},\, N_{\text{kern}}\, t_{\text{launch}}\big) + \alpha\, S(N) + t_{\text{samp}}, \qquad \eta_c = \underbrace{f_{\text{clk}}}_{\text{clock}} \cdot \underbrace{\eta_{\text{cluster}}}_{\text{wave}} \cdot \underbrace{\eta_{\text{active}}}_{\text{TC}} \cdot \underbrace{\eta_{\text{duty}}}_{\text{barrier}}$$

The host residual S(N): calibrate to engine steps, not generate()

Because decode steps are short, the non-overlapped host floor often dominates, so $S(N)$ must be calibrated to engine-step semantics, not generate() — a ~12.8× pitfall. The L40S floor was corrected from a stale 21.0 ms to 5.08 ms; the slope is ~0 on all four datacenter SKUs.

$$S(N) = s_{\text{floor}} + s_{\text{slope}}\,\max(0,\, N-1), \qquad \alpha = 1$$

Recalibrated $s_{\text{floor}}$ (ms): H100 2.37 · H200 3.73 · A100-80 2.81 · L40S 5.08; held-out decode MAPE 1.3–1.4% on Hopper/Ampere; the measured floor is ~47% of the eager 8B step.

The clock is not a constant

A camera-ready note makes the SM clock first-class: on the RTX 5090 (n=5) cap causally sets clock ($r=0.97$–$0.99$), clock sets throughput ($r\approx 1.00$), and a clock-lock collapses a 390 MHz spread to ~15 MHz (M=64: 114.82→99.49 TFLOP/s). An explicit clock factor cuts compute-bound error ~10× (36.7%→3.5%). On datacenter parts the chain is observational only (PACE denies clock-pinning).

Validated per-SKU droop (M=16384, n=5 means): A100-80 20.0% · H200 26.2% · H100 30.2% · L40S 42.7% · Blackwell Max-Q 38.0%. (An earlier 54.4% used a silicon-max framing and was de-inflated.) Under a tight cap the L40S throughput declines ~−15%; datacenter parts plateau.

Deep dives: The Clock Is Not a Constant (#13) · Clock & Power (#8)

The L2-reuse memory law — and a falsified cliff

The memory factor follows a single-curve L2-reuse saturation law (working-set over L2). As a standalone STREAM probe it collapses onto one curve across three SKUs at 4.59% pooled MAPE. But it does NOT transfer to real engine kernels (43.8% MAPE → live path uses the small-batch value), and a separate residency-cliff law that pre-registered 28.9 MB half-survival on a held-out L40S was honestly falsified out-of-distribution (observed 9–80 MB).

$$\eta_{\text{mem}}(x) = \eta_{\max}\,\frac{x}{x + x_{\text{half}}}, \qquad (\eta_{\max},\, x_{\text{half}}) = (0.90,\ 0.477)$$

Deep dives: Worth the Bytes? (#2, ASPLOS) · L2 Residency (#6, IISWC 2026)

Cross-GPU transfer and the metric-divergence lesson

On an unseen SKU the fit-free predictor (zero target profiling) has lower error than a spec-scaled A100-trained random forest; a profiled forest can't even rank unseen hardware (88–135% without rescaling). On the home A100 the memorizing forest still has lower error (1.14% vs 8.51%) — so the honest win is non-intrusiveness, not home accuracy. Metric divergence: round-1 factors improved Vidur's simulator metric but worsened real wall-clock decode, because Vidur GT cancels per-op errors and omits host overlap. The base predictor (not v2) ships.

$$T_{\text{step}}^{\text{Vidur}} = N_{\text{layers}}\cdot\!\!\sum_{\text{op}} \operatorname{median}_i\, t_{\text{op},i}^{\text{profiled}} \;\neq\; T_{\text{step}}^{\text{wall-clock}}, \qquad \text{factor} = 1.56\times$$
Marked honestly: these utility numbers are replayed/trace-scored, not a live loop — the only live loop (QLM) is mixed/negative (−26pp at high load), and the end-to-end live demo is PACE-queue-stuck. Cross-GPU is scored vs the Vidur metric, not wall-clock. The SKU-selection 100% is partly a tie-break artifact (honest ~53.8% pairwise). Committed R²/MAPE tables remain stale on the unmerged branch.

Full write-ups: Predictor Project Report (#10) · Predictor Experiments (#11) · Fit-Free Transfer (#4, ISPASS 2026) · GPU Datapath (#1) · The Host Term (#7)

6. Line 2 — MAS_comm main

The second main line attacks the same KV-reuse question from the angle of communication between agents. The frame below states the problem and its connection to the primary line, after which each subsection reports one piece — from the mature CacheBlend replication to the open MAS-integrated transfer.

Problem
When agents communicate as text, the receiver re-encodes the sender's words from scratch. Can it inherit the sender's KV cache directly instead?
Connects to
The same cross-context KV-adaptation problem agent-kvcache sidesteps; builds on CacheBlend; the vLLM branch reuses agent-kvcache serving knowledge.
Progress
The CacheBlend replication is mature; the headline MAS-integrated KV communication is implemented but not yet validated; the vLLM branch is coded but unrun on GPU.

6.1 CacheBlend HF port most mature

A HuggingFace port of CacheBlend (arXiv 2405.16444), validated end-to-end on real GPUs, produced a genuine finding: the near-parity is dataset-dependent. On Musique/SamSum, blended reuse matches full prefill (Musique rr=0.50: F1 0.261 vs 0.248), but on WikiMQA it collapses (F1 0.082 vs 0.182), and the paper-default ratio 0.18 underperforms the baseline on Llama-3.1-8B. The replication is partial and honest, not a clean confirmation.

The mechanical foundation is solid: single-agent KV reuse is validated at 3.5× prefill-skip (1.08s→0.31s), 8-bit KV quantization is reversible at 4.0× (MSE 2.6e-2), and a 22-layer round-trip is lossless. The open problem is composing them across agents.

6.2 MAS-integrated KV communication not validated

This is the headline goal of the whole line, and the one piece that does not yet work: the mechanism is in place end to end, but importing a cache verbatim across agents trades a large accuracy loss for marginal speed, so the result below is reported as a measured negative.

This is the line's headline goal and not yet a working result. On a 12-task probe (Llama-3.1-8B), text comm scored 91.7%; importing KV directly scored 66.7% (reuse) and 58.3% (blend) with only ~1.05× warm speedup. It trades a large accuracy loss for marginal speed because caches are imported verbatim — position drift, system-prompt mismatch, and tokenizer misalignment are uncorrected. The unsolved core is the cross-context re-encoding that agent-kvcache avoids.

6.3 MAS_comm_wt_vllm unrun on GPU

A parallel branch wraps the system in a backend-agnostic inference layer (~2,470 LOC, a ModelClient abstraction over vLLM/OpenAI/litellm/HF) exposing vLLM's prefix-caching and KV-transfer connectors, since PagedAttention cannot export per-layer KV like the HF path. The code is complete and unit-tested, but every passing test injects mocks — no real-GPU run, no committed reuse numbers, validation scripts still untracked. The capability is unproven on a live server, and conceptually weaker than CacheBlend's arbitrary cross-context reuse (it reuses only identical prefixes).

7. Side Lines & Not-Started

Beyond the two main lines, a band of secondary work sits at three different stages, and the cards below separate them honestly: one side line is paused with validated results behind it, one is actively producing the published predictor deepdives, and two named items do not exist as projects and are recorded as not-started.

vllm-continuum paused

The original continuum fork and ancestor of the best-performing policy. Validated sweep: Continuum cuts latency 50–74% vs FCFS (H100 JPS=15: 738.8s→271.6s, 15/15 on H100+H200), plus a validated negative (LMCache 4–19% slower on large-VRAM GPUs). Paused 2026-04-08 with A100/L40S cells incomplete.

agentic-ai-survey active

The survey site (489 commits, 114 in June) is where the predictor work is published — the six measurement deepdives (#1–#6) and predictor reports (#10–#12) this report links. infer_labs hands-on pages have drifted uncommitted since 2026-04-14.

jitserve-baseline · simai_pd_staging not started

Reported honestly as not-started. An exhaustive search across lab, PACE, the engine-room fork, and PACE scratch returned zero artifacts for either. JitServe is not even among the four wired baselines, and SimAI / CC_ENABLE=3 exists only as April-dated documentation.

8. Status Matrix

One row per topic: the maturity label that reproduces on disk, last-activity date, and a one-line headline. On a narrow screen the table scrolls sideways within its own frame.

Line Topic Status Last active Headline
agent-kvcache(line)active2026-06-15Primary line; scheduling, offloading, and tooling validated
agent-kvcachescheduling campaignvalidated2026-06-11Continuum 3.3–4.7× JCT; 4-policy × 6-GPU matrix
agent-kvcachecpu-offloadingvalidated2026-06-10Offload helps only under VRAM pressure: −66% / 2.95×
agent-kvcachetrace-viewervalidated2026-05-12Live 272KB viewer + server, 36 traces served
inference_improvement(line)active2026-06-30Standalone line; supplies agent-kvcache's cost model; published deepdive series
inference_improvementclock notevalidated2026-06-30a cap-to-clock-to-throughput causal chain; validated n=5 per-SKU droop (#13)
inference_improvementL2-reuse lawvalidated2026-06-16single-curve η_mem fit 4.59% MAPE; cliff law honestly falsified (#2,#6)
inference_improvementunified predictorin-progress2026-06-30301K rows; max-form rejected; tables stale, branch unmerged
inference_improvementcross-GPU transferreplayed2026-06-151.56× on unseen H100 (Vidur metric, not wall-clock)
agent-kvcachemath-modeling (SLO)in-progress2026-06-15Formal SLO model only in Overleaf; unverifiable locally
agent-kvcachejitserve-baselinenot started—Zero artifacts anywhere; not a wired baseline
MAS_comm(line)in-progress2026-05-29CacheBlend mature; MAS KV integration not validated
MAS_commCacheBlend HF portin-progress2026-05-29Dataset-sensitive: parity on Musique/SamSum, collapse on WikiMQA
MAS_commMAS-integrated KV commin-progress2026-05-2966.7% acc vs 92% text, only 1.05× speedup
MAS_commMAS_comm_wt_vllmin-progress2026-06-19vLLM backend + prefix-cache knobs; no real-GPU run
sidevllm-continuumpaused2026-04-0850–74% latency cut validated; A100 cells incomplete
sideagentic-ai-survey / predictoractive2026-06-30Predictor deepdives active (#1–6, #10–12); hands-on uncommitted
sidesimai_pd_stagingnot started—Absent as project; April docs only

9. Next Semester

The priorities follow directly from the open items above, organised by two themes: stabilize the promising-but-unstable work (chiefly the forward-latency model, which must reproduce its own tables first), and convert the implemented-but-unvalidated work into measured evidence (chiefly MAS_comm's KV communication and the vLLM branch). The six items below are ordered lowest-cost, highest-impact first.

1. Land the forward-latency model

Fix the self-test bound, regenerate the stale R²/MAPE tables, pin the predictor commit, then merge unified-modeling to main. Do not cite the current numbers until they reproduce.

2. Validate the model on agentic traffic

Score the calibrated predictor on agent-kvcache's real SWE-smith per-step features — the first cross-project validation, no GPU needed.

3. Close the MAS_comm accuracy gap

Implement a real cross-context re-encoding hook (RoPE re-application / boundary-token recompute) so KV transfer stops regressing accuracy from ~92% to ~60%.

4. Real-GPU evidence for the vLLM branch

Run the prefix-reuse demo on a live server and commit measured hit-rate and latency-savings numbers.

5. Version-control the SLO model

Export the Overleaf SLO model into the repo, ground it in the calibrated predictor, and cross-validate the fleet simulator against the measured JCT matrix.

6. Decide the side lines

Resume or retire vllm-continuum, commit the drifting infer_labs output, and reconcile the mismatched paper-IDs and PACE paths with the advisor.