docs: CG-15 agent A/B of the #1500 allocation change — gate fails on the control

Three repos, both arms codegraph-on, sonnet/high, 3 runs per arm.

PASS on the two medium repos: client-go (the reporter's Go shape, 2,001 of
2,454 files generated) and excalidraw hold Read 0 in every run of both arms,
excalidraw goes 34s -> 24s at the median with one fewer explore call, and the
generated clientsets/informers that took 10.5%% of a baseline envelope appear
in no new run.

FAIL on express, the small control, in 1 run of 3: 4 Reads and 52s against a
baseline that read once. Not agent variance — replaying that run's query
deterministically, lib/utils.js goes from 6,380 bytes whole to a 583-byte
cluster stub and the envelope shrinks 13.8K -> 9.2K against an unchanged
13,000 budget. The diagnostic shows the allocator was right and the render
loop was not: utils.js is the top-ranked file, was reserved 3,870 chars, and
spent 583. The whole-file bound (allowance + grace = 4,450) lands just under
the file's 5,293 bytes, so the whole-file render is declined and the unspent
reservation is dropped rather than redistributed.

Bar 1 is the hard gate, so per CG-15's acceptance rule the design goes back to
CG-12 — the budget is not to be widened to compensate. Two candidate fixes are
written up in the design doc.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

docs/benchmarks/explore-allocation-ab-1500.md

@@ -0,0 +1,192 @@
+# Agent A/B — score-proportional explore allocation (#1500 / epic CG-1)
+
+**Date:** 2026-08-04 · **New:** `feature/CG-1` @ `edce18f` · **Baseline:** `main` @ `49c11fc`
+· **Harness:** `scripts/agent-eval/ab-new-vs-baseline.sh`, `RUNS=3`, `--model sonnet --effort high`
+on every arm · **Both arms codegraph-on.**
+
+This is the epic's pass gate. The deterministic probes (CG-6/CG-14) prove the budget moved;
+only an agent A/B proves the agent stopped reading.
+
+**Verdict: the gate does not pass.** Bars 2 and 3 hold; **bar 1 (Read stays at 0) and bar 4
+(no regression on the control) fail on express**, with a reproducible, non-agent cause. Per
+the CG-15 acceptance rule the allocation design goes back to CG-12 — the budget is *not* to be
+widened to compensate. Root cause and the smallest honest fix are in [§Root cause](#root-cause).
+
+---
+
+## Method
+
+`ab-new-vs-baseline.sh` builds and indexes each arm separately (CG-5's generated-file flag is
+an index-time decision, so each arm must index with its own build), pre-warms a persistent
+daemon per run, and runs the same flow question 3× per arm. Both arms run with
+`CODEGRAPH_NO_PROMPT_HOOK=1` — the machine's ambient front-load hook resolves to whatever is
+in `dist/` and would inject context through a second, uncontrolled channel.
+
+Each prompt names codegraph as the lookup tool. That is **not** a forced-Read-0: the agent
+stays free to Read whenever explore's answer is insufficient, which is exactly what bar 1
+measures. It removes the one noise source that would otherwise swamp the signal — in a
+pre-run without it, one express run made **0 codegraph calls and 3 Reads**, measuring adoption
+(an axis this change does not touch) rather than allocation.
+
+Envelope share is measured with `parse-run.mjs --envelope --answer <glob>`, which parses the
+rendered markdown of the responses the agent actually received. The CG-4 diagnostic sidecar
+only exists on the new build, so it cannot measure the baseline arm; the markdown parse is the
+only instrument that measures both arms the same way.
+
+| Repo | Lang | Files | Generated | Tier | Role |
+|---|---|---|---|---|---|
+| `kubernetes/client-go` | Go | 2,454 | 2,001 | medium (2 calls / 28K) | **the #1500 shape** — generated CRUD beside hand-written machinery |
+| `excalidraw/excalidraw` | TS/React | 672 | 0 | medium (2 calls / 28K) | god-file concentration (`App.tsx`, 450 KB) |
+| `expressjs/express` | JS | 147 | 0 | small (1 call / 18K) | **control** |
+
+---
+
+## Results
+
+`explore` = codegraph_explore calls · `Read` = Read tool calls · `answer%` = share of the
+source envelope going to the files that answer the question. Three runs per arm, reported as
+the range — run-to-run variance is large and a single run means nothing.
+
+### client-go — "how does a shared informer keep its cache in sync and deliver events?"
+
+Answer set `tools/cache/**`; generated set `kubernetes/**`, `listers/**`, `applyconfigurations/**`,
+`informers/**`, `**/fake/**`.
+
+| arm | explore | **Read** | duration | answer% | generated% |
+|---|---|---|---|---|---|
+| **new** | 2 / 2 / 6 | **0 / 0 / 0** | 39–61s (med 50) | 74.2 / 96.9 / 82.6 | 4.0 / 0.0 / 5.2 |
+| baseline | 3 / 2 / 3 | 0 / 0 / 0 | 48–52s (med 49) | 78.1 / 97.1 / 71.3 | 10.5 / 0.0 / 5.1 |
+
+No Read in either arm. The medians overlap: on this query `tools/cache/**` already dominates
+graph relevance, so the baseline concentrated well without help. The #1500 signal is visible
+but small — the generated clientsets and per-resource informers
+(`informers/events/v1beta1/interface.go`, `kubernetes/typed/events/v1/fake/…`) take 10.5% of
+one baseline run's envelope and **never appear in any new run**.
+
+### excalidraw — "how does updating an element re-render the canvas on screen?"
+
+Answer set: `mutateElement.ts`, `App.tsx`, `renderer/**`, `scene/**`, `components/canvases/**`.
+
+| arm | explore | **Read** | duration | answer% |
+|---|---|---|---|---|
+| **new** | 2 / 3 / 2 | **0 / 0 / 0** | **23–32s (med 24)** | 74.4 / 67.8 / 84.7 |
+| baseline | 3 / 3 / 4 | 0 / 0 / 0 | 33–39s (med 34) | 74.1 / 64.0 / 79.5 |
+
+The clearest win: **29% faster at the median with one fewer explore call per run**, no Read in
+either arm. Concentration is why — the new arm resolves the flow in 2 calls where the baseline
+takes 3–4. (One baseline run burned a turn on a hallucinated `codegraph_..._explore` tool name;
+counted as-is.)
+
+### express — control — "how does res.send decide Content-Type and ETag?"
+
+Answer set `lib/**` (both arms deliver 100%; the whole answer lives in `lib/`, so this repo
+tests concentration *within* the answer set, not against noise).
+
+| arm | explore | **Read** | duration | answer% |
+|---|---|---|---|---|
+| **new** | 1 / 4 / 2 | **0 / 4 / 0** | 18 / 52 / 24s | 100 / 100 / 100 |
+| baseline | 2 / 2 / 2 | 1 / 1 / 1 | 23 / 28 / 27s | 100 / 100 / 100 |
+
+Two of three new runs are strictly better than every baseline run (0 Reads vs 1, faster). The
+third is the failure: **4 Reads of `lib/utils.js` and 52s**, a fallback the baseline never
+made. It is not agent variance — see below.
+
+---
+
+## Root cause
+
+Deterministic replay of the divergent run's own query on both builds, same index, no agent:
+
+```
+codegraph explore "res.send Content-Type ETag generateETag setETag" --path <express>
+```
+
+| file | baseline | new |
+|---|---|---|
+| `lib/utils.js` (5,293 B, 272 lines) | **6,380 (46.1%) — whole** | **583 (7.7%) — cluster stub** |
+| `lib/response.js` | 3,935 (28.4%) | 6,001 (64.9%) |
+| `lib/application.js` | 1,607 (11.6%) | 2,532 (27.4%) |
+| `lib/express.js` | 1,927 (13.9%) | — |
+| **source envelope** | **13,849** | **9,241** |
+
+`lib/utils.js` is where `compileETag`, `createETagGenerator`, `etag` and `wetag` live — half the
+answer. The CG-4 diagnostic on the new build:
+
+```
+envelope 10,295 delivered · 10,292 allocated of 13,000 budget
+allocation 12,398 reserved of 12,400 pool · cliff at weight 10.00 · nothing cliffed
+ #  deliv%   bytes  reserved  score   flags                render     file
+ 1    5.7%     583     3,870   56.0   named entry central  clusters   lib/utils.js
+ 2   57.0%   5,868     5,875   91.4   entry                clusters*  lib/response.js
+ 3   23.0%   2,369     2,653   34.5   entry central        clusters*  lib/application.js
+```
+
+`utils.js` is the **top-ranked** file (score 56.0, named + entry + central) and was **reserved
+3,870 chars — and spent 583 of them.** Nothing was cliffed. The allocator did its job; the
+render loop threw the reservation away.
+
+The mechanism is the whole-file bound at `src/mcp/tools.ts:4008`:
+
+```ts
+const WHOLE_FILE_MAX_CHARS = allowance + Math.min(WHOLE_FILE_GRACE_MAX,
+                                                  round(allowance * WHOLE_FILE_GRACE_FRACTION));
+```
+
+= `3,870 + min(800, 580)` = **4,450** < the file's 5,293 bytes, so the whole-file render is
+declined. Pre-CG-12 the bound was `maxCharsPerFile * 3` = 11,400, which the file cleared
+comfortably. The fallback cluster render only has 3 matched symbols to work with, so it emits
+583 chars and **3,287 chars of the reservation are simply lost** — which is also why the whole
+response shrank from 13.8K to 9.2K against an unchanged 13,000-char budget.
+
+This is CG-12's own acceptance criterion — *"no file that was previously unclipped becomes
+clipped"* — failing, and here it is the direct cause of an agent Read. CG-14 recorded one
+instance of it (`memory-budget.ts`) as a documented exception; this is the same defect
+observed in the wild, where it costs a round-trip.
+
+**Not systemic.** On both medium repos the render loop saturates (`[over budget] [TRUNCATED]`,
+23,599 of a 23,600 pool reserved), so there is no unspent budget to lose. The failure needs a
+file whose proportional reservation lands *below its own size* while its matched-symbol set is
+thin — likelier on small repos, where per-file reservations are smallest.
+
+### The fix belongs in CG-12, not here
+
+Per this task's acceptance rule the budget must **not** be widened to compensate. The defect is
+that a reservation can go unspent, so the fix is one of:
+
+1. **Let a large-enough reservation buy the whole file.** If `allowance >= k * fileSize` for
+   some k < 1 (utils.js: 3,870 / 5,293 = 0.73), render whole and let the bounded overshoot the
+   ceiling already tolerates absorb it — the bytes were reserved for this file anyway.
+2. **Redistribute what a file cannot spend.** After the render loop knows a file's realised
+   size, hand the shortfall to the next-ranked file instead of dropping it. This also fixes the
+   shrinking-envelope symptom directly.
+
+(1) is the smaller change and matches the observed shape; (2) is the more complete invariant
+("the pool is spent"). They compose.
+
+---
+
+## Bars
+
+| # | Bar | Verdict |
+|---|---|---|
+| 1 | **Read stays at 0** | **FAIL** — client-go 0/0/0 and excalidraw 0/0/0 both arms, but express run 2 makes 4 Reads the baseline never made, from a reproducible non-agent cause |
+| 2 | Correct-file share > 50% | **PASS** — new 67.8–100% across all 9 runs; self-query fixture 16% → 59.9%. (Caveat: on these three repos the *baseline* was already above 50%; the ~16% figure is the self-query fixture, not these repos) |
+| 3 | No wall-clock regression | **PASS at the median** — excalidraw 34s → 24s, express 27s → 24s, client-go 49s → 50s. The 52s express outlier is the failing run |
+| 4 | No regression on the control | **FAIL** — express, 1 run of 3 |
+
+Bar 1 is the hard gate and it fails, so the epic does not pass on this measurement regardless
+of bars 2 and 3.
+
+## Reproduce
+
+```bash
+# clone fresh (never eval on a private repo), index, then:
+RUNS=3 AGENT_EVAL_OUT=/tmp/ab-express \
+  scripts/agent-eval/ab-new-vs-baseline.sh <express> "<question>" main
+
+node scripts/agent-eval/parse-run.mjs /tmp/ab-express/run-new-2.jsonl --answer 'lib/**'
+
+# the deterministic core of the failure, no agent needed:
+CODEGRAPH_EXPLORE_DEBUG=1 node dist/bin/codegraph.js \
+  explore "res.send Content-Type ETag generateETag setETag" --path <express>
+```

docs/design/dynamic-dispatch-coverage-playbook.md

@@ -278,6 +278,28 @@ Status legend: ✅ done+validated · 🔬 hole identified · ⬜ not started.
 (Verify the exact supported set against `src/extraction/languages/` and
 `src/resolution/frameworks/` before starting — this table is a starting point.)
 
+### Retrieval A/Bs that are not coverage work
+
+Coverage decides whether a flow *exists* in the graph. A second class of change decides
+whether the answer explore returns is *sufficient* — how the byte envelope is divided across
+the files it found. Same pass bar (Read → 0, no wall-clock regression), same `--model sonnet
+--effort high` rule, but the harness is `ab-new-vs-baseline.sh` (new build vs baseline build,
+**both codegraph-on**) rather than `run-all.sh`'s with-vs-without, because the question is
+whether a change to codegraph helped, not whether codegraph helps.
+
+| Change | Repos | Result |
+|---|---|---|
+| **Score-proportional byte allocation** (#1500, epic CG-1) — 2026-08-04, `feature/CG-1` vs `main`, 3 runs/arm | client-go (Go, 2,454 f, 2,001 generated — the reporter's shape), excalidraw (TS, 672 f), express (JS, 147 f, control) | **Gate FAILED.** Read 0/0/0 both arms on client-go and excalidraw; excalidraw **34s → 24s median with one fewer explore call**; generated clientsets/informers drop from 10.5% of a baseline envelope to 0% in every new run. But express regressed in 1 of 3 runs (**4 Reads, 52s**) from a reproducible non-agent cause: a file whose proportional reservation lands below its own size no longer renders whole and its cluster render leaves the reservation **unspent** (`lib/utils.js` 6,380 B whole → 583 B stub, envelope 13.8K → 9.2K). Full record: [`docs/benchmarks/explore-allocation-ab-1500.md`](../benchmarks/explore-allocation-ab-1500.md) |
+
+Two harness lessons from that run, both now baked into `ab-new-vs-baseline.sh`:
+
+- **Name codegraph in the prompt for this class of A/B.** Whether the agent picks the tool at
+  all is an adoption axis a retrieval change does not touch; a run that never calls explore
+  measures nothing about how explore divides its bytes (one pre-run: 0 codegraph calls, 3
+  Reads). It is not a forced-Read-0 — the agent stays free to fall back, which is the bar.
+- **`CODEGRAPH_NO_PROMPT_HOOK=1` on both arms.** The machine's ambient front-load hook resolves
+  to whatever is in `dist/`, which the script itself rewrites between arms.
+
 ---
 
 ## 7. Known limits & gotchas (from the excalidraw/django work)

docs/design/explore-budget-allocation.md

@@ -446,3 +446,59 @@ Both were found by writing the invariant rather than by reading the code:
 tier monotonicity, the floor) reference the constants and hold at any value; one test pins
 the literals, so re-tuning a constant is a visible decision that says *re-run the probe*
 rather than a silent re-calibration of the fixtures.
+
+## CG-15 — what the agent A/B found: a reservation can go unspent
+
+The agent A/B (full record: [`../benchmarks/explore-allocation-ab-1500.md`](../benchmarks/explore-allocation-ab-1500.md))
+passed on both medium repos — client-go and excalidraw at Read 0 in every run, excalidraw
+**34s → 24s median with one fewer explore call**, the generated clientsets that took 10.5% of a
+baseline envelope gone from every new run — and **failed on the small control**, express, in 1
+run of 3: 4 Reads of `lib/utils.js` where the baseline made 1 Read of a different file.
+
+It is not agent variance. Replaying that run's own query deterministically:
+
+| `lib/utils.js` (5,293 B, 272 lines) | baseline | CG-12 |
+|---|---|---|
+| delivered | **6,380 (46.1%) whole** | **583 (7.7%) cluster stub** |
+| source envelope (budget 13,000) | 13,849 | 9,241 |
+
+The diagnostic says the allocator was right and the render loop was not:
+
+```
+allocation 12,398 reserved of 12,400 pool · nothing cliffed
+ #  deliv%   bytes  reserved  score   flags                render    file
+ 1    5.7%     583     3,870   56.0   named entry central  clusters  lib/utils.js
+```
+
+`utils.js` is the **top-ranked** file and was **reserved 3,870 chars — of which it spent 583.**
+The whole-file bound (`src/mcp/tools.ts:4008`) is
+`allowance + min(GRACE_MAX, allowance * GRACE_FRACTION)` = `3,870 + 580` = 4,450, just under the
+file's 5,293 bytes, so the whole-file render is declined; the fallback cluster render has three
+matched symbols to work with and emits 583 chars. The remaining 3,287 chars of the reservation
+are **not redistributed — they are lost**, which is why the response shrank by a third against
+an unchanged budget.
+
+This is CG-12's own acceptance criterion (*"no file that was previously unclipped becomes
+clipped"*) failing. CG-14 recorded one instance as a documented exception (`memory-budget.ts`);
+this is the same defect in the wild, where it costs an agent round-trip. It is **not** systemic:
+on both medium repos the render loop saturates (`[over budget] [TRUNCATED]`, 23,599 of a 23,600
+pool reserved) so there is nothing left to lose. It needs a file whose reservation lands below
+its own size while its matched-symbol set is thin — likeliest on small repos, where per-file
+reservations are smallest.
+
+### The two candidate fixes
+
+Widening the envelope is explicitly **not** one of them — that is the dial iter2 already proved
+doesn't work, and the bytes here were reserved for this file already.
+
+1. **Let a large-enough reservation buy the whole file.** Render whole when
+   `allowance >= k * fileSize` for some `k < 1` (utils.js sits at 0.73), letting the bounded
+   overshoot the hard ceiling already tolerates absorb the difference. Smaller change, matches
+   the observed shape.
+2. **Redistribute the shortfall.** Once the render loop knows a file's realised size, hand what
+   it cannot spend to the next-ranked file. The stronger invariant — *the pool is spent* — and
+   it fixes the shrinking-envelope symptom directly rather than case by case.
+
+They compose; (1) alone would have carried this case. Whichever lands needs the express shape as
+a hermetic fixture — a mid-sized top-ranked file with few matched symbols, sized just above its
+reservation — because nothing in the current suite has that shape, which is how it shipped.