docs: allocation efficiency — the metric, its guards, and the 103-run baseline (CG-9)

Records the sweep over every A/B log on this machine (103 sessions, 297
explore calls, 0 crashes) and, more usefully, the new-vs-baseline arm
table the metric exists for: express 82% → 100%, cg21/client-go 67% →
95%, two pairs going the other way.

States the caveat in the places it can be misread: the corpus median sits
in the eighties because these are flow questions whose answers name most
of the chain, the metric is byte-weighted, and an agent can use a file
without citing it. It compares two builds on one question; it is not an
absolute waste figure.

docs/benchmarks/explore-allocation-ab-1500.md

@@ -51,6 +51,11 @@ rendered markdown of the responses the agent actually received. The CG-4 diagnos
 only exists on the new build, so it cannot measure the baseline arm; the markdown parse is the
 only instrument that measures both arms the same way.
 
+That `--answer <glob>` is hand-specified ground truth. Every run now also reports the same
+intersection inferred from the agent's own final answer, so it needs no per-question setup —
+see [`explore-allocation-efficiency.md`](explore-allocation-efficiency.md), which re-scores
+the arms below (express: 82% baseline → 100% new).
+
 | Repo | Lang | Files | Generated | Tier | Role |
 |---|---|---|---|---|---|
 | `kubernetes/client-go` | Go | 2,454 | 2,001 | medium (2 calls / 28K) | **the #1500 shape** — generated CRUD beside hand-written machinery |

docs/benchmarks/explore-allocation-efficiency.md

@@ -0,0 +1,195 @@
+# Explore allocation efficiency
+
+**What it measures:** of the bytes a `codegraph_explore` response spent, what
+share went to files the agent's answer actually drew on.
+
+```
+allocation efficiency = bytes returned for files the final answer cited
+                        ───────────────────────────────────────────────
+                        all bytes returned
+```
+
+That is issue #1500 as a number. The envelope view
+([`explore-allocation-ab-1500.md`](explore-allocation-ab-1500.md)) already
+showed *how* a response was divided across files, but deciding which of those
+files mattered took a human writing `--answer 'lib/response.js'` per question.
+This reads the same intersection off the agent's own final answer, so every run
+reports it for free.
+
+**Read it as a relative metric.** Attribution is by citation, and an agent can
+use a file's source without ever naming it — to rule the file out, or to build a
+mental model it then writes up from somewhere else. So this compares two builds
+on the *same* question; it is not an absolute claim that 15% of an envelope was
+waste. [What it does not say](#what-it-does-not-say) is the full list.
+
+Harness-only, like the other two feedback metrics: nothing is emitted from the
+product, nothing leaves the machine. It is parsed out of transcripts we already
+write.
+
+---
+
+## Running it
+
+Every run prints it — `run-all.sh` and anything else that calls `parse-run.mjs`:
+
+```bash
+node scripts/agent-eval/parse-run.mjs /private/tmp/cg22/ab-express/run-baseline-1.jsonl
+```
+
+```
+  Explore allocation — share of returned bytes the answer used (2 calls, 4 files):
+    efficiency  81.9%  17,465 of 21,319 chars   (3/4 files cited; path-cited alone 64.9%)
+    call 1:  84.0%  10,121/12,048 chars  2/3 files
+    call 2:  79.2%  7,344/9,271 chars  2/3 files
+    *  34.9%   7447  lib/response.js  path
+    *  30.0%   6396  lib/utils.js  path
+       18.1%   3854  lib/express.js  —
+    *  17.0%   3622  lib/application.js  symbol `compileETag`
+```
+
+Interactive runs get the same block from `parse-session.mjs <project-dir>`.
+
+**Per call as well as per run.** A run-level number hides the shape that matters
+most — call 1 on target, call 3 pure noise. The per-call lines name which call
+spent its budget on files nothing came back to.
+
+`--answer <glob>` and `--envelope` still work and are unchanged: they are the
+*hand-specified* ground truth, useful when you want to score against the files
+you know answer the question rather than the ones the agent happened to name.
+
+---
+
+## How a file is judged used
+
+Two citation channels, ranked so the weaker one stays separable — the summary
+line always reports **path-cited alone** next to the combined number.
+
+**1. Path.** The answer names the file: `lib/response.js:126-220`, an absolute
+path, or the bare `utils.js:225` agents drop into prose. Bare basenames are
+accepted only when the extension is one the envelope actually shipped, which is
+what keeps `res.send` and `mime.contentType` — the same token shape — from
+reading as file citations.
+
+**2. Symbol.** The answer cites, inside a code span, a symbol that the file's
+section header lists as **defined**. This catches answers written almost
+entirely in symbol names. It is guarded three ways, all in the direction that
+costs us efficiency rather than inflating it:
+
+- **Definitions only.** The header renders `name(kind)` for every node in the
+  shipped clusters, and that includes call sites (`mutateElement(calls)` on a
+  file that merely calls it). Crediting those marked excalidraw's
+  `dragElements.ts` used because the answer named `mutateElement`.
+- **A definition beats an import alias.** `var compileETag = require('./utils')`
+  is a `variable` node, so a file that only re-binds a name loses to the file
+  that defines it, when both are in the envelope.
+- **A name on 3+ returned files identifies none of them.** Without this,
+  `send` or `get` marks half an envelope used.
+
+Prose mentions do not count — only code spans. A backtick is the agent marking
+the token as code, and that is the whole signal.
+
+**A file returned twice is charged twice**, because it occupied the window
+twice. Same accounting as the envelope view.
+
+**The answer** is the `result` event's text (one per resumed segment of a
+multi-turn session — all of them are answers, so all are pooled). An interactive
+transcript has no `result` event, so it falls back to the last main-thread
+assistant text; intermediate narration is deliberately excluded, or "let me look
+at App.tsx" would count as a citation.
+
+---
+
+## Baseline
+
+Swept over every A/B log on this machine, the same corpus CG-8 used: **103
+sessions** with at least one answered explore (74 single-question, 29 multi-turn
+3-question sessions), **297 explore calls**, 815 file sections, 0 crashes.
+
+| Slice | n | pooled | median | p25 | p75 | min | path-cited only |
+|---|---|---|---|---|---|---|---|
+| all | 103 | 85.6% | 90.1% | 82.0% | 97.7% | 20.1% | 80.2% |
+| single-question | 74 | 85.2% | 88.8% | 82.0% | 95.6% | 20.1% | 82.2% |
+| multi-turn (3 turns) | 29 | 86.2% | 92.1% | 85.0% | 100.0% | 57.4% | 77.5% |
+
+Per call: median 94.7%, p25 79.2%.
+
+**The absolute level is high, and that is a property of the corpus, not good
+news.** These are flow questions whose answers walk the reader through the
+chain, naming most files on it. The metric is byte-weighted, so it is dominated
+by whether the *largest* allocations landed on cited files — which is exactly
+the #1500 question, and also why the typical run scores in the eighties. The
+discrimination lives in the p25 and below, not around the median. **Never quote
+the median as "codegraph wastes 10% of what it returns."**
+
+**The tail is where the signal is.** The lowest run in the corpus is 20.1%:
+`cg8-val` — excalidraw's `canvasNonce` question, the documented data-flow
+frontier. Three explores returned 11 files / 60,694 chars; the answer drew on
+two of them (`Scene.ts`, `StaticCanvas.tsx`) and reconstructed the rest by
+reading `Renderer.ts` and `App.tsx`, which the envelope never shipped. That
+reproduces the 16–30% range CG-1 measured by hand on the self-query, and it
+reproduces it without anyone marking an answer set.
+
+### New-build vs baseline-build — the intended use
+
+Medians over the existing CG-15/CG-21/CG-22 A/B arms, both arms codegraph-on:
+
+| Pass / repo | baseline | new |
+|---|---|---|
+| cg15 / express | 82.0% | **100.0%** |
+| cg15 / excalidraw | 90.1% | 92.8% |
+| cg15 / client-go | 84.1% | 80.8% |
+| cg21 / express | 84.1% | **100.0%** |
+| cg21 / excalidraw | 91.3% | 97.5% |
+| cg21 / client-go | 67.1% | **95.0%** |
+| cg22 / express | 81.9% | **100.0%** |
+| cg22 / excalidraw | 89.3% | 79.2% |
+| cg22 / client-go | 86.6% | 93.3% |
+
+Express is the #1500 case and it moves the most: the baseline arm reliably spent
+18% of its envelope on `lib/express.js`, a file no answer ever cited, and the
+allocation fix cut it — 81.9 / 82.0 / 81.9% across three baseline runs, 100.0 /
+92.5 / 100.0% across three new ones. That is the change stated as a number
+rather than as "the express arm looks tighter now."
+
+Two pairs go the other way (cg15/client-go −3pt, cg22/excalidraw −10pt). The
+cg22/excalidraw spread is 89.3 / 85.6 / 90.8% baseline against 87.4 / 79.2 /
+78.2% new — real, but n=3 on a question with known run-to-run variance. Report
+the range, not the median of three.
+
+---
+
+## What it does not say
+
+- **Uncited is not unused.** An agent reads a file, decides it is not the
+  answer, and never mentions it. That was useful work and this metric calls it
+  waste. The error is one-sided and it is why the number is only comparable
+  between builds on the same question.
+- **Cited is not "needed at that size."** The metric asks which *files* earned
+  their bytes, not whether the right *lines within* a cited file were shipped.
+  A file cited by the answer scores 100% of its section even if the agent then
+  had to Read it for the part we clipped out. The sufficiency bucket
+  `Read a file we returned` is what catches that; the two metrics are
+  complementary, and CG-13 reports both.
+- **A basename match can land on the wrong file.** `src/index.ts` in the
+  envelope matches an answer citing `packages/element/src/index.ts`. Rare in
+  practice, but it is a real false-positive channel.
+- **Small-n.** Runs make 1–5 explore calls, so one run's percentage is coarse.
+  Compare over a pass (`RUNS>=2`, and the CG-13 7-repo campaign), never n=1.
+- **Efficiency is not value.** A response can be 100% efficient and useless — a
+  single small file the answer names in passing. Read it alongside sufficiency
+  ([`explore-sufficiency.md`](explore-sufficiency.md)) and occupancy
+  ([`residual-context-occupancy.md`](residual-context-occupancy.md)), which is
+  the point of wiring all three into the same run.
+
+---
+
+## Tests
+
+`node scripts/agent-eval/parse-run.mjs --selftest` covers the metric over
+synthetic transcripts with known answers, alongside the occupancy and
+sufficiency checks (68/68). The allocation cases: the one-of-three-files shape,
+bare-basename citations and the dotted-expression tokens that must *not* match,
+symbol attribution crediting the definer and not a caller, a definition beating
+an import alias, the 3-file ambiguity cutoff, prose-vs-code-span, per-call
+versus pooled accounting, a file returned twice being charged twice, both
+final-answer sources, and the end-to-end path through `parseSession`.