Kaynağa Gözat

merge: shrink a later cluster into the remainder instead of dropping it (CG-36)

A file whose top-ranked cluster was trivial kept it, dropped the cluster
carrying the answer WHOLE, and left most of its reservation unspent — because
only the first-chosen cluster could be shrunk. CG-31's carry-forward then
correctly handed that slack down the rank order, so the budget was not merely
unspent but REDIRECTED to weaker files.

django's sql/query.py (score 83, reserved 7,947) went from 1,923 delivered
chars to 10,082, and its envelope share from 7.7% to 40.4%; contrib/admin/
filters.py (score 18) went from 8,057 at 355% of its reservation down to 2,198
at 97%. All 8 starvation flags across the suite clear. Net +1,012 source chars.

The issue named the wrong fix point and the measurement said so: both real
cases lost on maxImportance, NOT on the density tiebreak the issue and its
duplicate (CG-37) suspected. Cluster ranking was left untouched, so the
Session.swift case density-first exists for still works — now pinned by a
dense-header fixture.

Accepted cost: okhttp trades its rank-6 file (score 21, reserved 1,999) for
+7,196 chars in the two files that answer the question, taking it from 6
delivered files to 5. django -159, okhttp -219 and tokio -25 source chars
against the epic tip; gin +1,176, alamofire +187, excalidraw +52. django also
stops cutting its epilogue.

Ships probe-file-spend.mjs, a standing suite-wide probe for reservation vs
spend, so this stays measurable — the original evidence came from ad-hoc
instrumentation that no longer existed and had to be re-derived by hand.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Colby McHenry 1 ay önce
ebeveyn
işleme
8a4623463d

+ 1 - 0
CHANGELOG.md

@@ -34,6 +34,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - When a `codegraph_explore` answer runs right up against its size limit, it now drops the trailing notes rather than a whole file's source. Previously the last file was cut even though trimming the notes alone would have fit, so a file that had already been read, ranked and rendered was thrown away at the last moment. Across a range of real projects this returns one more file and up to 20% more source per call.
 - When a `codegraph_explore` answer runs right up against its size limit, it now drops the trailing notes rather than a whole file's source. Previously the last file was cut even though trimming the notes alone would have fit, so a file that had already been read, ranked and rendered was thrown away at the last moment. Across a range of real projects this returns one more file and up to 20% more source per call.
 - Every file `codegraph_explore` decides to include now actually arrives. A file shown in full could still spend room set aside for files below it — the fix above covered files shown as excerpts but not files shown whole — and the answer's own size bookkeeping under-counted each file's heading, so the answer ran past its limit and a fully prepared file was discarded at the end. A file that no longer fits whole is now shown as excerpts instead of vanishing, and one that overshoots by a little is trimmed to fit rather than dropped.
 - Every file `codegraph_explore` decides to include now actually arrives. A file shown in full could still spend room set aside for files below it — the fix above covered files shown as excerpts but not files shown whole — and the answer's own size bookkeeping under-counted each file's heading, so the answer ran past its limit and a fully prepared file was discarded at the end. A file that no longer fits whole is now shown as excerpts instead of vanishing, and one that overshoots by a little is trimmed to fit rather than dropped.
 - The list of files an answer could not cover — the "explore these names for their source" pointers — is no longer thrown away when the answer is full. It is now budgeted for and trimmed to fit, so a full answer still tells you what it left out and which names to ask for next, instead of ending with no pointers at all.
 - The list of files an answer could not cover — the "explore these names for their source" pointers — is no longer thrown away when the answer is full. It is now budgeted for and trimmed to fit, so a full answer still tells you what it left out and which names to ask for next, instead of ending with no pointers at all.
+- When a `codegraph_explore` answer shows a file as excerpts, a large excerpt that no longer fit was dropped entirely instead of being shortened. If the file's first excerpt happened to be a trivial one — an import block, a one-line helper next to the code you asked about — the excerpt carrying the actual answer was the one thrown away, and the file came back with a quarter of the room it had been given. On real projects that meant the top-ranked file delivered a fraction of its share while a far less relevant file took the rest. Excerpts are now shortened to fit, whole method by whole method, and only dropped when what is left is too small to hold anything readable.
 - The blast-radius section of `codegraph_explore` flagged "no covering tests found" whenever no test called a symbol directly — falsely branding helpers that tests exercise through their callers as untested (about 40% of flagged symbols in a measured sample). The check now follows caller chains up to 3 hops and reports indirect coverage as "tested via callers"; when nothing is found it states exactly what was checked instead of an unconditional warning. Thanks @inth3shadows for measuring the false-positive rate. (#1475)
 - The blast-radius section of `codegraph_explore` flagged "no covering tests found" whenever no test called a symbol directly — falsely branding helpers that tests exercise through their callers as untested (about 40% of flagged symbols in a measured sample). The check now follows caller chains up to 3 hops and reports indirect coverage as "tested via callers"; when nothing is found it states exactly what was checked instead of an unconditional warning. Thanks @inth3shadows for measuring the false-positive rate. (#1475)
 
 
 ## [1.5.0] - 2026-07-21
 ## [1.5.0] - 2026-07-21

+ 170 - 0
__tests__/explore-cluster-starvation.test.ts

@@ -0,0 +1,170 @@
+/**
+ * Regression gate for CLUSTER-LEVEL STARVATION inside one file (task CG-36).
+ *
+ * A file's ranked clusters used to be all-or-nothing past the first one: the
+ * top-ranked cluster was taken (shrunk to fit if it had to be), and every
+ * cluster below it was rendered whole and then either fit the remainder or was
+ * dropped entirely. On a file whose top-ranked cluster is TRIVIAL that discards
+ * the answer — django's `db/models/sql/query.py` kept a 22-line glue cluster and
+ * dropped the 624-line `Query` body beneath it, spending 1,923 of a 7,947
+ * reservation, and okhttp's `RealInterceptorChain.kt` did the same behind its
+ * import header.
+ *
+ * What makes it hard to see is that the response stays FULL: the unspent
+ * reservation carries forward exactly as designed, so a lower-scoring file takes
+ * the bytes and every envelope-share measure still looks healthy. The gate is
+ * therefore per-file spend, not share.
+ *
+ * Two fixtures, pulling in opposite directions — read them together:
+ *
+ *   - `starved-cluster-ts` is the defect. Its answer-bearing cluster must be
+ *     SHRUNK into whatever the trivial cluster left, not dropped.
+ *   - `dense-header-ts` is the Session.swift shape that cluster ranking puts
+ *     importance ahead of density FOR. Its query's methods sit ~200 lines under
+ *     a dense property list, and they must keep winning the budget. Any future
+ *     rework of selection or shrinking has to satisfy both.
+ */
+import { describe, it, expect, beforeAll, afterAll } from 'vitest';
+import * as fs from 'fs';
+import * as path from 'path';
+import * as os from 'os';
+import CodeGraph from '../src/index';
+import { ToolHandler } from '../src/mcp/tools';
+import type { ExploreDiagnosticReport } from '../src/mcp/explore-diagnostics';
+
+interface Run {
+  dir: string;
+  cg: CodeGraph;
+  response: string;
+  report: ExploreDiagnosticReport;
+}
+
+/** Copy a fixture tree to a temp dir, index it, and run one explore call. */
+async function runFixture(fixture: string, query: string): Promise<Run> {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cg36-'));
+  fs.cpSync(path.join(__dirname, 'fixtures', fixture), dir, { recursive: true });
+  fs.rmSync(path.join(dir, '.codegraph'), { recursive: true, force: true });
+
+  const cg = CodeGraph.initSync(dir);
+  await cg.indexAll();
+
+  const sidecar = path.join(dir, 'explore-diag.jsonl');
+  const previous = process.env.CODEGRAPH_EXPLORE_DEBUG;
+  process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
+  let response: string;
+  try {
+    response = (await new ToolHandler(cg).execute('codegraph_explore', { query }))
+      .content?.[0]?.text ?? '';
+  } finally {
+    if (previous === undefined) delete process.env.CODEGRAPH_EXPLORE_DEBUG;
+    else process.env.CODEGRAPH_EXPLORE_DEBUG = previous;
+  }
+  const written = fs.readFileSync(sidecar, 'utf-8').trim().split('\n').filter(Boolean);
+  return { dir, cg, response, report: JSON.parse(written[written.length - 1]!) };
+}
+
+function teardown(run: Run | undefined): void {
+  if (!run) return;
+  run.cg.destroy();
+  if (fs.existsSync(run.dir)) fs.rmSync(run.dir, { recursive: true, force: true });
+}
+
+describe('CG-36 — a trivial cluster must not starve the answer-bearing one', () => {
+  const TARGET = 'src/pipeline/chain.ts';
+  const QUERY = 'how does a request travel from sendRequest to the socket';
+  let run: Run;
+  let target: ExploreDiagnosticReport['files'][number];
+
+  beforeAll(async () => {
+    run = await runFixture('starved-cluster-ts', QUERY);
+    target = run.report.files.find((f) => f.path === TARGET)!;
+  }, 120_000);
+
+  afterAll(() => teardown(run));
+
+  describe('fixture shape — if this rots, the gate below means nothing', () => {
+    it('renders through the cluster path, with the answer past the trivial helper', () => {
+      expect(target, `${TARGET} is not among the ranked candidates`).toBeDefined();
+      expect(target.render).toBe('clusters');
+      // The helper the entry point calls directly, and the class it does not.
+      const nodes = run.cg.getNodesInFile(TARGET);
+      const helper = nodes.find((n) => n.name === 'describeChain')!;
+      const proceed = nodes.find((n) => n.name === 'proceed')!;
+      expect(helper).toBeDefined();
+      expect(proceed).toBeDefined();
+      // Far enough apart to cluster separately at any gap threshold we ship.
+      expect(proceed.startLine - helper.endLine).toBeGreaterThan(20);
+    });
+
+    it('reserves the file the largest share, so an unspent share is a defect', () => {
+      expect(target.allowance ?? 0).toBeGreaterThan(4000);
+      const others = run.report.files.filter((f) => f.path !== TARGET);
+      for (const f of others) expect(f.allowance ?? 0).toBeLessThan(target.allowance!);
+    });
+  });
+
+  describe('the gate', () => {
+    it('spends most of the reservation it was given', () => {
+      // 28.8% on the CG-24 epic tip, 131% (its reservation plus carry-forward
+      // slack it can now actually use) with the fix. The bar is deliberately
+      // well below both so ordinary budget movement does not fail the suite.
+      expect(target.finalChars / target.allowance!).toBeGreaterThan(0.6);
+    });
+
+    it('delivers the flow the query asked about, not just the helper beside it', () => {
+      // Both ends of the in-file flow, in the cluster that used to be dropped.
+      expect(run.response).toContain('async proceed(request: PipelineRequest)');
+      expect(run.response).toContain('private async writeAndRead(request: PipelineRequest)');
+    });
+
+    it('keeps the response inside the hard ceiling', () => {
+      expect(run.report.envelope.chars).toBeLessThanOrEqual(run.report.budget.hardCeiling);
+    });
+  });
+});
+
+describe('CG-36 — a dense declaration block must not bury the query\'s methods', () => {
+  const TARGET = 'src/net/session.ts';
+  const QUERY = 'how does perform create a URLRequest and start the task';
+  let run: Run;
+  let target: ExploreDiagnosticReport['files'][number];
+
+  beforeAll(async () => {
+    run = await runFixture('dense-header-ts', QUERY);
+    target = run.report.files.find((f) => f.path === TARGET)!;
+  }, 120_000);
+
+  afterAll(() => teardown(run));
+
+  describe('fixture shape — if this rots, the gate below means nothing', () => {
+    it('has a dense low-importance header and the named methods far below it', () => {
+      expect(target, `${TARGET} is not among the ranked candidates`).toBeDefined();
+      expect(target.render).toBe('clusters');
+      const nodes = run.cg.getNodesInFile(TARGET);
+      const perform = nodes.find((n) => n.name === 'perform')!;
+      expect(perform).toBeDefined();
+      // The header block: many adjacent declarations above the first named
+      // method, which is what makes it the densest region of the file.
+      const above = nodes.filter((n) => n.endLine < perform.startLine
+        && (n.kind === 'property' || n.kind === 'field' || n.kind === 'method'));
+      expect(above.length).toBeGreaterThan(20);
+      expect(perform.startLine).toBeGreaterThan(150);
+    });
+  });
+
+  describe('the gate', () => {
+    it('delivers all three methods the query named', () => {
+      expect(run.response).toContain('async perform(url: string, method: string');
+      expect(run.response).toContain('didCreateURLRequest(request: URLRequest)');
+      expect(run.response).toContain('task(request: URLRequest, identifier: number)');
+    });
+
+    it('spends the file\'s reservation on them', () => {
+      expect(target.finalChars / target.allowance!).toBeGreaterThan(0.6);
+    });
+
+    it('keeps the response inside the hard ceiling', () => {
+      expect(run.report.envelope.chars).toBeLessThanOrEqual(run.report.budget.hardCeiling);
+    });
+  });
+});

+ 6 - 0
__tests__/fixtures/dense-header-ts/package.json

@@ -0,0 +1,6 @@
+{
+  "name": "dense-header-fixture",
+  "private": true,
+  "version": "0.0.0",
+  "type": "module"
+}

+ 23 - 0
__tests__/fixtures/dense-header-ts/src/core/queue.ts

@@ -0,0 +1,23 @@
+import type { URLSessionTask } from './types';
+
+export class RequestQueue {
+  private readonly waiting: URLSessionTask[] = [];
+  private running = 0;
+
+  enqueue(task: URLSessionTask, limit: number): void {
+    if (this.running < limit) {
+      this.running += 1;
+      return;
+    }
+    this.waiting.push(task);
+  }
+
+  release(): URLSessionTask | undefined {
+    this.running = Math.max(0, this.running - 1);
+    return this.waiting.shift();
+  }
+
+  get depth(): number {
+    return this.waiting.length;
+  }
+}

+ 27 - 0
__tests__/fixtures/dense-header-ts/src/core/request-builder.ts

@@ -0,0 +1,27 @@
+import type { CachePolicy, URLRequest } from './types';
+
+export function buildURLRequest(options: {
+  url: string;
+  method: string;
+  body?: Uint8Array;
+  headers: Record<string, string>;
+  timeout: number;
+  cachePolicy: CachePolicy;
+}): URLRequest {
+  const headers = { ...options.headers };
+  if (options.body && !headers['content-length']) {
+    headers['content-length'] = String(options.body.length);
+  }
+  return {
+    url: normalize(options.url),
+    method: options.method.toUpperCase(),
+    headers,
+    body: options.body,
+    timeout: options.timeout,
+    cachePolicy: options.cachePolicy,
+  };
+}
+
+function normalize(url: string): string {
+  return url.endsWith('/') && url.split('/').length > 4 ? url.slice(0, -1) : url;
+}

+ 23 - 0
__tests__/fixtures/dense-header-ts/src/core/task-factory.ts

@@ -0,0 +1,23 @@
+import type { RequestDelegate, TaskResponse, URLRequest, URLSessionTask } from './types';
+
+export function makeTask(options: {
+  identifier: number;
+  request: URLRequest;
+  delegate: RequestDelegate;
+  allowsCellularAccess: boolean;
+  waitsForConnectivity: boolean;
+  resourceTimeout: number;
+}): URLSessionTask {
+  const handlers: Array<(response: TaskResponse) => void> = [];
+  return {
+    identifier: options.identifier,
+    request: options.request,
+    state: 'initialized',
+    cancel() { this.state = 'cancelled'; },
+    onComplete(handler) { handlers.push(handler); },
+  };
+}
+
+export function resumeTask(task: URLSessionTask): void {
+  task.state = 'resumed';
+}

+ 42 - 0
__tests__/fixtures/dense-header-ts/src/core/types.ts

@@ -0,0 +1,42 @@
+export type CachePolicy = 'useProtocolCachePolicy' | 'reloadIgnoringLocalCacheData' | 'returnCacheDataElseLoad';
+export type RequestState = 'initialized' | 'resumed' | 'suspended' | 'cancelled' | 'finished';
+
+export interface URLRequest {
+  url: string;
+  method: string;
+  headers: Record<string, string>;
+  body?: Uint8Array;
+  timeout: number;
+  cachePolicy: CachePolicy;
+}
+
+export interface TaskResponse {
+  status: number;
+  headers: Record<string, string>;
+  body: Uint8Array;
+}
+
+export interface URLSessionTask {
+  identifier: number;
+  request: URLRequest;
+  state: RequestState;
+  cancel(): void;
+  onComplete(handler: (response: TaskResponse) => void): void;
+}
+
+export interface Adapter { adapt(request: URLRequest): URLRequest; }
+export interface Serializer { serialize(value: unknown): Uint8Array; }
+export interface Validator { validate(response: TaskResponse): { ok: boolean; reason?: string }; }
+export interface Retrier { shouldRetry(response: TaskResponse, verdict: { ok: boolean }): boolean; }
+export interface RedirectHandler { resolve(location: string, original: URLRequest): { url: string; method: string; body?: Uint8Array } | null; }
+export interface TrustEvaluator { evaluate(host: string): boolean; }
+export interface Credential { apply(request: URLRequest): URLRequest; }
+export interface Interceptor { name: string; adapt(request: URLRequest, session: unknown): Promise<URLRequest>; }
+export interface RequestDelegate { willSend(request: URLRequest): void; }
+export interface EventMonitor {
+  didAdaptRequest(request: URLRequest, interceptor: string): void;
+  didCreateTask(task: URLSessionTask, request: URLRequest): void;
+  didResumeTask(task: URLSessionTask): void;
+  didRetryTask(task: URLSessionTask, previousIdentifier: number): void;
+  didCompleteTask(task: URLSessionTask, response: TaskResponse): void;
+}

+ 3 - 0
__tests__/fixtures/dense-header-ts/src/index.ts

@@ -0,0 +1,3 @@
+export { Session } from './net/session';
+export { RequestQueue } from './core/queue';
+export { buildURLRequest } from './core/request-builder';

+ 285 - 0
__tests__/fixtures/dense-header-ts/src/net/session.ts

@@ -0,0 +1,285 @@
+import type {
+  Adapter,
+  CachePolicy,
+  Credential,
+  EventMonitor,
+  Interceptor,
+  RedirectHandler,
+  RequestDelegate,
+  RequestState,
+  Retrier,
+  Serializer,
+  TrustEvaluator,
+  URLRequest,
+  URLSessionTask,
+  Validator,
+} from '../core/types';
+import { buildURLRequest } from '../core/request-builder';
+import { makeTask, resumeTask } from '../core/task-factory';
+import { RequestQueue } from '../core/queue';
+
+/**
+ * The shape density-first ranking exists for: a class whose top-of-file header
+ * is a long, tightly-packed property list — dozens of adjacent declarations,
+ * each individually trivial — while the methods a flow question actually asks
+ * about live hundreds of lines below it.
+ *
+ * Ranked by density alone the header wins the file's whole budget and the
+ * methods are buried. The ranking puts importance first for exactly this
+ * reason, and density only breaks ties inside one importance tier.
+ */
+export class Session {
+  readonly identifier: string;
+  readonly adapter: Adapter;
+  readonly serializer: Serializer;
+  readonly validator: Validator;
+  readonly retrier: Retrier;
+  readonly redirectHandler: RedirectHandler;
+  readonly trustEvaluator: TrustEvaluator;
+  readonly eventMonitor: EventMonitor;
+  readonly cachePolicy: CachePolicy;
+  readonly credential: Credential | null;
+  readonly interceptors: Interceptor[];
+  readonly delegate: RequestDelegate;
+  readonly queue: RequestQueue;
+  readonly startRequestsImmediately: boolean;
+  readonly maximumConnectionsPerHost: number;
+  readonly timeoutIntervalForRequest: number;
+  readonly timeoutIntervalForResource: number;
+  readonly allowsCellularAccess: boolean;
+  readonly waitsForConnectivity: boolean;
+  readonly httpShouldUsePipelining: boolean;
+  readonly httpShouldSetCookies: boolean;
+  readonly httpMaximumConnectionsPerHost: number;
+  readonly sessionConfigurationName: string;
+  readonly requestState: RequestState;
+  readonly defaultHeaders: Record<string, string>;
+  readonly userAgent: string;
+  readonly acceptEncoding: string;
+  readonly acceptLanguage: string;
+  private taskCounter = 0;
+  private active = new Map<number, URLSessionTask>();
+
+  constructor(options: Partial<Session> & { identifier: string }) {
+    this.identifier = options.identifier;
+    this.adapter = options.adapter!;
+    this.serializer = options.serializer!;
+    this.validator = options.validator!;
+    this.retrier = options.retrier!;
+    this.redirectHandler = options.redirectHandler!;
+    this.trustEvaluator = options.trustEvaluator!;
+    this.eventMonitor = options.eventMonitor!;
+    this.cachePolicy = options.cachePolicy ?? 'useProtocolCachePolicy';
+    this.credential = options.credential ?? null;
+    this.interceptors = options.interceptors ?? [];
+    this.delegate = options.delegate!;
+    this.queue = options.queue ?? new RequestQueue();
+    this.startRequestsImmediately = options.startRequestsImmediately ?? true;
+    this.maximumConnectionsPerHost = options.maximumConnectionsPerHost ?? 6;
+    this.timeoutIntervalForRequest = options.timeoutIntervalForRequest ?? 60;
+    this.timeoutIntervalForResource = options.timeoutIntervalForResource ?? 604800;
+    this.allowsCellularAccess = options.allowsCellularAccess ?? true;
+    this.waitsForConnectivity = options.waitsForConnectivity ?? false;
+    this.httpShouldUsePipelining = options.httpShouldUsePipelining ?? false;
+    this.httpShouldSetCookies = options.httpShouldSetCookies ?? true;
+    this.httpMaximumConnectionsPerHost = options.httpMaximumConnectionsPerHost ?? 6;
+    this.sessionConfigurationName = options.sessionConfigurationName ?? 'default';
+    this.requestState = options.requestState ?? 'initialized';
+    this.defaultHeaders = options.defaultHeaders ?? {};
+    this.userAgent = options.userAgent ?? 'session/1.0';
+    this.acceptEncoding = options.acceptEncoding ?? 'br;q=1.0, gzip;q=0.9';
+    this.acceptLanguage = options.acceptLanguage ?? 'en;q=1.0';
+  }
+
+  // -- configuration accessors ----------------------------------------------
+  // Individually trivial, adjacent, and dense. On the density tiebreak alone
+  // this block outranks anything with a body worth reading.
+
+  get isBackground(): boolean {
+    return this.sessionConfigurationName === 'background';
+  }
+
+  get connectionLimit(): number {
+    return Math.min(this.maximumConnectionsPerHost, this.httpMaximumConnectionsPerHost);
+  }
+
+  get headerDefaults(): Record<string, string> {
+    return { ...this.defaultHeaders, 'user-agent': this.userAgent };
+  }
+
+  get acceptHeaders(): Record<string, string> {
+    return { 'accept-encoding': this.acceptEncoding, 'accept-language': this.acceptLanguage };
+  }
+
+  get activeCount(): number {
+    return this.active.size;
+  }
+
+  get isIdle(): boolean {
+    return this.active.size === 0;
+  }
+
+  get nextIdentifier(): number {
+    return this.taskCounter + 1;
+  }
+
+  get description(): string {
+    return `Session(${this.identifier}, ${this.sessionConfigurationName})`;
+  }
+
+  cancelAll(): void {
+    for (const task of this.active.values()) task.cancel();
+    this.active.clear();
+  }
+
+  taskFor(identifier: number): URLSessionTask | undefined {
+    return this.active.get(identifier);
+  }
+
+  headers(): Record<string, string> {
+    return { ...this.headerDefaults, ...this.acceptHeaders };
+  }
+
+  withUserAgent(userAgent: string): Session {
+    return new Session({ ...this, identifier: this.identifier, userAgent });
+  }
+
+  withTimeout(seconds: number): Session {
+    return new Session({ ...this, identifier: this.identifier, timeoutIntervalForRequest: seconds });
+  }
+
+  withInterceptor(interceptor: Interceptor): Session {
+    return new Session({
+      ...this,
+      identifier: this.identifier,
+      interceptors: [...this.interceptors, interceptor],
+    });
+  }
+
+  withCredential(credential: Credential): Session {
+    return new Session({ ...this, identifier: this.identifier, credential });
+  }
+
+  withCachePolicy(cachePolicy: CachePolicy): Session {
+    return new Session({ ...this, identifier: this.identifier, cachePolicy });
+  }
+
+  withQueue(queue: RequestQueue): Session {
+    return new Session({ ...this, identifier: this.identifier, queue });
+  }
+
+  withAdapter(adapter: Adapter): Session {
+    return new Session({ ...this, identifier: this.identifier, adapter });
+  }
+
+  withValidator(validator: Validator): Session {
+    return new Session({ ...this, identifier: this.identifier, validator });
+  }
+
+  withRetrier(retrier: Retrier): Session {
+    return new Session({ ...this, identifier: this.identifier, retrier });
+  }
+
+  withMonitor(eventMonitor: EventMonitor): Session {
+    return new Session({ ...this, identifier: this.identifier, eventMonitor });
+  }
+
+  // -- the flow ---------------------------------------------------------------
+  //
+  // The methods below are what a "how does a request get built and sent" question
+  // is about, and they sit hundreds of lines under the header block.
+
+  /**
+   * Turn a convenience call into a URLRequest, hand it to the adapter chain and
+   * start the resulting task. The entry point of the whole flow.
+   */
+  async perform(url: string, method: string, body?: Uint8Array): Promise<URLSessionTask> {
+    const initial = buildURLRequest({
+      url,
+      method,
+      body,
+      headers: this.headers(),
+      timeout: this.timeoutIntervalForRequest,
+      cachePolicy: this.cachePolicy,
+    });
+    const adapted = await this.adapt(initial);
+    return this.didCreateURLRequest(adapted);
+  }
+
+  /**
+   * Every interceptor gets a chance to rewrite the request before it becomes a
+   * task. Runs in registration order, and a thrown error aborts the whole call.
+   */
+  private async adapt(request: URLRequest): Promise<URLRequest> {
+    let current = request;
+    for (const interceptor of this.interceptors) {
+      current = await interceptor.adapt(current, this);
+      this.eventMonitor.didAdaptRequest(current, interceptor.name);
+    }
+    if (this.credential) current = this.credential.apply(current);
+    return current;
+  }
+
+  /**
+   * The adapted request is final: build the task around it, register it and —
+   * unless the session was told to wait — resume it immediately.
+   */
+  didCreateURLRequest(request: URLRequest): URLSessionTask {
+    this.taskCounter += 1;
+    const identifier = this.taskCounter;
+    const created = this.task(request, identifier);
+    this.active.set(identifier, created);
+    this.eventMonitor.didCreateTask(created, request);
+    if (this.startRequestsImmediately) this.resume(created);
+    return created;
+  }
+
+  /**
+   * Build the URLSessionTask for a request. Split out from
+   * `didCreateURLRequest` because retries rebuild the task without going back
+   * through the adapter chain.
+   */
+  task(request: URLRequest, identifier: number): URLSessionTask {
+    const created = makeTask({
+      identifier,
+      request,
+      delegate: this.delegate,
+      allowsCellularAccess: this.allowsCellularAccess,
+      waitsForConnectivity: this.waitsForConnectivity,
+      resourceTimeout: this.timeoutIntervalForResource,
+    });
+    created.onComplete((response) => {
+      this.active.delete(identifier);
+      const verdict = this.validator.validate(response);
+      if (!verdict.ok && this.retrier.shouldRetry(response, verdict)) {
+        this.retry(request, identifier);
+        return;
+      }
+      this.eventMonitor.didCompleteTask(created, response);
+    });
+    return created;
+  }
+
+  /** Put a built task on the queue and start it. */
+  resume(task: URLSessionTask): void {
+    this.queue.enqueue(task, this.connectionLimit);
+    resumeTask(task);
+    this.eventMonitor.didResumeTask(task);
+  }
+
+  /** Rebuild and restart a task the retrier asked for. */
+  private retry(request: URLRequest, previousIdentifier: number): void {
+    this.taskCounter += 1;
+    const retried = this.task(request, this.taskCounter);
+    this.active.set(this.taskCounter, retried);
+    this.eventMonitor.didRetryTask(retried, previousIdentifier);
+    this.resume(retried);
+  }
+
+  /** Follow a redirect by adapting and re-performing the new location. */
+  async follow(response: { location: string }, original: URLRequest): Promise<URLSessionTask> {
+    const target = this.redirectHandler.resolve(response.location, original);
+    if (!target) throw new Error(`redirect to ${response.location} refused`);
+    return this.perform(target.url, target.method, target.body);
+  }
+}

+ 6 - 0
__tests__/fixtures/starved-cluster-ts/package.json

@@ -0,0 +1,6 @@
+{
+  "name": "starved-cluster-fixture",
+  "private": true,
+  "version": "0.0.0",
+  "type": "module"
+}

+ 24 - 0
__tests__/fixtures/starved-cluster-ts/src/app/client.ts

@@ -0,0 +1,24 @@
+import { RequestChain, describeChain } from '../pipeline/chain';
+import type { PipelineRequest, PipelineResponse } from '../pipeline/types';
+import { openSocket } from '../transport/socket';
+
+/**
+ * The entry point a caller reaches for. Everything the chain does happens
+ * underneath this call, which is why a flow question names it.
+ */
+export async function sendRequest(request: PipelineRequest): Promise<PipelineResponse> {
+  const socket = openSocket(request.host, request.port);
+  const chain = new RequestChain(request, socket);
+  trace(describeChain(chain));
+  return chain.proceed(request);
+}
+
+export function trace(line: string): void {
+  if (process.env.PIPELINE_TRACE) process.stderr.write(`${line}\n`);
+}
+
+export async function sendAll(requests: PipelineRequest[]): Promise<PipelineResponse[]> {
+  const out: PipelineResponse[] = [];
+  for (const request of requests) out.push(await sendRequest(request));
+  return out;
+}

+ 14 - 0
__tests__/fixtures/starved-cluster-ts/src/app/config.ts

@@ -0,0 +1,14 @@
+export interface ClientConfig {
+  host: string;
+  port: number;
+  retries: number;
+  userAgent: string;
+}
+
+export function defaultConfig(): ClientConfig {
+  return { host: 'localhost', port: 8080, retries: 3, userAgent: 'pipeline/1.0' };
+}
+
+export function withHost(config: ClientConfig, host: string): ClientConfig {
+  return { ...config, host };
+}

+ 4 - 0
__tests__/fixtures/starved-cluster-ts/src/index.ts

@@ -0,0 +1,4 @@
+export { sendRequest, sendAll } from './app/client';
+export { RequestChain, describeChain } from './pipeline/chain';
+export { openSocket } from './transport/socket';
+export { defaultConfig } from './app/config';

+ 318 - 0
__tests__/fixtures/starved-cluster-ts/src/pipeline/chain.ts

@@ -0,0 +1,318 @@
+import type { PipelineRequest, PipelineResponse, Interceptor, Socket } from './types';
+import { encodeFrame, decodeFrame } from './framing';
+import { defaultInterceptors } from './interceptors';
+
+/**
+ * A one-line summary of a chain, used only by the tracing hook in the caller.
+ * It is TRIVIAL — it answers nothing about how a request travels — but it sits
+ * next to the entry point in the call graph, so its cluster carries the file's
+ * highest per-symbol importance.
+ */
+export function describeChain(chain: RequestChain): string {
+  return `chain(${chain.index}/${chain.size}) -> ${chain.hostLabel}`;
+}
+
+// ---------------------------------------------------------------------------
+//
+// Everything below is the part a "how does a request reach the socket" question
+// is actually asking about. It is separated from the helper above by more than
+// the cluster gap threshold, so it forms its own cluster — a large one, whose
+// symbols are reached transitively rather than named.
+//
+// ---------------------------------------------------------------------------
+
+export class RequestChain {
+  readonly index: number;
+  readonly size: number;
+  readonly hostLabel: string;
+  private readonly interceptors: Interceptor[];
+  private readonly socket: Socket;
+  private readonly request: PipelineRequest;
+  private connectTimeoutMs = 10_000;
+  private readTimeoutMs = 10_000;
+  private writeTimeoutMs = 10_000;
+  private calls = 0;
+
+  constructor(request: PipelineRequest, socket: Socket, index = 0, interceptors?: Interceptor[]) {
+    this.request = request;
+    this.socket = socket;
+    this.index = index;
+    this.interceptors = interceptors ?? defaultInterceptors();
+    this.size = this.interceptors.length;
+    this.hostLabel = `${request.host}:${request.port}`;
+  }
+
+  /**
+   * Run the request through the remaining interceptors and, once they are
+   * exhausted, hand it to the transport. This is the method the flow question
+   * is about: every hop between the caller and the socket passes through here.
+   */
+  async proceed(request: PipelineRequest): Promise<PipelineResponse> {
+    if (this.index >= this.size) {
+      return this.writeAndRead(request);
+    }
+    this.calls += 1;
+    if (this.calls > 1) {
+      throw new Error(`chain link ${this.index} called ${this.calls} times`);
+    }
+    const next = this.advance(request);
+    const interceptor = this.interceptors[this.index]!;
+    const response = await interceptor.intercept(next);
+    if (!response) {
+      throw new Error(`interceptor ${interceptor.name} returned no response`);
+    }
+    if (this.index + 1 < this.size && next.callCount() === 0) {
+      throw new Error(`interceptor ${interceptor.name} must call proceed()`);
+    }
+    return response;
+  }
+
+  /**
+   * The next link in the chain: the same chain with the cursor moved on and the
+   * timeouts carried over. Cloning here is what keeps each interceptor from
+   * mutating the chain the one before it is still holding.
+   */
+  advance(request: PipelineRequest): RequestChain {
+    const next = new RequestChain(request, this.socket, this.index + 1, this.interceptors);
+    next.connectTimeoutMs = this.connectTimeoutMs;
+    next.readTimeoutMs = this.readTimeoutMs;
+    next.writeTimeoutMs = this.writeTimeoutMs;
+    return next;
+  }
+
+  callCount(): number {
+    return this.calls;
+  }
+
+  /**
+   * The end of the chain: frame the request, put the bytes on the socket, wait
+   * for the reply and decode it. Past this point there is no more pipeline —
+   * this is the transport hop the question is looking for.
+   */
+  private async writeAndRead(request: PipelineRequest): Promise<PipelineResponse> {
+    const frame = encodeFrame(request);
+    await this.socket.connect(this.connectTimeoutMs);
+    await this.socket.write(frame, this.writeTimeoutMs);
+    const raw = await this.socket.read(this.readTimeoutMs);
+    const decoded = decodeFrame(raw);
+    return {
+      status: decoded.status,
+      headers: decoded.headers,
+      body: decoded.body,
+      request,
+    };
+  }
+
+  withConnectTimeout(ms: number): RequestChain {
+    const next = this.advance(this.request);
+    next.connectTimeoutMs = checkDuration('connectTimeout', ms);
+    return next;
+  }
+
+  withReadTimeout(ms: number): RequestChain {
+    const next = this.advance(this.request);
+    next.readTimeoutMs = checkDuration('readTimeout', ms);
+    return next;
+  }
+
+  withWriteTimeout(ms: number): RequestChain {
+    const next = this.advance(this.request);
+    next.writeTimeoutMs = checkDuration('writeTimeout', ms);
+    return next;
+  }
+
+  connectTimeout(): number {
+    return this.connectTimeoutMs;
+  }
+
+  readTimeout(): number {
+    return this.readTimeoutMs;
+  }
+
+  writeTimeout(): number {
+    return this.writeTimeoutMs;
+  }
+
+  /**
+   * Retry policy for the transport hop. Sits inside the same cluster as the
+   * proceed/advance pair, so it is part of what a shrink has to choose between.
+   */
+  async retryWrite(request: PipelineRequest, attempts: number): Promise<PipelineResponse> {
+    let lastError: unknown;
+    for (let attempt = 0; attempt < attempts; attempt += 1) {
+      try {
+        return await this.writeAndRead(request);
+      } catch (error) {
+        lastError = error;
+        await backoff(attempt);
+      }
+    }
+    throw lastError;
+  }
+
+  /** Whether the chain may still be resumed after a transport failure. */
+  canRetry(error: unknown): boolean {
+    if (this.index >= this.size) return false;
+    if (!(error instanceof Error)) return false;
+    return error.message.includes('timeout') || error.message.includes('reset');
+  }
+
+  /** The interceptor names, in the order the request will visit them. */
+  route(): string[] {
+    return this.interceptors.slice(this.index).map((i) => i.name);
+  }
+
+  /** A copy of the chain rewound to the first interceptor. */
+  rewind(): RequestChain {
+    return new RequestChain(this.request, this.socket, 0, this.interceptors);
+  }
+
+  /** Drop one interceptor by name and return the shortened chain. */
+  without(name: string): RequestChain {
+    const kept = this.interceptors.filter((i) => i.name !== name);
+    return new RequestChain(this.request, this.socket, this.index, kept);
+  }
+
+  /** Append an interceptor to the end of the chain. */
+  with(interceptor: Interceptor): RequestChain {
+    return new RequestChain(
+      this.request,
+      this.socket,
+      this.index,
+      [...this.interceptors, interceptor],
+    );
+  }
+
+  /** Close the transport this chain was built around. */
+  async close(): Promise<void> {
+    await this.socket.close();
+  }
+
+  /** Headers the transport hop will actually put on the wire. */
+  effectiveHeaders(): Record<string, string> {
+    const headers: Record<string, string> = { ...this.request.headers };
+    headers['host'] = this.hostLabel;
+    headers['x-chain-index'] = String(this.index);
+    headers['x-chain-size'] = String(this.size);
+    if (this.request.body) headers['content-length'] = String(this.request.body.length);
+    return headers;
+  }
+
+  /** The request as the next link will see it, with the chain's headers merged. */
+  prepared(): PipelineRequest {
+    return { ...this.request, headers: this.effectiveHeaders() };
+  }
+
+  /**
+   * Send the prepared request through the rest of the chain. The convenience
+   * wrapper most callers use instead of building the request themselves.
+   */
+  async send(): Promise<PipelineResponse> {
+    return this.proceed(this.prepared());
+  }
+
+  /** Whether the chain has any interceptor left before the transport hop. */
+  hasNext(): boolean {
+    return this.index < this.size;
+  }
+
+  /** The interceptor the next `proceed` will run, if there is one. */
+  peek(): Interceptor | undefined {
+    return this.interceptors[this.index];
+  }
+
+  /** Total configured wait for one attempt, across all three timeouts. */
+  totalTimeout(): number {
+    return this.connectTimeoutMs + this.readTimeoutMs + this.writeTimeoutMs;
+  }
+
+  /** Apply one timeout budget to all three phases at once. */
+  withTimeout(ms: number): RequestChain {
+    const next = this.advance(this.request);
+    const checked = checkDuration('timeout', ms);
+    next.connectTimeoutMs = checked;
+    next.readTimeoutMs = checked;
+    next.writeTimeoutMs = checked;
+    return next;
+  }
+
+  /**
+   * Run the chain and translate a transport failure into a response, so a
+   * caller that only cares about the status code never sees an exception.
+   */
+  async sendOrStatus(status: number): Promise<PipelineResponse> {
+    try {
+      return await this.send();
+    } catch {
+      return {
+        status,
+        headers: this.effectiveHeaders(),
+        body: new Uint8Array(),
+        request: this.request,
+      };
+    }
+  }
+
+  /** A short description of where in the chain this link sits. */
+  position(): string {
+    return `${this.index + 1} of ${this.size + 1}`;
+  }
+
+  /** The chain rebuilt around a different transport. */
+  onSocket(socket: Socket): RequestChain {
+    return new RequestChain(this.request, socket, this.index, this.interceptors);
+  }
+
+  /**
+   * Replay the request through the chain from the start, reusing the transport.
+   * Used when an interceptor decides the response it got is not usable and the
+   * whole pipeline has to run again against the same connection.
+   */
+  async replay(): Promise<PipelineResponse> {
+    const fresh = this.rewind();
+    try {
+      return await fresh.send();
+    } finally {
+      if (!fresh.hasNext()) await fresh.close();
+    }
+  }
+
+  /**
+   * Validate the chain before it runs: every interceptor named once, timeouts
+   * inside their bounds, and a transport still open at the end of it.
+   */
+  validate(): string[] {
+    const problems: string[] = [];
+    const seen = new Set<string>();
+    for (const interceptor of this.interceptors) {
+      if (seen.has(interceptor.name)) problems.push(`duplicate interceptor ${interceptor.name}`);
+      seen.add(interceptor.name);
+    }
+    if (this.connectTimeoutMs <= 0) problems.push('connect timeout must be positive');
+    if (this.readTimeoutMs <= 0) problems.push('read timeout must be positive');
+    if (this.writeTimeoutMs <= 0) problems.push('write timeout must be positive');
+    if (this.index > this.size) problems.push('chain cursor is past the end');
+    return problems;
+  }
+
+  /**
+   * The transport hop on its own, with the chain's timeouts but none of its
+   * interceptors — the escape hatch a caller uses to bypass the pipeline.
+   */
+  async direct(request: PipelineRequest): Promise<PipelineResponse> {
+    const problems = this.validate();
+    if (problems.length > 0) throw new Error(problems.join('; '));
+    return this.writeAndRead(request);
+  }
+}
+
+function checkDuration(name: string, ms: number): number {
+  if (!Number.isFinite(ms) || ms < 0) throw new Error(`${name} must be a positive duration`);
+  if (ms > 24 * 60 * 60 * 1000) throw new Error(`${name} is longer than a day`);
+  return Math.round(ms);
+}
+
+async function backoff(attempt: number): Promise<void> {
+  const ms = Math.min(1000, 25 * 2 ** attempt);
+  await new Promise((resolve) => setTimeout(resolve, ms));
+}

+ 26 - 0
__tests__/fixtures/starved-cluster-ts/src/pipeline/framing.ts

@@ -0,0 +1,26 @@
+import type { PipelineRequest } from './types';
+
+export function encodeFrame(request: PipelineRequest): Uint8Array {
+  const head = `${request.method} ${request.path}\n`;
+  const headers = Object.entries(request.headers).map(([k, v]) => `${k}: ${v}`).join('\n');
+  const text = `${head}${headers}\n\n`;
+  const body = request.body ?? new Uint8Array();
+  const out = new Uint8Array(text.length + body.length);
+  out.set(new TextEncoder().encode(text), 0);
+  out.set(body, text.length);
+  return out;
+}
+
+export function decodeFrame(raw: Uint8Array): { status: number; headers: Record<string, string>; body: Uint8Array } {
+  const text = new TextDecoder().decode(raw);
+  const split = text.indexOf('\n\n');
+  const head = split < 0 ? text : text.slice(0, split);
+  const lines = head.split('\n');
+  const status = Number.parseInt(lines[0]?.split(' ')[1] ?? '0', 10);
+  const headers: Record<string, string> = {};
+  for (const line of lines.slice(1)) {
+    const at = line.indexOf(': ');
+    if (at > 0) headers[line.slice(0, at)] = line.slice(at + 2);
+  }
+  return { status, headers, body: raw.slice(split < 0 ? raw.length : split + 2) };
+}

+ 21 - 0
__tests__/fixtures/starved-cluster-ts/src/pipeline/interceptors.ts

@@ -0,0 +1,21 @@
+import type { Interceptor } from './types';
+
+export function defaultInterceptors(): Interceptor[] {
+  return [retryInterceptor(), headerInterceptor(), logInterceptor()];
+}
+
+export function retryInterceptor(): Interceptor {
+  return { name: 'retry', intercept: (chain) => chain.proceed(currentRequest()) };
+}
+
+export function headerInterceptor(): Interceptor {
+  return { name: 'headers', intercept: (chain) => chain.proceed(currentRequest()) };
+}
+
+export function logInterceptor(): Interceptor {
+  return { name: 'log', intercept: (chain) => chain.proceed(currentRequest()) };
+}
+
+function currentRequest() {
+  return { host: 'localhost', port: 80, method: 'GET', path: '/', headers: {} };
+}

+ 27 - 0
__tests__/fixtures/starved-cluster-ts/src/pipeline/types.ts

@@ -0,0 +1,27 @@
+export interface PipelineRequest {
+  host: string;
+  port: number;
+  method: string;
+  path: string;
+  headers: Record<string, string>;
+  body?: Uint8Array;
+}
+
+export interface PipelineResponse {
+  status: number;
+  headers: Record<string, string>;
+  body: Uint8Array;
+  request: PipelineRequest;
+}
+
+export interface Interceptor {
+  name: string;
+  intercept(chain: { proceed(request: PipelineRequest): Promise<PipelineResponse> }): Promise<PipelineResponse>;
+}
+
+export interface Socket {
+  connect(timeoutMs: number): Promise<void>;
+  write(frame: Uint8Array, timeoutMs: number): Promise<void>;
+  read(timeoutMs: number): Promise<Uint8Array>;
+  close(): Promise<void>;
+}

+ 30 - 0
__tests__/fixtures/starved-cluster-ts/src/transport/socket.ts

@@ -0,0 +1,30 @@
+import type { Socket } from '../pipeline/types';
+
+/** Open a transport socket for a host/port pair. */
+export function openSocket(host: string, port: number): Socket {
+  let open = false;
+  const inbox: Uint8Array[] = [];
+  return {
+    async connect(timeoutMs: number) {
+      if (open) return;
+      await settle(timeoutMs);
+      open = true;
+    },
+    async write(frame: Uint8Array, timeoutMs: number) {
+      if (!open) throw new Error(`socket to ${host}:${port} is not connected`);
+      await settle(timeoutMs);
+      inbox.push(frame);
+    },
+    async read(timeoutMs: number) {
+      await settle(timeoutMs);
+      return inbox.shift() ?? new Uint8Array();
+    },
+    async close() {
+      open = false;
+    },
+  };
+}
+
+async function settle(timeoutMs: number): Promise<void> {
+  if (timeoutMs <= 0) throw new Error('timed out');
+}

+ 164 - 0
docs/benchmarks/explore-cluster-starvation-cg36.md

@@ -0,0 +1,164 @@
+# Deterministic measurement — cluster starvation inside one file (task CG-36)
+
+**Date:** 2026-08-06 · **Baseline:** `feature/CG-24` @ `76ab1fe` (the CG-24 epic tip) ·
+**Harness:** `scripts/agent-eval/probe-file-spend.mjs` and `probe-suite-envelope.mjs` over the
+deterministic 6-repo corpus at `/tmp/codegraph-corpus`, clean full-rebuilt indexes (CG-33), plus
+two hermetic fixtures through `probe-allocation.mjs`. No agent A/B: the claim is which bytes go
+to which file, and the agent runs are far too noisy to see a 2K shift.
+
+**Verdict: the defect is real, the diagnosis in the issue was half wrong, and the fix holds.**
+All 8 starvation flags across the suite clear; net **+1,012 source chars**. One repo (okhttp)
+trades its weakest file for +7,196 chars in the two that answer the question — stated in full
+below rather than smoothed over.
+
+---
+
+## The defect
+
+A file's ranked clusters were all-or-nothing past the first one. The top-ranked cluster was
+always taken — shrunk to the highest-importance whole symbol ranges that fit when it overran —
+and every cluster below it was rendered **whole** and then either fit the remainder or was
+dropped entirely. On a file whose top-ranked cluster is trivial, that discards the answer.
+
+What made it invisible: the response stays FULL. The unspent reservation carries forward exactly
+as CG-31 designed, so a lower-scoring file takes the bytes and every envelope-share measure still
+reads healthy. Measured on the epic tip:
+
+| repo | file | score | reserved | spent | share |
+|---|---|---|---|---|---|
+| django | `db/models/sql/query.py` | 83 | 7,947 | 1,923 | **24%** |
+| django | `contrib/admin/filters.py` | 18 | 2,271 | 8,057 | **355%** |
+| okhttp | `.../RealInterceptorChain.kt` | 86 | 6,058 | 1,474 | **24%** |
+| okhttp | `.../CallServerInterceptor.kt` | 20 | 1,974 | 5,832 | **295%** |
+
+A score-83 file spends a quarter of its reservation while a score-18 file takes 3.5× its own.
+
+## What the issue got wrong
+
+The issue named two candidate fix points and suspected the first: cluster ranking breaks ties on
+**density** (`score / span`) after `hasSpine` → `maxImportance`, which structurally favours a
+small trivial cluster over a large answer-bearing one. Dumping the cluster set says otherwise —
+in **both** real cases the loser lost on `maxImportance`, not on density:
+
+```
+django/db/models/sql/query.py   budget 10,135   spent 1,923
+  KEPT    1379–1400   span   22   score  14   maxImp 6   (check_related_objects — a glue symbol)
+  DROPPED  306– 929   span  624   score 290   maxImp 3   (133 members: the whole Query class)
+
+okhttp .../RealInterceptorChain.kt   budget 6,058   spent 1,474
+  KEPT      16–  44   span   29   score  44   maxImp 6   (package decl + import block)
+  DROPPED  113– 373   span  261   score 171   maxImp 3   (73 members: the chain itself)
+```
+
+`maxImportance` first is deliberate and protective — it is what stops Alamofire's `Session.swift`
+from losing its budget to the top-of-file property list — so **ranking was not touched**. The
+lever is the second fix point: stop dropping the loser whole.
+
+## The change
+
+Two sites, one rule — *hold the remainder while it is still worth a section*, which is CG-26's
+between-FILES lesson applied between CLUSTERS.
+
+1. **Selection.** A later cluster is now shrunk into what is left of the file's budget, by the
+   same whole-member rule the first cluster already used. Below `MIN_CHARS` (700) the remainder
+   cannot hold a readable block, so it stays a drop rather than a stutter of fragments the next
+   call's dedup has to shred around. The never-empty windowing floors may overrun that room; the
+   first cluster is allowed that overshoot, a later one is not.
+2. **The ceiling trim.** When the exact section cost overruns `renderCeiling`, the weakest chosen
+   cluster is re-rendered into the room that remains before being dropped. This one is worth
+   naming on its own: on excalidraw's `typeChecks.ts` the section-cost estimate missed by
+   **13 chars** and a 1,512-char cluster — the file's highest-*scoring* one, last in rank order
+   only because rank breaks ties on density — was thrown away to pay for it. Recovered 1,501 of
+   excalidraw's 1,449-char loss.
+
+## Suite result
+
+`node scripts/agent-eval/probe-file-spend.mjs`, 6 repos, clean rebuilds. Only files whose spend
+moved are listed; score is the candidate's ranking score, reserved its allocation.
+
+| repo | file | score | reserved | before | after |
+|---|---|---|---|---|---|
+| django | `db/models/sql/query.py` | 83 | 7,947 | 1,923 | **10,082** |
+| django | `contrib/admin/filters.py` | 18 | 2,271 | 8,057 | 2,198 |
+| django | `utils/autoreload.py` | 12 | 1,747 | 3,145 | 1,709 |
+| django | `db/models/fields/related_descriptors.py` | 11 | 1,660 | 2,516 | 1,493 |
+| excalidraw | `element/src/typeChecks.ts` | 23 | 2,740 | 3,102 | 2,573 |
+| excalidraw | `excalidraw/types.ts` | 14 | 1,942 | 819 | 1,372 |
+| okhttp | `.../RealInterceptorChain.kt` | 86 | 6,058 | 1,474 | **6,038** |
+| okhttp | `.../Interceptor.kt` | 64 | 4,697 | 2,027 | **4,659** |
+| okhttp | `.../RealCall.kt` | 52 | 3,972 | 3,628 | 3,922 |
+| okhttp | `.../Call.kt` | 54 | 4,097 | 4,097 | 2,073 |
+| okhttp | `.../CallServerInterceptor.kt` | 20 | 1,974 | 5,832 | 1,959 |
+| okhttp | `androidMain/.../AndroidDns.kt` | 21 | 1,999 | 1,812 | **0** |
+| tokio | `task/local.rs` | 40 | 4,361 | 4,599 | 4,798 |
+| tokio | `runtime/task/harness.rs` | 14 | 1,981 | 2,565 | 2,341 |
+| gin | `routergroup.go` | 87 | 5,782 | 3,273 | **5,632** |
+| gin | `tree.go` | 17 | 1,693 | 892 | 1,969 |
+| gin | `ginS/gins.go` | 26 | 2,213 | 4,431 | 2,171 |
+| alamofire | `Source/Core/Session.swift` | 34 | 2,792 | 2,797 | 3,396 |
+| alamofire | `Source/Core/Request.swift` | 148 | 9,100 | 8,865 | 8,453 |
+
+Bytes move up the score order in every repo. Envelope totals:
+
+| repo | source before | after | Δ | files | ceiling |
+|---|---|---|---|---|---|
+| django | 20,878 | 20,719 | −159 | 6 → 6 | 24,963 ≤ 25,000 |
+| excalidraw | 19,652 | 19,704 | +52 | 8 → 8 | 24,813 ≤ 25,000 |
+| okhttp | 18,870 | 18,651 | −219 | 6 → **5** | 24,985 ≤ 25,000 |
+| tokio | 21,607 | 21,582 | −25 | 5 → 5 | 24,777 ≤ 25,000 |
+| gin | 10,776 | 11,952 | **+1,176** | 4 → 4 | 14,655 ≤ 19,500 |
+| alamofire | 11,662 | 11,849 | +187 | 2 → 2 | 12,862 ≤ 19,500 |
+| **total** | **103,445** | **104,457** | **+1,012** | | |
+
+Starvation flags: **8 → 0**.
+
+## The one cost, stated plainly
+
+okhttp drops its rank-6 file, `androidMain/.../AndroidDns.kt` (score 21, a platform DNS helper on
+a question about the interceptor chain), and 219 source chars, in exchange for +4,564 to
+`RealInterceptorChain.kt` and +2,632 to `Interceptor.kt` — the two files that answer the question.
+
+This is not a new defect and it is not the fix over-reaching. okhttp's reservations are
+**structurally over-subscribed**: the allocator splits `maxOutputChars` charging a flat
+`FILE_OVERHEAD` of 200 per file while a real header runs 300–500, so the sum of promises
+(~22,800 source + ~2,100 of real headers) exceeds what the ~24,760-char render ceiling can hold.
+`owedPayableBelow` already refuses to hold bytes back for a file it can see will be dropped, and
+AndroidDns.kt is the file past that line. On the epic tip it survived only because the files above
+it under-spent — by luck, not by design. Closing the over-subscription means charging the
+allocator per-file header estimates rather than the flat 200; that is a wider change than this
+issue, and CG-26 deliberately kept `FILE_OVERHEAD` as the allocator's own constant.
+
+## What did NOT change
+
+- **Cluster ranking.** `hasSpine` → `maxImportance` → density → score → span, untouched.
+- **Alamofire `Session.swift`.** The shape density-first exists for; it *gains* 599 chars.
+- **The factory-closure outcome (CG-27).** `probe-factory-closure.mjs`: 7 of 11 inner closure
+  definitions delivered, identical to the epic tip.
+- **The reservation invariant (CG-31/CG-26).** `explore-reservation-invariant.test.ts` green;
+  every repo stays at or under its hard ceiling.
+- **All four allocation fixtures pass** — `payroll-go`, `self-query`, and the two added here.
+
+## What ships so this stays measurable
+
+- `scripts/agent-eval/probe-file-spend.mjs` — the standing per-file reservation-vs-delivered
+  sweep. It flags a **pair**, never a single file: a large share unspent *while* a materially
+  lower-scoring file overspends. Either alone is legitimate (a small file has less to say;
+  carry-forward is the mechanism that hands its slack down), which is why the envelope probe
+  could never see this. Exit code 1 on any flag, so it gates.
+- `__tests__/fixtures/starved-cluster-ts/` — django's and okhttp's shape reduced to a fixture.
+  Fails on the epic tip (28.8% of reservation, neither `proceed` nor `writeAndRead` delivered),
+  passes with the fix.
+- `__tests__/fixtures/dense-header-ts/` — the `Session.swift` shape, byte-identical on both
+  builds. The counterweight: it fails if a future change lets density outrank importance again.
+- `spendShareAtLeast` in `probe-allocation.mjs`, and `__tests__/explore-cluster-starvation.test.ts`
+  pinning both fixtures in `npm test`.
+
+## Method note
+
+None of this is visible in the rendered markdown. To see it you must dump the cluster set —
+patch `dist/mcp/tools.js` just before `let assembled = assembleSection(chosenIndices);` and log
+`fileBudget` / `projectedChars` / each ranked cluster's span, score, `maxImportance`, chosen flag
+and members. Reading only the response makes member-selection effects look like budget effects.
+Equally, a source-chars diff between builds is not automatically a regression: excalidraw's
+−1,449 on the first cut was the elastic epilogue expanding into room a 13-char accounting error
+had released, not source lost to allocation.

+ 9 - 6
docs/benchmarks/explore-noise-epic-cg24.md

@@ -47,18 +47,21 @@ change at all**.
 | **CG-25** | Recognize `Generated by <tool> by running <command>` banners. Precision held by requiring two `by` clauses, so ordinary prose does not match. |
 | **CG-25** | Recognize `Generated by <tool> by running <command>` banners. Precision held by requiring two `by` clauses, so ordinary prose does not match. |
 | **CG-28** | Damp declaration-only files that nothing in the index depends on. Does not stack with the generated penalty (`Math.min`), and naming a declaration symbol exempts its file. |
 | **CG-28** | Damp declaration-only files that nothing in the index depends on. Does not stack with the generated penalty (`Math.min`), and naming a declaration symbol exempts its file. |
 | **CG-33 / CG-35** | Incremental sync converges with a rebuild, plus a regression suite that fails when the fix is disabled. |
 | **CG-33 / CG-35** | Incremental sync converges with a rebuild, plus a regression suite that fails when the fix is disabled. |
+| **CG-36** | A later cluster is shrunk into the remainder rather than dropped whole — at selection, and again in the ceiling trim. All 8 starvation flags across the suite clear; +1,012 source chars net. `explore-cluster-starvation-cg36.md`. |
 
 
 Deterministic across the 6-repo suite: no repo truncates, none loses a file,
 Deterministic across the 6-repo suite: no repo truncates, none loses a file,
 okhttp gains one, every repo lands at or under the 25,000 hard ceiling.
 okhttp gains one, every repo lands at or under the 25,000 hard ceiling.
 
 
 ## Open
 ## Open
 
 
-**[CG-36]** — a file's non-first clusters are never shrunk, so a trivial cluster
-starves the answer-bearing one. Found as a byproduct of CG-27's measurement and
-confirmed firing on the epic tip: `query.py` leaves 81% of its budget unspent,
-keeping a score-14 cluster and dropping a score-290 one. Two candidate fix
-points — the density tiebreak in selection, and the never-shrink rule — and
-density-first must keep working (the `Session.swift` case).
+Nothing. CG-36, the last one, shipped 2026-08-06.
+
+Its own measurement is worth carrying forward, because the issue named the wrong
+fix point: both real cases (`query.py`, `RealInterceptorChain.kt`) lost on
+`maxImportance`, **not** on the density tiebreak the issue suspected. Ranking was
+left alone; the never-shrink rule was the lever. Full numbers and the one cost
+(okhttp trades its rank-6 file for +7,196 chars in the two that answer the
+question) in `explore-cluster-starvation-cg36.md`.
 
 
 ## Closed because measurement contradicted them
 ## Closed because measurement contradicted them
 
 

+ 126 - 1
scripts/agent-eval/allocation-fixtures.json

@@ -22,7 +22,14 @@
     "actually about) and `incidental` (what wins the envelope today on name collisions).",
     "actually about) and `incidental` (what wins the envelope today on name collisions).",
     "Assertions are on the DELIVERED envelope unless suffixed `Allocated`; delivered is",
     "Assertions are on the DELIVERED envelope unless suffixed `Allocated`; delivered is",
     "what the agent got, allocated is what the render loop chose before the hard ceiling.",
     "what the agent got, allocated is what the render loop chose before the hard ceiling.",
-    "Shares are fractions of the whole response, meta-text included, so they never sum to 1."
+    "Shares are fractions of the whole response, meta-text included, so they never sum to 1.",
+    "",
+    "CG-36 adds two more fixtures and a per-file `spendShareAtLeast` gate. The share gates",
+    "above ask which files WON the envelope; that one asks whether a file that won its share",
+    "then spent it. `starved-cluster` and `dense-header` are the two halves of the same",
+    "tradeoff and must be read together — one fails if a trivial cluster starves the",
+    "answer-bearing one, the other fails if the fix for that buries a query's own methods",
+    "under a dense declaration block."
   ],
   ],
   "fixtures": [
   "fixtures": [
     {
     {
@@ -105,6 +112,124 @@
         "verdict": "ALL GATES PASS. Answer group 78.7% (from 25.6% at baseline), generated layer 0.0% (from 57.4%). All four hand-written files deliver source, including payslip_builder.go — `func (s *Service) BuildPayslip`, the 'calculate' half of the question, finally reaches the agent. The generated files are still NAMED with their symbols and line numbers under 'Not shown above', so withholding their bytes costs ~100 chars each instead of ~4,500 and stays one follow-up explore away."
         "verdict": "ALL GATES PASS. Answer group 78.7% (from 25.6% at baseline), generated layer 0.0% (from 57.4%). All four hand-written files deliver source, including payslip_builder.go — `func (s *Service) BuildPayslip`, the 'calculate' half of the question, finally reaches the agent. The generated files are still NAMED with their symbols and line numbers under 'Not shown above', so withholding their bytes costs ~100 chars each instead of ~4,500 and stays one follow-up explore away."
       }
       }
     },
     },
+    {
+      "id": "starved-cluster",
+      "title": "CG-36 — a trivial top-ranked cluster starving the answer-bearing one",
+      "kind": "fixture",
+      "path": "__tests__/fixtures/starved-cluster-ts",
+      "query": "how does a request travel from sendRequest to the socket",
+      "rationale": [
+        "django's `db/models/sql/query.py` and okhttp's `RealInterceptorChain.kt`, reduced",
+        "to a fixture. `chain.ts` holds a one-line `describeChain` helper at the top —",
+        "trivial, but a direct callee of the query's entry point, so its cluster carries",
+        "the file's highest per-symbol importance — and, past the cluster gap, the",
+        "`RequestChain` class that actually answers the question. The helper's cluster wins",
+        "the one guaranteed-and-shrinkable slot; the class then does not fit the remainder.",
+        "",
+        "Before CG-36 the class was dropped WHOLE and the file delivered 1,985 of a 6,904",
+        "reservation. That is not merely unspent budget: the slack carries forward to",
+        "lower-ranked files, so the response stays full and every envelope-share gate",
+        "passes while the answer is missing. Hence `spendShareAtLeast`."
+      ],
+      "groups": {
+        "answer": [
+          "src/pipeline/chain.ts",
+          "src/transport/**",
+          "src/app/client.ts"
+        ],
+        "incidental": [
+          "src/app/config.ts",
+          "src/pipeline/framing.ts"
+        ]
+      },
+      "assert": {
+        "topFileGroup": "answer",
+        "spendShareAtLeast": {
+          "src/pipeline/chain.ts": 0.6
+        },
+        "mustDeliverBytes": [
+          "src/pipeline/chain.ts"
+        ],
+        "$mustContainComment": "The two ends of the in-file flow: the chain hop and the transport hop it terminates in. Both live in the cluster that used to be dropped whole.",
+        "mustContain": [
+          "async proceed(request: PipelineRequest)",
+          "private async writeAndRead(request: PipelineRequest)"
+        ]
+      },
+      "baseline": {
+        "measuredOn": "2026-08-06",
+        "note": "The CG-24 epic tip (76ab1fe), before CG-36. 3,725 chars of source delivered in total.",
+        "delivered": {
+          "src/pipeline/chain.ts": 1985,
+          "src/app/client.ts": 997,
+          "src/pipeline/types.ts": 743
+        },
+        "verdict": "FAILS spendShareAtLeast and both needles. chain.ts spends 1,985 of its 6,904 reservation (28.8%) — it keeps the `describeChain` cluster and drops the `RequestChain` cluster whole, so neither `proceed` nor `writeAndRead` reaches the agent. Nothing else in the response is wrong: the file still ranks #1 by score and is still reserved the largest slice."
+      },
+      "afterCG36": {
+        "measuredOn": "2026-08-06",
+        "note": "10,802 chars of source delivered in total, nothing truncated.",
+        "delivered": {
+          "src/pipeline/chain.ts": 9062,
+          "src/app/client.ts": 997,
+          "src/pipeline/types.ts": 743
+        },
+        "verdict": "ALL GATES PASS. The `RequestChain` cluster is now SHRUNK into the remainder by the same whole-member rule the first cluster already used, instead of being dropped whole, so chain.ts delivers 9,062 chars including `proceed`, `advance` and `writeAndRead` — the whole in-file flow the question asks for."
+      }
+    },
+    {
+      "id": "dense-header",
+      "title": "CG-36 — the Session.swift shape density-first ranking exists for",
+      "kind": "fixture",
+      "path": "__tests__/fixtures/dense-header-ts",
+      "query": "how does perform create a URLRequest and start the task",
+      "rationale": [
+        "The counterweight to `starved-cluster`, and the reason CG-36 did NOT touch cluster",
+        "ranking. `session.ts` opens with a 60-line property list and a run of trivial",
+        "accessors — many adjacent, individually worthless declarations, i.e. the densest",
+        "block in the file — while `perform`, `didCreateURLRequest` and `task`, which the",
+        "query names, sit ~200 lines below it.",
+        "",
+        "Ranked on density alone the header block takes the file's whole budget and the",
+        "methods are buried; that is Alamofire's Session.swift, the case the",
+        "importance-then-density order was built for. Any future change to selection or",
+        "shrinking has to keep this passing as well as `starved-cluster` — they pull in",
+        "opposite directions, which is exactly why both are here."
+      ],
+      "groups": {
+        "answer": [
+          "src/net/**",
+          "src/core/request-builder.ts",
+          "src/core/task-factory.ts"
+        ],
+        "incidental": [
+          "src/core/queue.ts"
+        ]
+      },
+      "assert": {
+        "topFileGroup": "answer",
+        "answerShareOfSourceAtLeast": 0.8,
+        "spendShareAtLeast": {
+          "src/net/session.ts": 0.6
+        },
+        "$mustContainComment": "All three named symbols are deep in the file, past the dense header block. If density ever outranks importance again, these are the first thing to go.",
+        "mustContain": [
+          "async perform(url: string, method: string",
+          "didCreateURLRequest(request: URLRequest)",
+          "task(request: URLRequest, identifier: number)"
+        ]
+      },
+      "afterCG36": {
+        "measuredOn": "2026-08-06",
+        "note": "11,695 chars of source delivered, BYTE-IDENTICAL to the CG-24 epic tip (76ab1fe) — this fixture pins behaviour CG-36 deliberately left alone.",
+        "delivered": {
+          "src/net/session.ts": 9007,
+          "src/core/types.ts": 1957,
+          "src/core/task-factory.ts": 731
+        },
+        "verdict": "ALL GATES PASS, on the epic tip and on CG-36 alike. session.ts spends 9,007 of its 9,009 reservation and the response carries all three named methods from the bottom of the file. The dense header block is not what won the budget."
+      }
+    },
     {
     {
       "id": "self-query",
       "id": "self-query",
       "title": "This repo — incidental `explore`/`BUDGET` matches in the agent-eval scripts",
       "title": "This repo — incidental `explore`/`BUDGET` matches in the agent-eval scripts",

+ 17 - 0
scripts/agent-eval/probe-allocation.mjs

@@ -199,6 +199,23 @@ function evaluate(fixture, report, text) {
         : 'not among the ranked candidates',
         : 'not among the ranked candidates',
     );
     );
   }
   }
+  // Reservation-vs-delivered, per file (CG-36). The share gates above ask which
+  // files won the envelope; this asks whether a file that WON its share then
+  // actually spent it. A file can rank #1, be reserved the largest slice, and
+  // still deliver a quarter of it because the cluster carrying the answer was
+  // dropped whole instead of shrunk — and the share gates read that as a pass,
+  // since the unspent bytes carry forward and the envelope stays full.
+  for (const [path, floor] of Object.entries(want.spendShareAtLeast ?? {})) {
+    const rec = report.files.find((f) => f.path === path);
+    const spent = rec && rec.allowance ? rec.finalChars / rec.allowance : 0;
+    add(
+      `${path} spends >= ${pct(floor)} of its reservation`,
+      !!rec && rec.allowance > 0 && spent >= floor,
+      rec
+        ? `${num(rec.finalChars)} delivered of a ${num(rec.allowance ?? 0)} reservation (${pct(spent)})`
+        : 'not among the ranked candidates',
+    );
+  }
   for (const needle of want.mustContain ?? []) {
   for (const needle of want.mustContain ?? []) {
     add(`response contains "${needle}"`, text.includes(needle), text.includes(needle) ? 'present' : 'absent');
     add(`response contains "${needle}"`, text.includes(needle), text.includes(needle) ? 'present' : 'absent');
   }
   }

+ 195 - 0
scripts/agent-eval/probe-file-spend.mjs

@@ -0,0 +1,195 @@
+#!/usr/bin/env node
+/**
+ * Per-file reservation-vs-delivered sweep for `codegraph_explore` (CG-36).
+ *
+ * `probe-suite-envelope.mjs` answers "how much source did the response deliver";
+ * this answers the question one level down — "did the bytes go to the files that
+ * earned them". The CG-36 defect was invisible to the envelope probe because the
+ * envelope stayed full: a rank-#3 file spent 24% of its reservation, the slack
+ * carried forward exactly as designed, and a far weaker file spent 3.5x its own.
+ * The response looked healthy; the ANSWER-bearing file had been starved.
+ *
+ * So the flag here is a PAIR, not a per-file threshold: a file that leaves a
+ * large share of its reservation unspent WHILE a materially lower-scoring file
+ * spends well over its own. Either alone is legitimate — a small file simply has
+ * less to say, and carry-forward is the mechanism that hands its slack down.
+ *
+ * Numbers come from the CG-4 diagnostic (`CODEGRAPH_EXPLORE_DEBUG`), so this
+ * measures the shipping allocator rather than re-deriving shares from markdown.
+ *
+ * Usage (needs a current `npm run build`, and full-REBUILT indexes — CG-33):
+ *   node scripts/agent-eval/probe-file-spend.mjs
+ *   node scripts/agent-eval/probe-file-spend.mjs --json > /tmp/new.json
+ *   node scripts/agent-eval/probe-file-spend.mjs --baseline /tmp/base.json
+ *   node scripts/agent-eval/probe-file-spend.mjs django --all   # every file, not just flags
+ *   CORPUS=/tmp/codegraph-corpus node scripts/agent-eval/probe-file-spend.mjs
+ *
+ * Exit code is 1 when any repo carries a starvation flag, so this can gate.
+ */
+import { mkdtempSync, readFileSync, rmSync, existsSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join, resolve } from 'node:path';
+import { pathToFileURL } from 'node:url';
+
+const CORPUS = process.env.CORPUS ?? '/tmp/codegraph-corpus';
+
+/** Same six repos and queries the CG-30/CG-31/CG-26 envelope tables use. */
+const SUITE = [
+  { id: 'django', q: 'How does a QuerySet turn into SQL and fetch rows from the database?' },
+  { id: 'excalidraw', q: 'How does updating an element re-render the canvas on screen?' },
+  { id: 'okhttp', q: 'How does a call go through the interceptor chain to the network?' },
+  { id: 'tokio', q: 'How does a spawned task get scheduled and run by a worker?' },
+  { id: 'gin', q: 'How does a registered route handler get invoked for an incoming HTTP request?' },
+  { id: 'alamofire', q: 'How does a request get built and sent through the session?' },
+];
+
+/**
+ * Starvation thresholds. A flag needs BOTH sides — the starved file and the
+ * overspending one it lost the bytes to.
+ *
+ * `MIN_RESERVED` keeps the noise out: under it, "80% unspent" is a few hundred
+ * chars and means nothing. `SCORE_RATIO` is what makes the pair meaningful —
+ * a higher-scoring file underspending while a *comparable* one overspends is
+ * ordinary; the defect is a materially weaker file taking the bytes.
+ */
+const STARVED_SHARE = 0.5;   // spent < half its reservation
+const OVERSPEND_RATIO = 1.5; // spent > 1.5x its own reservation
+const SCORE_RATIO = 2;       // ...while scoring less than half the starved file
+const MIN_RESERVED = 2000;   // ignore files whose reservation is too small to matter
+
+const argv = process.argv.slice(2);
+const asJson = argv.includes('--json');
+const showAll = argv.includes('--all');
+const baselineAt = argv.includes('--baseline') ? argv[argv.indexOf('--baseline') + 1] : null;
+const only = argv.filter((a) => !a.startsWith('--') && a !== baselineAt);
+
+const say = (s = '') => { if (!asJson) console.log(s); };
+const num = (n) => Math.round(n).toLocaleString('en-US');
+const pct = (f) => `${(f * 100).toFixed(1)}%`;
+
+const load = (rel) => import(pathToFileURL(resolve(rel)).href);
+const idx = await load('dist/index.js');
+const toolsMod = await load('dist/mcp/tools.js');
+const CodeGraph = idx.default?.default ?? idx.default ?? idx.CodeGraph;
+const ToolHandler = toolsMod.ToolHandler ?? toolsMod.default?.ToolHandler;
+if (typeof CodeGraph?.openSync !== 'function' || typeof ToolHandler !== 'function') {
+  console.error('could not resolve CodeGraph/ToolHandler from dist/ — run `npm run build`');
+  process.exit(2);
+}
+
+/**
+ * Pair up the starved with the overspenders they lost bytes to. Only files the
+ * render loop actually reached (a reservation and a render mode) take part —
+ * a cliffed or max-files file never had bytes to spend.
+ */
+function findStarvation(files) {
+  const spenders = files.filter(
+    (f) => f.allowance !== null && f.allowance > 0 && f.render && f.render !== 'backref',
+  );
+  const flags = [];
+  for (const s of spenders) {
+    if (s.allowance < MIN_RESERVED) continue;
+    if (s.finalChars >= s.allowance * STARVED_SHARE) continue;
+    for (const o of spenders) {
+      if (o.path === s.path) continue;
+      if (o.finalChars <= o.allowance * OVERSPEND_RATIO) continue;
+      if (o.score * SCORE_RATIO > s.score) continue;
+      flags.push({
+        starved: s.path,
+        starvedScore: s.score,
+        starvedReserved: s.allowance,
+        starvedSpent: s.finalChars,
+        overspent: o.path,
+        overspentScore: o.score,
+        overspentReserved: o.allowance,
+        overspentSpent: o.finalChars,
+      });
+    }
+  }
+  return flags;
+}
+
+const tmp = mkdtempSync(join(tmpdir(), 'cg-spend-'));
+const results = [];
+try {
+  for (const { id, q } of SUITE) {
+    if (only.length > 0 && !only.includes(id)) continue;
+    const repo = join(CORPUS, id);
+    if (!existsSync(join(repo, '.codegraph', 'codegraph.db'))) {
+      say(`${id}: no index at ${repo} — skipped`);
+      continue;
+    }
+    const sidecar = join(tmp, `${id}.jsonl`);
+    process.env.CODEGRAPH_EXPLORE_DEBUG = sidecar;
+    const cg = CodeGraph.openSync(repo);
+    const h = new ToolHandler(cg);
+    await h.execute('codegraph_explore', { query: q });
+    try { cg.close?.(); } catch { /* best effort */ }
+    const report = JSON.parse(readFileSync(sidecar, 'utf8').trim().split('\n').pop());
+    const files = report.files.map((f) => ({
+      path: f.path,
+      rank: f.rank,
+      score: f.score,
+      allowance: f.allowance,
+      spendable: f.spendable,
+      finalChars: f.finalChars,
+      render: f.render,
+      skipped: f.skipped,
+      spent: f.allowance ? f.finalChars / f.allowance : null,
+    }));
+    results.push({
+      repo: id,
+      sourceChars: report.envelope.sourceChars,
+      files,
+      flags: findStarvation(files),
+    });
+  }
+} finally {
+  rmSync(tmp, { recursive: true, force: true });
+}
+
+if (asJson) {
+  console.log(JSON.stringify(results, null, 2));
+} else {
+  const base = baselineAt ? JSON.parse(readFileSync(baselineAt, 'utf8')) : null;
+  const byRepo = new Map((base ?? []).map((r) => [r.repo, r]));
+  for (const r of results) {
+    const b = byRepo.get(r.repo);
+    say(`\n${r.repo}  —  ${num(r.sourceChars)} source chars`
+      + (b ? `  (baseline ${num(b.sourceChars)})` : ''));
+    say(' #   score  reserved   spent   spent%   render        file');
+    say('-'.repeat(96));
+    const flagged = new Set(r.flags.flatMap((f) => [f.starved, f.overspent]));
+    for (const f of r.files) {
+      if (f.allowance === null || f.allowance === 0) continue;
+      if (!showAll && !flagged.has(f.path) && f.spent > STARVED_SHARE && f.spent < OVERSPEND_RATIO) continue;
+      const mark = flagged.has(f.path) ? '*' : ' ';
+      say(
+        `${String(f.rank).padStart(2)}${mark} ${String(Math.round(f.score)).padStart(6)} `
+        + `${num(f.allowance).padStart(9)} ${num(f.finalChars).padStart(7)} `
+        + `${pct(f.spent).padStart(7)}   ${(f.render ?? f.skipped ?? '—').padEnd(13)} ${f.path}`,
+      );
+    }
+    for (const f of r.flags) {
+      say(`  FLAG: ${f.starved} (score ${Math.round(f.starvedScore)}) spent `
+        + `${num(f.starvedSpent)}/${num(f.starvedReserved)} while ${f.overspent} `
+        + `(score ${Math.round(f.overspentScore)}) spent ${num(f.overspentSpent)}/${num(f.overspentReserved)}`);
+    }
+  }
+  const total = results.reduce((n, r) => n + r.flags.length, 0);
+  say('');
+  say(total === 0
+    ? 'No file leaves a large share of its reservation unspent while a weaker file overspends.'
+    : `STARVATION: ${total} flag(s) across `
+      + `${results.filter((r) => r.flags.length > 0).map((r) => r.repo).join(', ')}.`);
+  if (base) {
+    const worse = results.filter((r) => {
+      const b = byRepo.get(r.repo);
+      return b && (r.flags.length > b.flags.length || r.sourceChars < b.sourceChars);
+    });
+    say(worse.length === 0
+      ? 'No repo flags more or delivers less than the baseline.'
+      : `REGRESSION vs baseline: ${worse.map((r) => r.repo).join(', ')}.`);
+  }
+  if (total > 0) process.exitCode = 1;
+}

+ 70 - 16
src/mcp/tools.ts

@@ -5240,9 +5240,11 @@ export class ToolHandler {
         // agent to Read, negating the savings. But "always taken" is not "taken at
         // agent to Read, negating the savings. But "always taken" is not "taken at
         // any size": when it overruns the reservation it is SHRUNK to the
         // any size": when it overruns the reservation it is SHRUNK to the
         // highest-importance whole symbol ranges inside it, so a single-cluster
         // highest-importance whole symbol ranges inside it, so a single-cluster
-        // god-file spends its allotment instead of the whole response's. Later
-        // clusters are never shrunk — they either fit or wait for another call.
+        // god-file spends its allotment instead of the whole response's.
         const first = chosenIndices.size === 0;
         const first = chosenIndices.size === 0;
+        // A spine cluster (the rendered call path) is the flow answer — it may run
+        // past the per-file budget up to the spine ceiling; non-spine clusters obey
+        // the normal per-file budget.
         const cap = rc.c.hasSpine ? SPINE_CEILING : fileBudget;
         const cap = rc.c.hasSpine ? SPINE_CEILING : fileBudget;
         // CG-30: shrinking keeps the top member whole however big it is, so bound
         // CG-30: shrinking keeps the top member whole however big it is, so bound
         // how far that member may overshoot — the same 1.5x-of-reservation bound
         // how far that member may overshoot — the same 1.5x-of-reservation bound
@@ -5250,25 +5252,45 @@ export class ToolHandler {
         // cap is never windowed). A spine cluster's cap already IS that bound, so
         // cap is never windowed). A spine cluster's cap already IS that bound, so
         // this holds it to it rather than letting the member rule walk past it.
         // this holds it to it rather than letting the member rule walk past it.
         const ceiling = Math.max(cap, SPINE_CEILING);
         const ceiling = Math.max(cap, SPINE_CEILING);
-        const section = renderCluster(rc.c, first ? cap : Infinity, first ? ceiling : Infinity);
-        const text = sectionText(section.parts);
-        const sectionLen = text.length + (!first && text.length > 0 ? GAP_MARKER.length : 0);
         if (first) {
         if (first) {
+          const section = renderCluster(rc.c, cap, ceiling);
           renderedClusters.set(rc.idx, section);
           renderedClusters.set(rc.idx, section);
           anyClusterShrunk = anyClusterShrunk || section.shrunk;
           anyClusterShrunk = anyClusterShrunk || section.shrunk;
           chosenIndices.add(rc.idx);
           chosenIndices.add(rc.idx);
-          projectedChars += sectionLen;
+          projectedChars += sectionText(section.parts).length;
           continue;
           continue;
         }
         }
-        // A spine cluster (the rendered call path) is the flow answer — include it
-        // past the per-file budget up to the spine ceiling; non-spine clusters obey
-        // the normal per-file budget.
-        const fits = projectedChars + sectionLen <= fileBudget;
-        const spineFits = rc.c.hasSpine && projectedChars + sectionLen <= SPINE_CEILING;
-        if (!fits && !spineFits) continue;
+        // Later clusters used to be all-or-nothing: rendered whole, then taken
+        // only if the whole thing fit the remainder. On a file whose top-ranked
+        // cluster is TRIVIAL that discards the answer and leaves the reservation
+        // unspent — django's `sql/query.py` keeps a 22-line glue cluster (one
+        // importance-6 bridging symbol) and drops the 624-line `Query` body
+        // beneath it whole, spending 1,923 of 7,947; the slack then carries
+        // forward to a file scoring a fifth as much (CG-36). Same shape in
+        // okhttp's `RealInterceptorChain.kt`, where an import header displaces
+        // the chain itself.
+        //
+        // So a later cluster is shrunk INTO the remainder by the same whole-member
+        // rule the first one already uses — CG-26's between-FILES lesson ("hold the
+        // remainder while it is still worth a section; zeroing it delivers
+        // nothing") applied between CLUSTERS. Below `MIN_CHARS` the remainder can't
+        // hold one readable block, so it stays a drop rather than a stutter of
+        // fragments the next call's dedup then has to shred around.
+        const room = cap - projectedChars - GAP_MARKER.length;
+        if (room < EXPLORE_ALLOCATION.MIN_CHARS) continue;
+        const section = renderCluster(rc.c, room, room);
+        const text = sectionText(section.parts);
+        if (text.length === 0) continue;
+        // The never-empty floors inside the windowing may overrun `room` (a
+        // 12-line minimum window on a file of very long lines). The first cluster
+        // is allowed that overshoot — an empty section is worse — but a later one
+        // is not: it would be spending a lower-ranked FILE's reservation for a
+        // fragment. Drop it, exactly as before.
+        if (projectedChars + text.length + GAP_MARKER.length > cap) continue;
         renderedClusters.set(rc.idx, section);
         renderedClusters.set(rc.idx, section);
+        anyClusterShrunk = anyClusterShrunk || section.shrunk;
         chosenIndices.add(rc.idx);
         chosenIndices.add(rc.idx);
-        projectedChars += sectionLen;
+        projectedChars += text.length + GAP_MARKER.length;
       }
       }
 
 
       // Emit chosen clusters in source order so the file reads top-to-bottom.
       // Emit chosen clusters in source order so the file reads top-to-bottom.
@@ -5345,15 +5367,47 @@ export class ToolHandler {
       let chosenNow = chosenIndices;
       let chosenNow = chosenIndices;
       const costOfSection = (header: string, body: string) =>
       const costOfSection = (header: string, body: string) =>
         header.length + 2 + (body.length > 0 ? body.length + lang.length + 11 : 0);
         header.length + 2 + (body.length > 0 ? body.length + lang.length + 11 : 0);
+      // The weakest cluster is SHRUNK into the room that is left before it is
+      // dropped (CG-36). Dropping it whole makes this loop as all-or-nothing as
+      // the selection above it was, and at the same cost: on excalidraw's
+      // `typeChecks.ts` the estimate missed by 13 chars and a 1,512-char cluster
+      // — the file's highest-SCORING one, last only because rank breaks ties on
+      // density — was thrown away to pay for it. Below MIN_CHARS the remainder
+      // cannot hold a readable block, and only then is the cluster dropped.
+      const reshrunkOnce = new Set<number>();
       while (totalChars + costOfSection(fileHeader, assembled.text) > renderCeiling
       while (totalChars + costOfSection(fileHeader, assembled.text) > renderCeiling
              && chosenNow.size > 1) {
              && chosenNow.size > 1) {
         // Weakest first: `rankedClusters` is best-first, so walk it backwards.
         // Weakest first: `rankedClusters` is best-first, so walk it backwards.
-        const trimmed = new Set(chosenNow);
+        let weakest = -1;
         for (let i = rankedClusters.length - 1; i >= 0; i--) {
         for (let i = rankedClusters.length - 1; i >= 0; i--) {
           const idx = rankedClusters[i]!.idx;
           const idx = rankedClusters[i]!.idx;
-          if (trimmed.has(idx)) { trimmed.delete(idx); break; }
+          if (chosenNow.has(idx)) { weakest = idx; break; }
+        }
+        if (weakest < 0) break;
+        const over = totalChars + costOfSection(fileHeader, assembled.text) - renderCeiling;
+        const current = renderedClusters.get(weakest)!;
+        const currentLen = sectionText(current.parts).length;
+        const room = currentLen - over;
+        let reduced = false;
+        // One attempt per cluster: a second pass means the first re-render did
+        // not buy enough (the header moved with it), and the cluster is then
+        // dropped rather than whittled a few chars at a time.
+        if (room >= EXPLORE_ALLOCATION.MIN_CHARS && !reshrunkOnce.has(weakest)) {
+          reshrunkOnce.add(weakest);
+          const reshrunk = renderCluster(clusters[weakest]!, room, room);
+          const reshrunkLen = sectionText(reshrunk.parts).length;
+          // Strictly smaller, or this loop cannot make progress and would spin.
+          if (reshrunkLen > 0 && reshrunkLen < currentLen) {
+            renderedClusters.set(weakest, reshrunk);
+            anyClusterShrunk = true;
+            reduced = true;
+          }
+        }
+        if (!reduced) {
+          const trimmed = new Set(chosenNow);
+          trimmed.delete(weakest);
+          chosenNow = trimmed;
         }
         }
-        chosenNow = trimmed;
         assembled = assembleSection(chosenNow);
         assembled = assembleSection(chosenNow);
         fileHeader = headerFor(assembled.symbols);
         fileHeader = headerFor(assembled.symbols);
         anyFileTrimmed = true;
         anyFileTrimmed = true;