perf(resolution): defer checkpoints and double-buffer persist during resolution, byte-identical graphs (#1320)

Fresh init on a 4,402-file Java repo (dubbo): 18.2s -> 13.5s (-26%), with
the resolution phase going 12.6s -> 7.9s (-38%). Graphs verified
byte-identical on both the pool path (dubbo) and the sequential path
(excalidraw).

Two changes:

1. The fastInit+pool path restored WAL for the resolver workers but left
   wal_autocheckpoint at its default, so the persist loop inline-checkpointed
   hot pages all phase long (#1231's pathology inside resolution — measured
   at 58% of resolution wall). Checkpointing is now deferred behind the
   bounded valve and folded once at maintenance, mirroring the deferWal path.

2. The resolution loop is double-buffered: batch k+1 is prefetched (OFFSET
   past batch k's still-pending rows, under an explicit ORDER BY rowid) and
   fanned out across the pool while batch k's ref cleanup runs on the main
   thread. Batch settle-waits dropped 2572ms -> 117ms.

Correctness invariant found by the byte-identical gate and now documented in
the loop: batch k+1's resolution READS batch k's edges (resolveMethodOnType
walks supertype chains over extends/implements edges that resolution itself
inserts), so edges must persist BEFORE the next batch fans out; only the ref
cleanup overlaps.

Also extends the CODEGRAPH_SYNTH_TIMINGS instrumentation with phase labels
(grammar-init, parse-loop, fts-rebuild, resolver-reinit, resolution,
callback-synthesis) and pool timings (worker open/resolve, per-batch mode,
persist), so the next profile is one env var away.

Suite green (2444). Sync path timings unchanged. Pool floor re-validated:
forced-on at 40k refs is still net-slower, so the 150k threshold stands.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -12,6 +12,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 ### New Features
 
 - Reference resolution now runs in parallel on large projects. When a project has enough pending references to make it worthwhile (roughly 150k+, typical for big Java/Kotlin/Spring codebases), resolution fans out across worker threads while results are applied in the exact order the single-threaded path would have used — the graph comes out byte-for-byte identical, about twice as fast end-to-end on a 4,000-file Java project in our testing. Small projects keep the single-threaded path automatically (the fan-out costs more than it saves there). Set `CODEGRAPH_NO_PARALLEL_RESOLVE=1` to disable, or `CODEGRAPH_PARALLEL_RESOLVE_MIN=<count>` to tune when it engages.
+- Indexing large projects got another sizeable speedup — about a quarter less wall-clock on the same 4,000-file Java project, with the graph still byte-for-byte identical. Two changes: the database no longer interleaves expensive checkpoint housekeeping into the middle of resolution on a fresh index (it's folded once at the end instead), and while one batch's results are being written out, the worker threads are already resolving the next batch instead of sitting idle.
 - Indexing is significantly faster — a fresh `codegraph init` on a medium TypeScript project takes about a third less wall-clock time, with the same graph produced byte-for-byte. The gains come from batching database writes, storing files on a dedicated writer thread, memoizing repeated import-resolution lookups, skipping per-row search-index maintenance during the bulk build (rebuilt once at the end), and — on completely fresh databases only — deferring disk durability until the index completes, since an interrupted first index is simply re-run. Set `CODEGRAPH_NO_FAST_INIT=1` to keep full crash-durability during the initial build, or `CODEGRAPH_NO_STORE_WORKER=1` to store on the main thread.
 - `codegraph install` and `codegraph upgrade` now offer CodeGraph Pro beta access after finishing — answer yes, type your email, and you join the same waitlist as the getcodegraph.com homepage form. Strictly opt-in and asked at most once per machine total: nothing is sent unless you say yes and enter an email, either answer is remembered so no later install or upgrade ever re-asks, and non-interactive runs (`--yes`, scripts, CI) never see the question.
 - Every release is now cryptographically verifiable: npm packages publish with npm provenance (the "Provenance" badge on npmjs.com, proving each version was built by this repository's release workflow from a specific commit), and the GitHub Release bundles carry signed build attestations you can check with `gh attestation verify <file> -R colbymchenry/codegraph`.

src/db/queries.ts

@@ -2061,8 +2061,13 @@ export class QueryBuilder {
    */
   getUnresolvedReferencesBatch(offset: number, limit: number): UnresolvedReference[] {
     if (!this.stmts.getUnresolvedBatch) {
+      // ORDER BY rowid is load-bearing for the pipelined resolution loop: it
+      // prefetches batch k+1 at OFFSET batch_k.length while batch k's rows are
+      // still pending, which is only exact under a stable enumeration. (A plain
+      // scan and the status index both return rowid order anyway — this pins
+      // it.)
       this.stmts.getUnresolvedBatch = this.db.prepare(
-        "SELECT * FROM unresolved_refs WHERE status = 'pending' LIMIT ? OFFSET ?"
+        "SELECT * FROM unresolved_refs WHERE status = 'pending' ORDER BY rowid LIMIT ? OFFSET ?"
       );
     }
     const rows = this.stmts.getUnresolvedBatch.all(limit, offset) as UnresolvedRefRow[];

src/extraction/index.ts

@@ -1520,7 +1520,9 @@ export class ExtractionOrchestrator {
     // in file order preserves the #1015 determinism exactly.
     storeWriterOpts?: { dbPath: string; fastInit: boolean } | null
   ): Promise<IndexResult> {
+    const tGrammar = Date.now();
     await initGrammars();
+    if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] grammar-init: ${Date.now() - tGrammar}ms`);
     const startTime = Date.now();
     const errors: ExtractionError[] = [];
     let filesIndexed = 0;
@@ -1816,6 +1818,7 @@ export class ExtractionOrchestrator {
       }
     };
 
+    const tParseLoop = Date.now();
     for (let i = 0; i < files.length; i += FILE_IO_BATCH_SIZE) {
       if (signal?.aborted) { aborted = true; break; }
 
@@ -1907,6 +1910,7 @@ export class ExtractionOrchestrator {
         }
       }
     }
+    if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] parse-loop: ${Date.now() - tParseLoop}ms`);
 
     if (signal?.aborted || aborted) {
       if (storeWriter) await storeWriter.close();

src/index.ts

@@ -464,6 +464,9 @@ export class CodeGraph {
       const deferWal = !fastInit && process.env.CODEGRAPH_NO_WAL_DEFER !== '1' && this.db.getJournalMode() === 'wal';
       let walValve: WalCheckpointValve | null = null;
       let priorAutocheckpoint = 1000;
+      // Set when the fastInit+pool path below defers autocheckpointing, so the
+      // finally knows to restore the interval on that path too.
+      let restoreAutocheckpoint = false;
       if (deferWal) {
         priorAutocheckpoint = this.db.getWalAutocheckpoint();
         this.db.setWalAutocheckpoint(0);
@@ -503,7 +506,9 @@ export class CodeGraph {
             freshDb ? { dbPath: this.db.getPath(), fastInit } : null
           );
         } finally {
+          const tFts = Date.now();
           this.db.endBulkNodeLoad();
+          if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] fts-rebuild: ${Date.now() - tFts}ms`);
         }
 
         // Fold the parse phase's WAL BEFORE the first post-parse reads
@@ -521,10 +526,12 @@ export class CodeGraph {
         // and silently drop themselves. Re-initializing here gives them a
         // chance to see the actual project before resolution runs.
         if (result.success && result.filesIndexed > 0) {
+          const tReinit = Date.now();
           this.resolver.initialize();
           // Cross-file finalization (e.g. NestJS RouterModule prefixes). Runs
           // before resolution so updated names show up in subsequent reads.
           this.resolver.runPostExtract();
+          if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] resolver-reinit: ${Date.now() - tReinit}ms`);
         }
 
         // Resolve references to create call/import/extends edges
@@ -542,6 +549,24 @@ export class CodeGraph {
             try {
               this.db.getDb().pragma('synchronous = NORMAL');
               this.db.getDb().pragma('journal_mode = WAL');
+              // Defer auto-checkpointing for the resolution phase, same
+              // rationale as the deferWal path above: at the default 1000-page
+              // interval, the persist loop's edge inserts + ref deletes make
+              // SQLite re-write hot B-tree pages into the main DB file inline
+              // on the writer over and over (#1231's pathology — measured as
+              // ~58% of the resolution phase on a 255k-ref repo). The valve
+              // bounds WAL growth off-thread; runMaintenance does the final
+              // fold and the finally restores the interval.
+              priorAutocheckpoint = this.db.getWalAutocheckpoint();
+              this.db.setWalAutocheckpoint(0);
+              restoreAutocheckpoint = true;
+              walValve = new WalCheckpointValve(
+                this.db,
+                undefined,
+                undefined,
+                options.verbose ? (m) => console.log(`[wal-valve] ${m}`) : undefined
+              );
+              walValve.start();
             } catch { /* keep current mode; resolution still works sequentially */ }
           }
 
@@ -551,6 +576,7 @@ export class CodeGraph {
             total: unresolvedCount,
           });
 
+          const tResolve = Date.now();
           await this.resolveReferencesBatched(
             (current, total) => {
               options.onProgress?.({
@@ -567,6 +593,7 @@ export class CodeGraph {
               });
             }
           );
+          if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] resolution: ${Date.now() - tResolve}ms`);
 
           // Second pass: chained calls whose method lives on a supertype the
           // receiver conforms to (protocol-extension / inherited / default-
@@ -658,7 +685,7 @@ export class CodeGraph {
         // (SQLite replays the WAL on the next open) and the follow-up write
         // that folds it is the known cost of a failed run.
         if (walValve) { walValve.stop(); await walValve.drain(); }
-        if (deferWal) {
+        if (deferWal || restoreAutocheckpoint) {
           try { this.db.setWalAutocheckpoint(priorAutocheckpoint); } catch { /* connection may be closing */ }
         }
         if (fastInit) {

src/resolution/index.ts

@@ -1357,52 +1357,98 @@ export class ReferenceResolver {
     // costs zero wall-clock. Any failure downgrades to sequential permanently.
     let pool: ResolverPool | null = null;
     let poolReady = false;
+    const tPoolStart = Date.now();
     if (parallel && total >= minRefsForPool()) {
       pool = ResolverPool.tryCreate(parallel.dbPath, this.projectRoot);
       pool?.ready().then(
-        () => { poolReady = true; },
+        () => {
+          poolReady = true;
+          if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[pool-timing] pool ready after ${Date.now() - tPoolStart}ms`);
+        },
         () => { void pool?.destroy().catch(() => undefined); pool = null; }
       );
     }
 
-    // Process in batches. We always read from offset 0 because every ref the
-    // batch processed leaves the pending set (resolved rows are deleted,
+    // Process in PIPELINED batches (double-buffer). The enumeration is the
+    // head of the pending set in rowid order; every ref a persisted batch
+    // processed leaves the pending set (resolved rows are deleted,
     // unresolvable ones flip to status='failed'), shifting the remaining
     // pending rows forward.
     let prevRemaining = Number.POSITIVE_INFINITY;
-    try {
-    while (true) {
-      const batch = this.queries.getUnresolvedReferencesBatch(0, batchSize);
-      if (batch.length === 0) break;
 
-      let result: ResolutionResult;
+    // Fan-out result of ResolverPool.resolveBatch, settled (never rejecting)
+    // so a fan-out begun before the previous batch's persist can't produce an
+    // unhandled rejection while it waits to be awaited.
+    type PoolSettled =
+      | { ok: true; out: Awaited<ReturnType<ResolverPool['resolveBatch']>> }
+      | { ok: false; err: unknown };
+    type InFlight = { mode: 'pool'; settled: Promise<PoolSettled> } | { mode: 'seq' };
+
+    // Begin one batch: fan out to the pool when it's ready and the batch is
+    // big enough — workers then resolve batch k+1 WHILE the main thread
+    // persists batch k (persist measured at ~58% of resolution wall on a
+    // 255k-ref repo, all of it previously spent with the pool idle).
+    // Sequential batches stay lazy: they run on the main thread at settle
+    // time, where an early start would only contend with the persist.
+    const beginBatch = (batch: UnresolvedReference[]): InFlight => {
       if (pool && poolReady && ResolverPool.worthParallel(batch.length)) {
-        try {
-          const out = await pool.resolveBatch(batch);
-          // Deferred post-pass refs ride back from the workers; re-queue them
-          // in admission order so the post-passes see the sequential order.
-          this.appendDeferredFromWorkers(out.deferredChain, out.deferredThisMember);
-          result = {
-            resolved: out.resolved,
-            unresolved: out.unresolved,
+        return {
+          mode: 'pool',
+          settled: pool.resolveBatch(batch).then(
+            (out) => ({ ok: true as const, out }),
+            (err: unknown) => ({ ok: false as const, err })
+          ),
+        };
+      }
+      return { mode: 'seq' };
+    };
+
+    // Settle an in-flight batch to a ResolutionResult. Deferred post-pass refs
+    // are appended HERE, in loop order — never inside the fan-out promise — so
+    // admission order stays exactly the sequential order even while a later
+    // batch resolves concurrently. A pool failure downgrades to sequential
+    // permanently and re-resolves this batch on the main thread.
+    const settleBatch = async (
+      inFlight: InFlight,
+      batch: UnresolvedReference[]
+    ): Promise<ResolutionResult> => {
+      if (inFlight.mode === 'pool') {
+        const settled = await inFlight.settled;
+        if (settled.ok) {
+          this.appendDeferredFromWorkers(settled.out.deferredChain, settled.out.deferredThisMember);
+          return {
+            resolved: settled.out.resolved,
+            unresolved: settled.out.unresolved,
             stats: {
               total: batch.length,
-              resolved: out.resolved.length,
-              unresolved: out.unresolved.length,
-              byMethod: out.byMethod,
+              resolved: settled.out.resolved.length,
+              unresolved: settled.out.unresolved.length,
+              byMethod: settled.out.byMethod,
             },
           };
-        } catch (err) {
-          logDebug('Parallel resolution failed; falling back to sequential', {
-            error: err instanceof Error ? err.message : String(err),
-          });
-          await pool.destroy().catch(() => undefined);
-          pool = null;
-          result = await this.resolveBatchYielding(batch, maybeYield);
         }
-      } else {
-        result = await this.resolveBatchYielding(batch, maybeYield);
+        logDebug('Parallel resolution failed; falling back to sequential', {
+          error: settled.err instanceof Error ? settled.err.message : String(settled.err),
+        });
+        if (pool) await pool.destroy().catch(() => undefined);
+        pool = null;
       }
+      return this.resolveBatchYielding(batch, maybeYield);
+    };
+
+    try {
+    let batch = this.queries.getUnresolvedReferencesBatch(0, batchSize);
+    let inFlight: InFlight | null = batch.length > 0 ? beginBatch(batch) : null;
+    while (batch.length > 0 && inFlight) {
+      // Prefetch the NEXT batch before this one persists: this batch's rows
+      // are still pending (nothing has mutated the table since they were
+      // read), so skipping exactly batch.length rows in the same rowid
+      // enumeration yields the following batch.
+      const nextBatch = this.queries.getUnresolvedReferencesBatch(batch.length, batchSize);
+
+      const tBatch = Date.now();
+      const result = await settleBatch(inFlight, batch);
+      if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[pool-timing] batch ${inFlight.mode}: ${batch.length} refs in ${Date.now() - tBatch}ms`);
 
       // Persist in bounded sub-transactions with yields between: a whole
       // batch's edge insert / keyed deletes are otherwise one solid
@@ -1412,14 +1458,27 @@ export class ReferenceResolver {
       // land before their refs are deleted, so a kill mid-way re-resolves
       // the remainder idempotently on the next run/sweep (#1187).
       const PERSIST_CHUNK = 1000;
-
-      // Persist edges immediately
+      const tPersist = Date.now();
+
+      // Persist edges BEFORE fanning out the next batch: later batches read
+      // this batch's edges — resolveMethodOnType walks supertype chains over
+      // `extends`/`implements` edges that earlier batches resolved, so a
+      // receiver typed as a subclass only reaches a method declared on its
+      // base class if those edges are visible. (Validated on dubbo: fanning
+      // out first downgraded exactly those supertype-method resolutions from
+      // the 0.9 typed-receiver path to the 0.65 word-overlap fallback.)
       const edges = this.createEdges(result.resolved);
       for (let i = 0; i < edges.length; i += PERSIST_CHUNK) {
         this.queries.insertEdges(edges.slice(i, i + PERSIST_CHUNK));
         await maybeYield();
       }
 
+      // NOW fan the next batch out — workers see exactly the edge state the
+      // sequential baseline would (every batch ≤ this one committed), while
+      // the main thread spends the REST of the persist (ref deletes + failed
+      // parking below) overlapped with their resolution — the double-buffer.
+      const nextInFlight = nextBatch.length > 0 ? beginBatch(nextBatch) : null;
+
       // Clean up resolved refs so they don't appear in the next batch —
       // by row id, so a same-key sibling ref in a LATER batch (same caller
       // calling the same callee at another line) is left pending for its own
@@ -1448,6 +1507,8 @@ export class ReferenceResolver {
         await maybeYield();
       }
 
+      if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[pool-timing] batch persist: ${Date.now() - tPersist}ms`);
+
       // Aggregate stats
       aggregateStats.total += result.stats.total;
       aggregateStats.resolved += result.stats.resolved;
@@ -1470,18 +1531,25 @@ export class ReferenceResolver {
       // batch one and left the rest of the table as permanent orphans (#1187).
       // The count-based guard below catches the true no-progress case.
 
-      // Non-progress guard (defense-in-depth). Because we re-read from offset 0
-      // each pass, the PENDING population MUST shrink every iteration — resolved
-      // refs are deleted and unresolvable ones are marked failed above, and both
-      // leave the pending set the batch reader sees. If it didn't shrink, a
-      // resolver returned a match whose `original.referenceName` differs from the
-      // stored row, so the keyed delete/update no-ops, and we'd re-read +
-      // re-resolve + re-insert the same rows forever (the runaway that grew a
-      // 99-file repo to 5M edges / 1.4 GB before the Go-fallback fix). Stop
-      // rather than grow the graph without bound.
+      // Non-progress guard (defense-in-depth). Each iteration enumerates from
+      // the head of the pending set, so the PENDING population MUST shrink
+      // every iteration — resolved refs are deleted and unresolvable ones are
+      // marked failed above, and both leave the pending set the batch reader
+      // sees. If it didn't shrink, a resolver returned a match whose
+      // `original.referenceName` differs from the stored row, so the keyed
+      // delete/update no-ops, and we'd re-read + re-resolve + re-insert the
+      // same rows forever (the runaway that grew a 99-file repo to 5M edges /
+      // 1.4 GB before the Go-fallback fix). Stop rather than grow the graph
+      // without bound. (An in-flight prefetched batch is abandoned unsettled —
+      // fan-out has no side effects until settleBatch appends its results.)
       const remaining = this.queries.getUnresolvedReferencesCount();
       if (remaining >= prevRemaining) break;
       prevRemaining = remaining;
+
+      // Advance the pipeline: the prefetched batch (already fanned out when
+      // the pool is on) becomes the current one.
+      batch = nextBatch;
+      inFlight = nextInFlight;
     }
     } finally {
       if (pool) await pool.destroy().catch(() => undefined);
@@ -1491,6 +1559,7 @@ export class ReferenceResolver {
     // synthesize observer/callback dispatch edges (dispatcher → registered
     // callbacks) that static parsing leaves out. Best-effort — never fail the
     // index on it. See docs/design/callback-edge-synthesis.md.
+    const tSynth = Date.now();
     try {
       aggregateStats.byMethod['callback-synthesis'] = await synthesizeCallbackEdges(
         this.queries,
@@ -1500,6 +1569,7 @@ export class ReferenceResolver {
     } catch {
       // synthesis is additive and optional; ignore failures
     }
+    if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[phase-timing] callback-synthesis: ${Date.now() - tSynth}ms`);
 
     return {
       resolved: [],

src/resolution/resolver-worker.ts

@@ -43,19 +43,24 @@ port.on('message', (msg: InMessage) => {
   try {
     switch (msg.type) {
       case 'open': {
+        const tOpen = Date.now();
         const created = createDatabase(msg.dbPath, { readOnly: true });
         db = created.db;
         db.pragma('busy_timeout = 5000');
         db.pragma('cache_size = -32000');
+        const tDb = Date.now();
         const queries = new QueryBuilder(db);
         resolver = new ReferenceResolver(msg.projectRoot, queries);
         resolver.initialize();
+        if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[pool-timing] worker open: db=${tDb - tOpen}ms init=${Date.now() - tDb}ms`);
         port.postMessage({ type: 'ready' });
         break;
       }
       case 'resolve': {
         if (!resolver) throw new Error('resolver-worker: resolve before open');
+        const tRes = Date.now();
         const out = resolver.resolveListForAdmission(msg.refs);
+        if (process.env.CODEGRAPH_SYNTH_TIMINGS) console.error(`[pool-timing] worker resolve: ${msg.refs.length} refs in ${Date.now() - tRes}ms`);
         port.postMessage({ type: 'result', id: msg.id, ...out });
         break;
       }