perf(sync): defer WAL autocheckpoint for the whole incremental run (#1312)

The #1242 fix (WAL deferral + checkpoint valve, the 26x win on
HDD-class storage) was wired only into indexAll. CodeGraph.sync never
touched wal_autocheckpoint, so every incremental run kept the default
1000-page cadence and re-triggered the #1231 per-page checkpoint
thrash — a 7-file sync took 2m 2s at 0-2% CPU on the reporter's
hardware, because the cost scales with the EXISTING database's hot
pages, not the change size.

sync now mirrors indexAll exactly: defer autocheckpoint + start the
valve for the run, fold the store phase's WAL before the post-store
reads, restore the interval in the finally. Same kill switch
(CODEGRAPH_NO_WAL_DEFER=1). Idle valve cost is one timer, so
watcher-frequency syncs stay cheap.

Fixes #1248

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

CHANGELOG.md

@@ -18,6 +18,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ### Fixes
 
+- `codegraph sync` now gets the same slow-disk fix that made full indexing fast in 1.4.0: database checkpointing is deferred for the whole incremental run instead of firing every few megabytes of writes. On mechanical drives and other high-latency storage, a small sync on a large index no longer stalls for minutes at near-zero CPU — the cost of a sync scales with what changed, not with the size of the existing index. The same `CODEGRAPH_NO_WAL_DEFER=1` switch turns it off. (#1248)
 - C functions declared with a project-specific attribute macro in front of a typedef'd return type (`SEC_ATTR UINT32 MyFunc(VOID)` — common in embedded and kernel code) are now indexed under their real names. Previously the parser tripped over the unknown macro and stored the parameter list as the function name, leaving entries like `"(VOID)"` in the graph and making the real function unfindable. (#1211)
 - C++ methods defined out-of-line inside a namespace (`namespace sim { Output MyClass::Apply(...) { ... } }`) now carry the namespace in their qualified name, matching their class. Fully-qualified call sites from other files (`sim::MyClass::Apply(...)`) resolve to the definition again, so `codegraph callers` and file impact no longer come up empty for this pattern. (#1291)
 - C++ methods defined out-of-line on a template class (`template <typename T> T Box<T>::get() { ... }`) no longer keep the template parameter list in their qualified name. They now index as `Box::get` — identical to an inline definition of the same method — so they link to their class and resolve from call sites again, and pathological multi-line template parameter lists can no longer blow the qualified name past filesystem name limits. (#1286)

__tests__/wal-deferral.test.ts

@@ -177,17 +177,18 @@ describe('WalCheckpointValve', () => {
   });
 });
 
-describe('indexAll WAL deferral end-to-end', () => {
-  function writeFixtureProject(): void {
-    fs.mkdirSync(path.join(tmpDir, 'src'), { recursive: true });
-    for (let i = 0; i < 8; i++) {
-      fs.writeFileSync(
-        path.join(tmpDir, 'src', `mod${i}.ts`),
-        `export function fn${i}(x: number): number { return helper${i}(x) + ${i}; }\n` +
-        `function helper${i}(x: number): number { return x * ${i}; }\n`
-      );
-    }
+function writeFixtureProject(): void {
+  fs.mkdirSync(path.join(tmpDir, 'src'), { recursive: true });
+  for (let i = 0; i < 8; i++) {
+    fs.writeFileSync(
+      path.join(tmpDir, 'src', `mod${i}.ts`),
+      `export function fn${i}(x: number): number { return helper${i}(x) + ${i}; }\n` +
+      `function helper${i}(x: number): number { return x * ${i}; }\n`
+    );
   }
+}
+
+describe('indexAll WAL deferral end-to-end', () => {
 
   it('produces the same graph with and without deferral, and restores the interval', async () => {
     writeFixtureProject();
@@ -215,3 +216,81 @@ describe('indexAll WAL deferral end-to-end', () => {
     }
   });
 });
+
+describe('sync WAL deferral end-to-end (#1248)', () => {
+  // The #1242 fix originally landed only on indexAll; sync stayed at the
+  // default 1000-page autocheckpoint and reproduced the #1231 HDD thrash on
+  // every incremental run (2 minutes for a 7-file sync). These pin that sync
+  // defers during the run, restores after — success AND no-change paths —
+  // and that a deferred sync produces the same graph as an undeferred one.
+  it('defers the autocheckpoint interval DURING sync and restores it after', async () => {
+    writeFixtureProject();
+    const cg = CodeGraph.initSync(tmpDir);
+    await cg.indexAll();
+    const conn = (cg as unknown as { db: DatabaseConnection }).db;
+
+    fs.writeFileSync(
+      path.join(tmpDir, 'src', 'mod0.ts'),
+      `export function fn0(x: number): number { return helper0(x) + 100; }\n` +
+      `function helper0(x: number): number { return x * 100; }\n`
+    );
+
+    // Sample the interval mid-run from inside the progress callback — the
+    // store loop is exactly where the #1248 thrash happened.
+    const midRunIntervals: number[] = [];
+    const result = await cg.sync({
+      onProgress: () => {
+        try { midRunIntervals.push(conn.getWalAutocheckpoint()); } catch { /* ignore */ }
+      },
+    });
+    expect(result.filesModified).toBe(1);
+    expect(midRunIntervals.length).toBeGreaterThan(0);
+    expect(midRunIntervals.every((v) => v === 0)).toBe(true);
+    // Scoped to the run: back on the default afterwards.
+    expect(conn.getWalAutocheckpoint()).toBe(1000);
+    await cg.close();
+  });
+
+  it('restores the interval on a no-change sync too', async () => {
+    writeFixtureProject();
+    const cg = CodeGraph.initSync(tmpDir);
+    await cg.indexAll();
+    const conn = (cg as unknown as { db: DatabaseConnection }).db;
+    const result = await cg.sync();
+    expect(result.filesAdded + result.filesModified + result.filesRemoved).toBe(0);
+    expect(conn.getWalAutocheckpoint()).toBe(1000);
+    await cg.close();
+  });
+
+  it('produces the same sync result with and without deferral', async () => {
+    writeFixtureProject();
+    const cg1 = CodeGraph.initSync(tmpDir);
+    await cg1.indexAll();
+    fs.writeFileSync(
+      path.join(tmpDir, 'src', 'mod1.ts'),
+      `export function fn1(x: number): number { return helper1(x) + 111; }\n` +
+      `function helper1(x: number): number { return x * 111; }\n`
+    );
+    const r1 = await cg1.sync();
+    const counts1 = { modified: r1.filesModified, nodes: r1.nodesUpdated };
+    await cg1.close();
+
+    fs.rmSync(path.join(tmpDir, '.codegraph'), { recursive: true, force: true });
+
+    process.env.CODEGRAPH_NO_WAL_DEFER = '1';
+    try {
+      const cg2 = CodeGraph.initSync(tmpDir);
+      await cg2.indexAll();
+      fs.writeFileSync(
+        path.join(tmpDir, 'src', 'mod1.ts'),
+        `export function fn1(x: number): number { return helper1(x) + 222; }\n` +
+        `function helper1(x: number): number { return x * 222; }\n`
+      );
+      const r2 = await cg2.sync();
+      expect({ modified: r2.filesModified, nodes: r2.nodesUpdated }).toEqual(counts1);
+      await cg2.close();
+    } finally {
+      delete process.env.CODEGRAPH_NO_WAL_DEFER;
+    }
+  });
+});

src/index.ts

@@ -705,6 +705,31 @@ export class CodeGraph {
       } catch {
         return { filesChecked: 0, filesAdded: 0, filesModified: 0, filesRemoved: 0, nodesUpdated: 0, durationMs: 0 };
       }
+      // Defer WAL auto-checkpointing for the whole incremental run, exactly
+      // as indexAll does for the bulk path (#1231): sync's store loop and its
+      // resolution passes churn the same FTS + secondary-index hot pages, and
+      // at the default 1000-page cadence the inline checkpoints re-write them
+      // over and over — on HDD-class storage a 7-file sync took 2 minutes at
+      // 0-2% CPU (#1248). The cost scales with the EXISTING database size,
+      // not the change size, so small syncs on big indexes hurt most. The
+      // valve bounds WAL growth off-thread; runMaintenance at the end does
+      // the final fold-up before the interval is restored in the finally.
+      // Same kill switch as indexAll: CODEGRAPH_NO_WAL_DEFER=1. Idle valve
+      // cost is one timer, so watcher-frequency syncs stay cheap.
+      const deferWal = process.env.CODEGRAPH_NO_WAL_DEFER !== '1' && this.db.getJournalMode() === 'wal';
+      let walValve: WalCheckpointValve | null = null;
+      let priorAutocheckpoint = 1000;
+      if (deferWal) {
+        priorAutocheckpoint = this.db.getWalAutocheckpoint();
+        this.db.setWalAutocheckpoint(0);
+        walValve = new WalCheckpointValve(
+          this.db,
+          undefined,
+          undefined,
+          options.verbose ? (m) => console.log(`[wal-valve] ${m}`) : undefined
+        );
+        walValve.start();
+      }
       try {
         // Captured BEFORE the sync runs: the sync's own incremental writes
         // populate vocab rows for the files it touches, so an end-of-sync
@@ -716,6 +741,11 @@ export class CodeGraph {
 
         const result = await this.orchestrator.sync(options.onProgress);
 
+        // Fold the store phase's WAL BEFORE the post-store reads below
+        // (resolution reads on the main thread) — same rationale as
+        // indexAll's fold between store and resolution.
+        if (walValve) await walValve.foldNow();
+
         // Cross-file finalization (e.g. NestJS RouterModule prefixes). Run on
         // every sync that touched files so edits to `app.module.ts` propagate
         // to controllers in unchanged files. The pass is idempotent and cheap
@@ -881,6 +911,14 @@ export class CodeGraph {
 
         return result;
       } finally {
+        // Mirror indexAll's teardown: stop the valve, then restore the
+        // auto-checkpoint interval (runMaintenance above already folded the
+        // WAL on the success path; on the error path SQLite replays it on
+        // the next open).
+        if (walValve) { walValve.stop(); await walValve.drain(); }
+        if (deferWal) {
+          try { this.db.setWalAutocheckpoint(priorAutocheckpoint); } catch { /* connection may be closing */ }
+        }
         this.fileLock.release();
       }
     });