Просмотр исходного кода

fix(ui): show synthesis as a 'Linking dynamic dispatch' phase; mute node:sqlite warning spam (#1299)

Two first-run UX bugs surfaced by indexing a real 1,342-file C repo:

1. After 'Resolving refs' hit 100%, the ~40 dynamic-dispatch synthesis
   passes ran with no progress surface, so the bar sat frozen at 100%
   long enough to read as a hang (the C fn-pointer pass alone can hold
   for a while on C-heavy repos). Synthesis now reports per-pass
   progress through a new 'linking' IndexProgress phase, rendered as
   'Linking dynamic dispatch'. The step total is pinned by a test to
   the synthesizer's actual __mark() count so adding a pass without
   bumping it fails loudly.

2. node:sqlite's ExperimentalWarning is emitted once per THREAD, so the
   main process plus every parse worker printed it mid-index,
   interleaved with the progress UI. All launch paths now pass
   --disable-warning=ExperimentalWarning: both bundle launchers, the
   Windows npm-shim invocation, and the CLI self-relaunch
   (NODE_RUNTIME_FLAGS, deliberately excluded from the re-exec gate so
   an older installed launcher never triggers a pointless re-exec, and
   version-gated off nodes older than the flag).

Verified end-to-end on the same repo: zero warnings, live linking bar,
byte-identical graph (50,520 nodes / 148,232 edges). Full suite green.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Colby Mchenry 1 месяц назад
Родитель
Сommit
ad5300a601

+ 2 - 0
CHANGELOG.md

@@ -18,6 +18,8 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 - Callers and impact analysis no longer silently under-count a function that calls the same callee many times. When one caller contained several call sites to the same callee and an internal resolution batch boundary happened to split them, cleanup after the first batch removed the later sites' pending rows before they were ever attempted — their edges were never created, deterministically, and which edges went missing shifted with unrelated changes to the project's total reference count. Post-pass cleanup now targets the exact database row each processed reference came from. Found while validating the operator-call fix on nlohmann/json, where `write_cbor`'s 11 calls to `to_char_type` indexed as 10. (#1269)
 - C++ explicit operator calls — `a.operator+(b)`, `p->operator+(b)`, `a.operator[](3)`, and the other symbolic forms — now produce a `calls` edge to the operator method, so an operator invoked only through the explicit syntax no longer looks uncalled in callers and impact analysis. tree-sitter parses these call sites with the operator name stranded in an error node (never as a normal member access), so the call's target was silently read as just the receiver variable; the operator name is now recovered from the error node and resolved through receiver-type inference like any other member call — a same-named operator on an unrelated class can never capture the edge. Infix uses (`a + b`, `a[i]`) need real type inference and are tracked separately. (#1247)
+- `codegraph init` and `codegraph index` no longer look hung after "Resolving refs" reaches 100%. The dynamic-dispatch linking that runs after resolution (callbacks, React re-renders, C function pointers, and the rest) had no progress display, so on repos where it takes a while — large C codebases especially — the bar just sat frozen at 100% until it finished. That work now shows as its own "Linking dynamic dispatch" progress phase.
+- Indexing no longer prints repeated "SQLite is an experimental feature" warnings that garbled the progress display. The warning comes from Node's built-in SQLite and fired once per parsing worker; it's now suppressed on every launch path.
 
 ## [1.4.1] - 2026-07-10
 

+ 67 - 0
__tests__/synthesis-progress.test.ts

@@ -0,0 +1,67 @@
+/**
+ * Progress reporting for the callback-edge synthesis tail.
+ *
+ * Synthesis runs AFTER the resolution bar reaches 100%, so before this it had
+ * no progress surface at all — on synthesizer-heavy repos (e.g. large C
+ * codebases hitting the fn-pointer pass) the CLI sat frozen at
+ * "Resolving refs 100%" long enough that users concluded the index hung and
+ * killed it. These tests pin (a) that indexing emits the dedicated 'linking'
+ * phase with monotonic per-pass progress, and (b) that the advertised step
+ * total stays in sync with the synthesizer's actual pass list.
+ */
+import { describe, it, expect } from 'vitest';
+import * as fs from 'fs';
+import * as os from 'os';
+import * as path from 'path';
+import { CodeGraph, IndexProgress } from '../src/index';
+import { SYNTH_PROGRESS_STEPS } from '../src/resolution/callback-synthesizer';
+
+describe('synthesis progress ("Linking dynamic dispatch" phase)', () => {
+  it('SYNTH_PROGRESS_STEPS matches the synthesizer’s actual __mark() step count', () => {
+    // The constant is cosmetic (progress denominator), but drift makes the bar
+    // end early or jump to 100% — adding a pass must bump it. Every step site
+    // calls __mark('<label>') with a string literal, so count those.
+    const src = fs.readFileSync(
+      path.join(__dirname, '../src/resolution/callback-synthesizer.ts'),
+      'utf8'
+    );
+    const stepSites = (src.match(/__mark\('/g) ?? []).length;
+    expect(SYNTH_PROGRESS_STEPS).toBe(stepSites);
+  });
+
+  it('indexing emits a monotonic linking phase ending at the full step count', async () => {
+    const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-synth-progress-'));
+    try {
+      fs.writeFileSync(path.join(dir, 'a.ts'), 'export function helper() { return 1; }\n');
+      fs.writeFileSync(
+        path.join(dir, 'b.ts'),
+        "import { helper } from './a';\nexport function main() { return helper(); }\n"
+      );
+
+      const events: IndexProgress[] = [];
+      const cg = await CodeGraph.init(dir, {
+        index: true,
+        onProgress: (p) => events.push(p),
+      });
+      await cg.close();
+
+      const linking = events.filter((e) => e.phase === 'linking');
+      expect(linking.length).toBeGreaterThan(0);
+      // Emitted up-front so the phase label flips as soon as synthesis starts…
+      expect(linking[0]!.current).toBe(0);
+      // …and every step reports against the same total, monotonically.
+      expect(linking.every((e) => e.total === SYNTH_PROGRESS_STEPS)).toBe(true);
+      for (let i = 1; i < linking.length; i++) {
+        expect(linking[i]!.current).toBeGreaterThanOrEqual(linking[i - 1]!.current);
+      }
+      expect(linking[linking.length - 1]!.current).toBe(SYNTH_PROGRESS_STEPS);
+
+      // The linking phase comes after resolution has finished.
+      const lastResolving = events.map((e) => e.phase).lastIndexOf('resolving');
+      const firstLinking = events.map((e) => e.phase).indexOf('linking');
+      expect(firstLinking).toBeGreaterThan(lastResolving);
+    } finally {
+      fs.rmSync(dir, { recursive: true, force: true });
+    }
+  });
+});

+ 46 - 3
__tests__/wasm-runtime-flags.test.ts

@@ -16,6 +16,8 @@ import * as os from 'os';
 import * as path from 'path';
 import {
   WASM_RUNTIME_FLAGS,
+  NODE_RUNTIME_FLAGS,
+  nodeRuntimeFlagsFor,
   processHasWasmRuntimeFlags,
   buildRelaunchArgv,
 } from '../src/extraction/wasm-runtime-flags';
@@ -42,6 +44,36 @@ describe('WASM_RUNTIME_FLAGS', () => {
   });
 });
 
+describe('NODE_RUNTIME_FLAGS', () => {
+  it('suppresses the node:sqlite ExperimentalWarning on this runtime', () => {
+    // The warning is emitted once per THREAD (main + every parse worker), so
+    // during indexing it repeatedly interleaves with the progress UI. Prove
+    // the flag both launches node and actually silences the warning.
+    expect(NODE_RUNTIME_FLAGS).toContain('--disable-warning=ExperimentalWarning');
+    const res = spawnSync(
+      process.execPath,
+      [...NODE_RUNTIME_FLAGS, '-e', "require('node:sqlite'); process.exit(0)"],
+      { encoding: 'utf8' }
+    );
+    expect(res.status, res.stderr).toBe(0);
+    expect(res.stderr).not.toMatch(/ExperimentalWarning/);
+  });
+
+  it('is empty on nodes too old for --disable-warning (fatal "bad option" there)', () => {
+    expect(nodeRuntimeFlagsFor('20.10.0')).toEqual([]);
+    expect(nodeRuntimeFlagsFor('21.2.0')).toEqual([]);
+    expect(nodeRuntimeFlagsFor('20.11.0')).toContain('--disable-warning=ExperimentalWarning');
+    expect(nodeRuntimeFlagsFor('21.3.0')).toContain('--disable-warning=ExperimentalWarning');
+    expect(nodeRuntimeFlagsFor('22.5.0')).toContain('--disable-warning=ExperimentalWarning');
+  });
+
+  it('is NOT required by the re-exec gate (old-launcher compat)', () => {
+    // An installed bundle launcher that passes only the WASM flags must not
+    // trigger a pointless re-exec over a cosmetic warning flag.
+    expect(processHasWasmRuntimeFlags(['--liftoff-only'])).toBe(true);
+  });
+});
+
 describe('processHasWasmRuntimeFlags', () => {
   it('is true only when every required flag is present', () => {
     expect(processHasWasmRuntimeFlags(['--liftoff-only'])).toBe(true);
@@ -55,8 +87,9 @@ describe('processHasWasmRuntimeFlags', () => {
 });
 
 describe('buildRelaunchArgv', () => {
-  it('places the wasm flags first, then the script and its args', () => {
+  it('places our flags first, then the script and its args', () => {
     expect(buildRelaunchArgv('/x/codegraph.js', ['index', '/repo'], [])).toEqual([
+      ...NODE_RUNTIME_FLAGS,
       '--liftoff-only',
       '/x/codegraph.js',
       'index',
@@ -66,8 +99,18 @@ describe('buildRelaunchArgv', () => {
 
   it('preserves other existing node flags without duplicating ours', () => {
     expect(
-      buildRelaunchArgv('/x/codegraph.js', ['status'], ['--liftoff-only', '--enable-source-maps'])
-    ).toEqual(['--liftoff-only', '--enable-source-maps', '/x/codegraph.js', 'status']);
+      buildRelaunchArgv('/x/codegraph.js', ['status'], [
+        '--liftoff-only',
+        ...NODE_RUNTIME_FLAGS,
+        '--enable-source-maps',
+      ])
+    ).toEqual([
+      ...NODE_RUNTIME_FLAGS,
+      '--liftoff-only',
+      '--enable-source-maps',
+      '/x/codegraph.js',
+      'status',
+    ]);
   });
 
   it('produces an argv that actually launches node WITH the flag applied', () => {

+ 4 - 2
scripts/build-bundle.sh

@@ -81,7 +81,7 @@ rm -f "$STAGE/lib/package-lock.json"
 # runs are covered too; passing it here avoids that extra spawn.)
 if [ "$OSFAM" = "win32" ]; then
   cp "$NODE_BIN" "$STAGE/node.exe"
-  printf '@"%%~dp0..\\node.exe" --liftoff-only "%%~dp0..\\lib\\dist\\bin\\codegraph.js" %%*\r\n' \
+  printf '@"%%~dp0..\\node.exe" --liftoff-only --disable-warning=ExperimentalWarning "%%~dp0..\\lib\\dist\\bin\\codegraph.js" %%*\r\n' \
     > "$STAGE/bin/codegraph.cmd"
 else
   cp "$NODE_BIN" "$STAGE/node"
@@ -104,7 +104,9 @@ DIR="$(cd "$(dirname "$SELF")/.." && pwd)"
 CODEGRAPH_HOST_PPID="${CODEGRAPH_HOST_PPID:-$PPID}"
 export CODEGRAPH_HOST_PPID
 # --liftoff-only: avoid the V8 turboshaft WASM Zone OOM (issues #293/#298).
-exec "$DIR/node" --liftoff-only "$DIR/lib/dist/bin/codegraph.js" "$@"
+# --disable-warning=ExperimentalWarning: mute node:sqlite's per-thread
+# "experimental feature" warning that otherwise interleaves with the progress UI.
+exec "$DIR/node" --liftoff-only --disable-warning=ExperimentalWarning "$DIR/lib/dist/bin/codegraph.js" "$@"
 LAUNCH
   chmod +x "$STAGE/bin/codegraph"
 fi

+ 5 - 1
scripts/npm-shim.js

@@ -98,8 +98,12 @@ function launcherIn(dir) {
 // --liftoff-only keeps tree-sitter's WASM grammars off V8's turboshaft tier to
 // avoid the Zone OOM on Node >= 22 (issues #293/#298). The unix bin/codegraph
 // launcher already passes it; on Windows we invoke node.exe directly so add it.
+// --disable-warning=ExperimentalWarning mutes node:sqlite's per-thread
+// "experimental feature" warning, which otherwise prints once per parse worker
+// mid-index, shredding the progress UI. The bundled node.exe is always new
+// enough for both flags.
 function liftoff(entry) {
-  return ['--liftoff-only', entry].concat(process.argv.slice(2));
+  return ['--liftoff-only', '--disable-warning=ExperimentalWarning', entry].concat(process.argv.slice(2));
 }
 
 // Download + cache the platform bundle from GitHub Releases. Returns

+ 1 - 1
src/extraction/index.ts

@@ -73,7 +73,7 @@ const WORKER_RECYCLE_INTERVAL = 250;
  * Progress callback for indexing operations
  */
 export interface IndexProgress {
-  phase: 'scanning' | 'parsing' | 'storing' | 'resolving';
+  phase: 'scanning' | 'parsing' | 'storing' | 'resolving' | 'linking';
   current: number;
   total: number;
   currentFile?: string;

+ 29 - 2
src/extraction/wasm-runtime-flags.ts

@@ -40,6 +40,31 @@ import { spawnSync } from 'child_process';
  */
 export const WASM_RUNTIME_FLAGS: readonly string[] = ['--liftoff-only'];
 
+/**
+ * Node CLI options (not V8 flags) passed alongside the WASM flags on every
+ * launch path. `--disable-warning=ExperimentalWarning` mutes node:sqlite's
+ * "SQLite is an experimental feature" warning, which is emitted once per
+ * THREAD — the main process plus every parse worker — so during indexing it
+ * prints repeatedly, interleaved with the progress UI. Node options apply
+ * process-wide (workers inherit them), so the command line covers everything.
+ *
+ * Deliberately NOT part of the {@link processHasWasmRuntimeFlags} re-exec
+ * gate: a launcher passing only the WASM flags (an older installed bundle
+ * running a newer dist) must not trigger a whole re-exec over a cosmetic
+ * warning.
+ */
+export function nodeRuntimeFlagsFor(nodeVersion: string): readonly string[] {
+  // `--disable-warning` landed in Node 20.11 / 21.3; older nodes treat it as
+  // a fatal "bad option" at spawn. Those runtimes can't run codegraph anyway
+  // (node:sqlite needs >= 22.5), but let them reach our own version messaging
+  // instead of a cryptic spawn failure.
+  const [major = 0, minor = 0] = nodeVersion.split('.').map(Number);
+  const supported = major > 21 || (major === 21 && minor >= 3) || (major === 20 && minor >= 11);
+  return supported ? ['--disable-warning=ExperimentalWarning'] : [];
+}
+
+export const NODE_RUNTIME_FLAGS: readonly string[] = nodeRuntimeFlagsFor(process.versions.node);
+
 /**
  * Env var set on the relaunched child so a detection slip can never cause an
  * infinite re-exec loop. Also lets users force-disable the relaunch.
@@ -76,8 +101,10 @@ export function buildRelaunchArgv(
   scriptArgs: readonly string[],
   execArgv: readonly string[] = process.execArgv
 ): string[] {
-  const preserved = execArgv.filter((arg) => !WASM_RUNTIME_FLAGS.includes(arg));
-  return [...WASM_RUNTIME_FLAGS, ...preserved, scriptPath, ...scriptArgs];
+  const preserved = execArgv.filter(
+    (arg) => !WASM_RUNTIME_FLAGS.includes(arg) && !NODE_RUNTIME_FLAGS.includes(arg)
+  );
+  return [...NODE_RUNTIME_FLAGS, ...WASM_RUNTIME_FLAGS, ...preserved, scriptPath, ...scriptArgs];
 }
 
 /**

+ 53 - 23
src/index.ts

@@ -509,13 +509,22 @@ export class CodeGraph {
             total: unresolvedCount,
           });
 
-          await this.resolveReferencesBatched((current, total) => {
-            options.onProgress?.({
-              phase: 'resolving',
-              current,
-              total,
-            });
-          });
+          await this.resolveReferencesBatched(
+            (current, total) => {
+              options.onProgress?.({
+                phase: 'resolving',
+                current,
+                total,
+              });
+            },
+            (done, totalPasses) => {
+              options.onProgress?.({
+                phase: 'linking',
+                current: done,
+                total: totalPasses,
+              });
+            }
+          );
 
           // Second pass: chained calls whose method lives on a supertype the
           // receiver conforms to (protocol-extension / inherited / default-
@@ -735,13 +744,22 @@ export class CodeGraph {
               total: unresolvedCount,
             });
 
-            await this.resolveReferencesBatched((current, total) => {
-              options.onProgress?.({
-                phase: 'resolving',
-                current,
-                total,
-              });
-            });
+            await this.resolveReferencesBatched(
+              (current, total) => {
+                options.onProgress?.({
+                  phase: 'resolving',
+                  current,
+                  total,
+                });
+              },
+              (done, totalPasses) => {
+                options.onProgress?.({
+                  phase: 'linking',
+                  current: done,
+                  total: totalPasses,
+                });
+              }
+            );
           }
         }
 
@@ -766,13 +784,22 @@ export class CodeGraph {
             total: orphanCount,
           });
 
-          await this.resolveReferencesBatched((current, total) => {
-            options.onProgress?.({
-              phase: 'resolving',
-              current,
-              total,
-            });
-          });
+          await this.resolveReferencesBatched(
+            (current, total) => {
+              options.onProgress?.({
+                phase: 'resolving',
+                current,
+                total,
+              });
+            },
+            (done, totalPasses) => {
+              options.onProgress?.({
+                phase: 'linking',
+                current: done,
+                total: totalPasses,
+              });
+            }
+          );
         }
 
         if (filesChanged || orphanCount > 0) {
@@ -1001,8 +1028,11 @@ export class CodeGraph {
    * Resolve references in batches to keep memory bounded on large codebases.
    * Processes chunks of unresolved refs, persisting results after each batch.
    */
-  async resolveReferencesBatched(onProgress?: (current: number, total: number) => void): Promise<ResolutionResult> {
-    return this.resolver.resolveAndPersistBatched(onProgress);
+  async resolveReferencesBatched(
+    onProgress?: (current: number, total: number) => void,
+    onSynthesisProgress?: (done: number, total: number) => void
+  ): Promise<ResolutionResult> {
+    return this.resolver.resolveAndPersistBatched(onProgress, undefined, onSynthesisProgress);
   }
 
   /**

+ 24 - 1
src/resolution/callback-synthesizer.ts

@@ -3446,7 +3446,20 @@ async function laravelEventEdges(ctx: ResolutionContext, onYield: MaybeYield): P
  * Sidekiq Worker.perform_async → #perform + Laravel event(new X) → listener handle).
  * Returns the count added. Never throws into indexing — callers wrap in try/catch.
  */
-export async function synthesizeCallbackEdges(queries: QueryBuilder, ctx: ResolutionContext): Promise<number> {
+
+/**
+ * Number of progress steps synthesizeCallbackEdges reports: one per `__mark()`
+ * call (every synthesis pass, plus the dedupe-merge and edge-insert steps).
+ * Cosmetic only — drift just makes the progress bar end early or jump — and a
+ * test pins it to the actual `__mark(` call count so adding a pass without
+ * bumping this fails loudly instead of silently skewing the bar.
+ */
+export const SYNTH_PROGRESS_STEPS = 40;
+export async function synthesizeCallbackEdges(
+  queries: QueryBuilder,
+  ctx: ResolutionContext,
+  onProgress?: (done: number, total: number) => void
+): Promise<number> {
   // Each sub-pass below is a whole-graph scan, and there are ~30 of them, all
   // running synchronously on the indexer's main thread. Their AGGREGATE can run
   // for well over a minute on a large repo — long enough for the #850 liveness
@@ -3456,6 +3469,14 @@ export async function synthesizeCallbackEdges(queries: QueryBuilder, ctx: Resolu
   // watchdog still catches that. See ./cooperative-yield.
   const yieldToLoop = createYielder();
 
+  // Synthesis runs AFTER the resolution progress bar reaches 100%, so without
+  // its own progress the UI freezes at "Resolving refs 100%" for the whole
+  // tail — long enough on big repos that users conclude the index hung and
+  // kill it. Report each completed pass; the caller surfaces it as its own
+  // progress phase. Emit 0/total up front so the phase flips immediately.
+  let passesDone = 0;
+  onProgress?.(0, SYNTH_PROGRESS_STEPS);
+
   // Per-pass wall-clock timing to stderr, opt-in via CODEGRAPH_SYNTH_TIMINGS
   // (=1: passes over 250ms; =all: every pass). This is the diagnostic that
   // located both the #1091/#1122 watchdog stalls and the #1212 OOM — keep it.
@@ -3467,6 +3488,8 @@ export async function synthesizeCallbackEdges(queries: QueryBuilder, ctx: Resolu
     if (process.env.CODEGRAPH_SYNTH_TIMINGS && (dt > 250 || process.env.CODEGRAPH_SYNTH_TIMINGS === 'all')) {
       console.error(`[synth-timing] ${label}: ${dt}ms`);
     }
+    passesDone++;
+    onProgress?.(Math.min(passesDone, SYNTH_PROGRESS_STEPS), SYNTH_PROGRESS_STEPS);
   };
 
   // Language gating: one indexed DISTINCT over the files table lets a pass

+ 7 - 2
src/resolution/index.ts

@@ -1258,7 +1258,8 @@ export class ReferenceResolver {
    */
   async resolveAndPersistBatched(
     onProgress?: (current: number, total: number) => void,
-    batchSize: number = 5000
+    batchSize: number = 5000,
+    onSynthesisProgress?: (done: number, total: number) => void
   ): Promise<ResolutionResult> {
     // Resolution runs on the indexer's MAIN thread, and the #850 liveness
     // watchdog SIGKILLs a process whose event loop stalls past its window (60s
@@ -1375,7 +1376,11 @@ export class ReferenceResolver {
     // callbacks) that static parsing leaves out. Best-effort — never fail the
     // index on it. See docs/design/callback-edge-synthesis.md.
     try {
-      aggregateStats.byMethod['callback-synthesis'] = await synthesizeCallbackEdges(this.queries, this.context);
+      aggregateStats.byMethod['callback-synthesis'] = await synthesizeCallbackEdges(
+        this.queries,
+        this.context,
+        onSynthesisProgress
+      );
     } catch {
       // synthesis is additive and optional; ignore failures
     }

+ 1 - 0
src/ui/shimmer-progress.ts

@@ -6,6 +6,7 @@ const PHASE_NAMES: Record<string, string> = {
   parsing: 'Parsing code',
   storing: 'Storing data',
   resolving: 'Resolving refs',
+  linking: 'Linking dynamic dispatch',
 };
 
 export interface IndexProgress {