feat(npm): restore programmatic/embedded SDK API (#354) (#603)

The 0.9.x thin-installer turned @colbymchenry/codegraph into a bin-only
shim: require("@colbymchenry/codegraph") threw MODULE_NOT_FOUND and no
types shipped, breaking embedded library consumers (e.g. Electron apps)
upgrading from 0.8.0.

Restore programmatic use without re-bloating the thin shim or duplicating
the ~49 MB of grammars the per-platform bundle already carries:

- main -> npm-sdk.js re-exports the installed per-platform bundle's compiled
  library (lib/dist/index.js) at runtime, reusing that bundle's own deps; it
  falls back to a self-healed cache bundle, else throws an actionable error.
- types -> ship the .d.ts tree only (~590 KB) in the main package, built from
  the same release so it can never skew from the runtime it re-exports.
- exports map resolves the `types` condition (nodenext) and the default entry.
- DatabaseConnection + QueryBuilder are now top-level exports, so embedded
  callers get the building blocks from the package entry instead of deep
  dist/ imports (which the shim no longer ships).

The CLI/MCP `bin` keeps execing the bundled Node; only library consumers run
on their own runtime, which must be Node 22.5+ for the built-in node:sqlite.

Validated end-to-end: built a real darwin-arm64 bundle, packed the npm
packages, installed them into a throwaway consumer, and confirmed require()
plus a full init/indexAll/searchNodes round-trip and the low-level
DatabaseConnection/QueryBuilder path all work on the host Node; types resolve
under both nodenext and classic node resolution; and the CLI shim still
launches. New hermetic tests cover npm-sdk resolution, cache fallback, and the
missing-bundle error.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

CHANGELOG.md

@@ -17,6 +17,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - Swift deferred-validation flows (and similar "handler array" patterns) now connect end-to-end in `codegraph_trace` and `codegraph_explore` — following a request's lifecycle reaches the validators registered with `.validate { … }` instead of dead-ending where the framework runs them by iterating a stored list of closures. Any pattern where closures are appended to a collection and later invoked by looping over it is now traced.
 - `codegraph_explore` now spells out the dynamic-dispatch relationships of the symbols you ask about — e.g. "the closures registered here are run by `didCompleteTask`" — so the indirect hops you'd otherwise grep to reconstruct are listed alongside the call flow.
 - `codegraph_explore` answers multi-phase questions that span a large "god file" far more completely. For a flow like "build, send, and validate a request" — where one big file holds the build chain and the validate logic lives in others — it now keeps every method *on the flow path* in full, collapses the file's off-path methods to one-line signatures, and guarantees each phase's defining file is shown (instead of truncating at a fixed size and dropping whichever phase came last, which sent you to read it by hand). Incidental files that merely name-drop the flow are still trimmed, so the response stays focused on the code that answers the question.
+- CodeGraph is usable as an embedded library again: `require("@colbymchenry/codegraph")` and `import` now resolve the programmatic API — the `CodeGraph` class plus building blocks like `DatabaseConnection`, `QueryBuilder`, `initGrammars`, and `FileLock` — so you can drive the graph directly from your own app (for example an Electron process) instead of only through the CLI or MCP server. Embedding runs on your own runtime, so it needs Node 22.5+ for the built-in SQLite. (#354)
 
 ### Fixes
 

README.md

@@ -500,8 +500,14 @@ When running as an MCP server, CodeGraph exposes these tools to Claude Code:
 
 ## Library Usage
 
+CodeGraph can be embedded directly. The npm package re-exports its programmatic
+API, so both `import` and `require` resolve the `CodeGraph` class in your own
+process — handy for embedding it in an app (e.g. an Electron main process).
+
 ```typescript
 import CodeGraph from '@colbymchenry/codegraph';
+// CommonJS works too:
+//   const { CodeGraph } = require('@colbymchenry/codegraph');
 
 const cg = await CodeGraph.init('/path/to/project');
 // Or: const cg = await CodeGraph.open('/path/to/project');
@@ -520,6 +526,21 @@ cg.unwatch(); // stop watching
 cg.close();
 ```
 
+Lower-level building blocks are exported from the same entry point for callers
+that drive the graph directly: `DatabaseConnection`, `QueryBuilder`,
+`getDatabasePath`, `initGrammars` / `loadGrammarsForLanguages`, and `FileLock`.
+
+**Embedding requirements**
+
+- Install from npm (`npm i @colbymchenry/codegraph`) so the matching
+  per-platform package — which carries the compiled library and its
+  dependencies — is fetched alongside the shim.
+- The API runs on **your** runtime, so it needs **Node 22.5+** for the built-in
+  `node:sqlite` (Electron qualifies when its bundled Node is 22.5+). The CLI and
+  MCP server are unaffected — they run on the self-contained bundled runtime.
+- TypeScript types ship with the package. As with any Node-targeting library,
+  keep `@types/node` available and `skipLibCheck: true` (the common default).
+
 ---
 
 ## Configuration

__tests__/npm-sdk.test.ts

@@ -0,0 +1,107 @@
+/**
+ * Programmatic/embedded SDK entry (`scripts/npm-sdk.js`) tests (issue #354).
+ *
+ * The published main package is a thin shim: the CLI `bin` (npm-shim.js) execs
+ * the bundled Node, while `main` (npm-sdk.js) lets embedded consumers
+ * `require("@colbymchenry/codegraph")` on their OWN Node by re-exporting the
+ * compiled library that ships inside the per-platform optionalDependency
+ * (@colbymchenry/codegraph-<target>/lib/dist/index.js).
+ *
+ * These tests stand up a temp main-package dir with a fake platform package as a
+ * resolvable sibling, then require the SDK in a child process — so resolution,
+ * the self-heal cache fallback, and the missing-bundle error are exercised
+ * hermetically with no real bundle, network, or registry.
+ */
+
+import { describe, it, expect } from 'vitest';
+import { spawnSync } from 'child_process';
+import * as fs from 'fs';
+import * as os from 'os';
+import * as path from 'path';
+
+const SDK_SRC = path.join(__dirname, '..', 'scripts', 'npm-sdk.js');
+const target = `${process.platform}-${process.arch}`;
+const VERSION = '9.9.9-test';
+
+function mkTmp(label: string): string {
+  return fs.mkdtempSync(path.join(os.tmpdir(), `cg-sdk-${label}-`));
+}
+
+// A temp node_modules with the main package (npm-sdk.js + package.json). The
+// fake platform package, when present, is written as a resolvable sibling so the
+// SDK's `require.resolve('@colbymchenry/codegraph-<target>/...')` walks to it.
+function makeConsumer(): { root: string; mainPkg: string } {
+  const root = mkTmp('consumer');
+  const mainPkg = path.join(root, 'node_modules', '@colbymchenry', 'codegraph');
+  fs.mkdirSync(mainPkg, { recursive: true });
+  fs.copyFileSync(SDK_SRC, path.join(mainPkg, 'npm-sdk.js'));
+  fs.writeFileSync(
+    path.join(mainPkg, 'package.json'),
+    JSON.stringify({ name: '@colbymchenry/codegraph', version: VERSION, main: 'npm-sdk.js' }) + '\n'
+  );
+  return { root, mainPkg };
+}
+
+// Write a fake compiled library that exports a sentinel, at the given lib/dist
+// root (used both for the platform package and the self-heal cache bundle).
+function writeFakeLib(libDistDir: string, sentinel: string): void {
+  fs.mkdirSync(libDistDir, { recursive: true });
+  fs.writeFileSync(
+    path.join(libDistDir, 'index.js'),
+    `module.exports = { SENTINEL: ${JSON.stringify(sentinel)}, CodeGraph: function CodeGraph() {} };\n`
+  );
+}
+
+function installPlatformPackage(root: string, sentinel: string): void {
+  const pkgRoot = path.join(root, 'node_modules', '@colbymchenry', `codegraph-${target}`);
+  writeFakeLib(path.join(pkgRoot, 'lib', 'dist'), sentinel);
+  fs.writeFileSync(
+    path.join(pkgRoot, 'package.json'),
+    JSON.stringify({ name: `@colbymchenry/codegraph-${target}`, version: VERSION }) + '\n'
+  );
+}
+
+// require() the SDK in a child process so each case gets a fresh module cache.
+function requireSdk(mainPkg: string, env: Record<string, string> = {}) {
+  const code =
+    `try { const m = require(${JSON.stringify(path.join(mainPkg, 'npm-sdk.js'))});` +
+    ` process.stdout.write(JSON.stringify({ sentinel: m.SENTINEL, cg: typeof m.CodeGraph })); }` +
+    ` catch (e) { process.stderr.write(String(e && e.message || e)); process.exit(7); }`;
+  const r = spawnSync(process.execPath, ['-e', code], {
+    encoding: 'utf8',
+    env: { ...process.env, ...env },
+  });
+  return { status: r.status, stdout: r.stdout, stderr: r.stderr };
+}
+
+describe('npm-sdk programmatic entry', () => {
+  it('re-exports the installed platform bundle library', () => {
+    const { root, mainPkg } = makeConsumer();
+    installPlatformPackage(root, 'platform-lib');
+    // Isolate from any real self-healed cache on this machine.
+    const r = requireSdk(mainPkg, { CODEGRAPH_INSTALL_DIR: path.join(root, '.empty-cache') });
+    expect(r.status).toBe(0);
+    expect(JSON.parse(r.stdout)).toEqual({ sentinel: 'platform-lib', cg: 'function' });
+  });
+
+  it('falls back to a self-healed cache bundle when the optional dep is absent', () => {
+    const { root, mainPkg } = makeConsumer(); // no platform package installed
+    const cacheDir = path.join(root, 'cache');
+    writeFakeLib(
+      path.join(cacheDir, 'bundles', `${target}-${VERSION}`, 'lib', 'dist'),
+      'cache-lib'
+    );
+    const r = requireSdk(mainPkg, { CODEGRAPH_INSTALL_DIR: cacheDir });
+    expect(r.status).toBe(0);
+    expect(JSON.parse(r.stdout)).toEqual({ sentinel: 'cache-lib', cg: 'function' });
+  });
+
+  it('throws an actionable error when no bundle is installed or cached', () => {
+    const { root, mainPkg } = makeConsumer(); // no platform package, empty cache
+    const r = requireSdk(mainPkg, { CODEGRAPH_INSTALL_DIR: path.join(root, '.empty-cache') });
+    expect(r.status).toBe(7);
+    expect(r.stderr).toContain(`@colbymchenry/codegraph-${target}`);
+    expect(r.stderr).toContain('not installed');
+    expect(r.stderr).toContain('registry.npmjs.org');
+  });
+});

scripts/npm-sdk.js

@@ -0,0 +1,75 @@
+'use strict';
+//
+// Programmatic / embedded SDK entry for @colbymchenry/codegraph (issue #354).
+//
+// The CLI/MCP `bin` (npm-shim.js) execs the per-platform bundle's OWN Node 24 so
+// the tool never depends on the user's runtime. Embedded library consumers are
+// the opposite case: they already run their own Node and just want the compiled
+// API — `require("@colbymchenry/codegraph")` returning the CodeGraph class et al.
+//
+// The compiled library + its production dependencies (web-tree-sitter,
+// tree-sitter-wasms, …) ship INSIDE the per-platform bundle, at
+//   @colbymchenry/codegraph-<platform>-<arch>/lib/dist/index.js
+// (with the deps in the sibling lib/node_modules). Re-exporting that bundle keeps
+// the main package thin — no second 50 MB copy of the grammars — while making the
+// SDK work in the consumer's process. Types are a separate concern: the main
+// package ships its own dist/**/*.d.ts tree (pointed at by `types`), built from
+// the same release so it can never skew from the runtime it re-exports.
+//
+// node:sqlite (Node >= 22.5) is required to OPEN a graph, but only lazily inside
+// the SQLite adapter — so loading this module is safe on older Node, and the
+// node:sqlite requirement surfaces with an actionable error only when a DB is
+// actually opened. Heavy extraction additionally wants the bundled launcher's
+// --liftoff-only flag (the WASM Zone-OOM guard, issues #293/#298); an embedded
+// host that drives large indexing should pass that flag to its own Node.
+
+var path = require('path');
+var os = require('os');
+var fs = require('fs');
+
+var target = process.platform + '-' + process.arch; // e.g. darwin-arm64, linux-x64
+var pkg = '@colbymchenry/codegraph-' + target;
+
+module.exports = require(resolveLibrary());
+
+// Locate the compiled library entry inside the installed per-platform bundle.
+// Throws an actionable error (rather than a bare MODULE_NOT_FOUND) when no bundle
+// is present, so an embedded consumer knows exactly what to install.
+function resolveLibrary() {
+  // 1) The npm-installed optional dependency — the normal case.
+  try {
+    return require.resolve(pkg + '/lib/dist/index.js');
+  } catch (e) {
+    /* fall through to the self-healed cache */
+  }
+
+  // 2) A bundle the CLI shim self-healed from GitHub Releases into the cache
+  //    (issue #303). Same node/lib/bin layout as the npm package. We only REUSE a
+  //    cached bundle here — unlike the CLI shim we never trigger a network
+  //    download from inside require(), which must stay synchronous and cheap.
+  var cached = cachedLibrary();
+  if (cached) return cached;
+
+  throw new Error(
+    'codegraph: the programmatic API is unavailable because the platform bundle\n' +
+    '(' + pkg + ') is not installed.\n' +
+    'The compiled library ships inside that per-platform optional dependency.\n' +
+    'Fixes:\n' +
+    '  - install from the official npm registry so the matching bundle is fetched:\n' +
+    '      npm i @colbymchenry/codegraph --registry=https://registry.npmjs.org\n' +
+    '  - or run the CLI once (e.g. `npx @colbymchenry/codegraph status`) to\n' +
+    '    self-heal the bundle into ~/.codegraph, then require() will find it.'
+  );
+}
+
+function cachedLibrary() {
+  try {
+    var version = require(path.join(__dirname, 'package.json')).version;
+    var base = process.env.CODEGRAPH_INSTALL_DIR || path.join(os.homedir(), '.codegraph');
+    var lib = path.join(base, 'bundles', target + '-' + version, 'lib', 'dist', 'index.js');
+    if (fs.existsSync(lib)) return lib;
+  } catch (e) {
+    /* no readable cache → caller reports the install guidance */
+  }
+  return null;
+}

scripts/pack-npm.sh

@@ -72,8 +72,26 @@ for archive in "${archives[@]}"; do
 done
 
 # Main shim package.
+#   npm-shim.js  CLI/MCP launcher (execs the bundled Node) — the `bin`.
+#   npm-sdk.js   programmatic/embedded entry (#354): re-exports the installed
+#                platform bundle's compiled library — the `main`.
+#   dist/        the .d.ts tree only (types). The runtime .js stays in the
+#                per-platform bundle so its deps aren't duplicated here.
 cp "$ROOT/scripts/npm-shim.js" "$NPM/main/npm-shim.js"
+cp "$ROOT/scripts/npm-sdk.js" "$NPM/main/npm-sdk.js"
 [ -f "$ROOT/README.md" ] && cp "$ROOT/README.md" "$NPM/main/README.md"
+
+# Ship the type declarations so `types`/`exports.types` resolve. Built from this
+# same release, so they can't skew from the runtime npm-sdk.js re-exports.
+[ -f "$ROOT/dist/index.d.ts" ] || ( echo "[pack-npm] building dist for .d.ts" >&2 && cd "$ROOT" && npm run build >/dev/null )
+ROOT="$ROOT" DEST="$NPM/main" node -e '
+  const fs=require("fs"), path=require("path");
+  const src=path.join(process.env.ROOT,"dist"), dest=path.join(process.env.DEST,"dist");
+  fs.cpSync(src, dest, { recursive:true, filter(s){
+    try { return fs.statSync(s).isDirectory() || s.endsWith(".d.ts"); } catch (e) { return false; }
+  }});
+'
+
 VERSION="$VERSION" SCOPE="$SCOPE" TARGETS="${targets[*]}" \
   node -e '
     const fs=require("fs");
@@ -85,8 +103,14 @@ VERSION="$VERSION" SCOPE="$SCOPE" TARGETS="${targets[*]}" \
       version: process.env.VERSION,
       description: "Local-first code intelligence for AI agents (MCP). Self-contained — bundles its own runtime.",
       bin: { codegraph: "npm-shim.js" },
+      main: "npm-sdk.js",
+      types: "dist/index.d.ts",
+      exports: {
+        ".": { types: "./dist/index.d.ts", default: "./npm-sdk.js" },
+        "./package.json": "./package.json"
+      },
       optionalDependencies: opt,
-      files: ["npm-shim.js","README.md"],
+      files: ["npm-shim.js","npm-sdk.js","dist","README.md"],
       license: "MIT"
     }, null, 2) + "\n");
   ' "$NPM/main/package.json"

src/index.ts

@@ -50,7 +50,12 @@ import { FileWatcher, WatchOptions, PendingFile, LockUnavailableError } from './
 
 // Re-export types for consumers
 export * from './types';
-export { getDatabasePath } from './db';
+// Storage building blocks for embedded/SDK consumers that drive the graph
+// directly (open a DB, run prepared queries) rather than through the CodeGraph
+// facade. Exposed from the package entry so they no longer require deep imports
+// into dist/ (issue #354).
+export { getDatabasePath, DatabaseConnection } from './db';
+export { QueryBuilder } from './db/queries';
 export {
   getCodeGraphDir,
   isInitialized,