diff --git a/AGENTS.md b/AGENTS.md index c1e5595fc..f17314009 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -47,6 +47,7 @@ Invoke them by name (e.g., `/office-hours`). | `/canary` | Post-deploy monitoring loop using the browse daemon. | | `/landing-report` | Read-only dashboard for the workspace-aware ship queue. | | `/document-release` | Update all docs to match what you just shipped. | +| `/document-generate` | Generate Diataxis docs (tutorial / how-to / reference / explanation) from code. | | `/setup-deploy` | One-time deploy config detection (Fly.io, Render, Vercel, etc.). | | `/gstack-upgrade` | Update gstack to the latest version. | diff --git a/CHANGELOG.md b/CHANGELOG.md index 55cc068f9..f27defa4b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,57 @@ # Changelog +## [1.37.0.0] - 2026-05-14 + +## **Split-engine gbrain: remote MCP for brain, local PGLite for code.** +## **Symbol-aware code search now coexists with cross-machine knowledge.** + +Path 4 (Remote MCP) setup gets a new opt-in at Step 4.5: a tiny local PGLite (~30s, ~120 MB) for `gbrain code-def`, `code-refs`, `code-callers` per worktree. The remote brain keeps holding artifacts, transcripts, and cross-machine queries. The two engines stay independent. Transcripts route to the artifacts repo on remote-MCP machines, the brain admin's pull job indexes them, and the local PGLite stays code-only with no transcript pollution. A new `gbrain_local_status` field on `gstack-gbrain-detect` distinguishes ok / no-cli / missing-config / broken-config / broken-db; `/sync-gbrain` and the sync orchestrator both gate on it so a dead Postgres URL gives a clear remediation message instead of two stages of ERR output. + +`/setup-gbrain` Step 1.5 (new) detects a broken local engine on re-run and offers four options: Retry the probe, Switch to PGLite (one-way, .bak rollback on failure), Switch brain mode (fall through to Step 2's path picker), or Quit. `/sync-gbrain` Step 1.5 (new) STOPs cleanly on broken-config / broken-db with a remediation message and SKIPs code+memory in `missing-config + remote-http` so the brain-sync push to the artifacts repo still runs. + +### The numbers that matter + +Source: `bun test test/gbrain-local-status.test.ts test/gbrain-detect-shape.test.ts test/gbrain-sync-skip.test.ts test/gbrain-init-rollback.test.ts test/gstack-upgrade-migration-v1_37_0_0.test.ts` — 5 new gate-tier test files, 27 cases, all green in ~5s. Periodic-tier E2E `test/skill-e2e-setup-gbrain-path4-local-pglite.test.ts` runs the full Path 4 + Step 4.5 Yes flow against a stub MCP and passes in 280s. + +| Surface | Before | After | +|---|---|---| +| Path 4 + `/sync-gbrain --full` output (Garry's broken-db state) | `ERR code source registration failed: gbrain not configured (run /setup-gbrain)` + `ERR memory gbrain import exited 1: Cannot connect to database` | `SKIP code skipped — local engine broken-db — config points at unreachable DB; see /setup-gbrain Step 1.5` + brain-sync runs normally | +| `bin/gstack-gbrain-detect` runtime | bash + jq, single-purpose probe | TypeScript shebang script sharing the `localEngineStatus()` classifier with the orchestrator. 10 JSON fields, 9 existing keys byte-compat; one new `gbrain_local_status` enum. Memoized resolvers cut ~400ms of duplicate fork-exec per skill preamble. | +| Status probe cost | `gbrain doctor --json` without `--fast` could hang up to 5s on dead DB | `gbrain doctor --json --fast` (3s ceiling) + DB-reachability via `gbrain sources list --json` stderr classification (~80ms steady), 60s TTL cache keyed on `{HOME, PATH, gbrain bin, gbrain version, config mtime}` | +| Path 4 user discovers code search | Hidden — only `/sync-gbrain` errors hint at it | `/gstack-upgrade` migration v1.37.0.0 prints a one-time notice when `gbrain_mcp_mode == remote-http` AND `gbrain_local_status == missing-config`. `gstack-config set local_code_index_offered true` to silence. | +| Transcripts indexed in remote brain | Local-only `gbrain import` writes to the LOCAL engine, polluting PGLite if user opts into Step 4.5 | `gstack-memory-ingest` detects remote-http MCP, persists staged markdown to `~/.gstack/transcripts/run--/` instead of tmpdir, skips local `gbrain import`. `bin/gstack-brain-sync` allowlist now covers `transcripts/run-*/*.md`; brain admin pulls and indexes. | + +### Itemized changes + +#### Added + + +- `lib/gbrain-local-status.ts` — shared 5-state engine status classifier (`ok` / `no-cli` / `missing-config` / `broken-config` / `broken-db`) with 60s TTL cache and `--no-cache` flag. Probes via `gbrain sources list --json` + stderr classification reusing the exact patterns from `lib/gbrain-sources.ts:66-67`. +- `/setup-gbrain` Step 1.5 — broken-db remediation with 4 options (Retry / Switch to PGLite / Switch brain mode / Quit). PGLite switch is rollback-safe: `mv ~/.gbrain/config.json` to a timestamped `.bak`, `gbrain init --pglite`, on non-zero exit restore the .bak verbatim. +- `/setup-gbrain` Step 4.5 — Path 4 opt-in for local PGLite code search. Yes path runs `gstack-gbrain-install` (idempotent) + `gbrain init --pglite --json` with the same rollback semantics. No path keeps Path 4 as remote-MCP-only. +- `/sync-gbrain` Step 1.5 — pre-flight local engine status check. STOPs on broken-config / broken-db with remediation, SKIPs code+memory in `missing-config + remote-http` so brain-sync still runs. +- `gstack-upgrade/migrations/v1.37.0.0.sh` — one-time discoverability notice for existing Path 4 users whose machine has no local engine yet. +- `bin/gstack-brain-sync` allowlist — `transcripts/run-*/*.md` so remote-MCP transcripts persisted to `~/.gstack/transcripts/` reach the artifacts repo. +- New test files (gate-tier, all mocked, no real gbrain): `gbrain-local-status.test.ts` (11 cases), `gbrain-detect-shape.test.ts` (8 cases), `gbrain-sync-skip.test.ts` (5 cases), `gbrain-init-rollback.test.ts` (3 cases), `gstack-upgrade-migration-v1_37_0_0.test.ts` (5 cases). +- Periodic-tier E2E `skill-e2e-setup-gbrain-path4-local-pglite.test.ts` for the full Path 4 + Step 4.5 Yes flow. + +#### Changed + +- `bin/gstack-gbrain-detect` — rewritten bash → TypeScript shebang script. Filename unchanged so existing skill preamble callers shell out without edits. 9 existing JSON fields preserve name + type + semantics; new `gbrain_local_status` field added. Documented dependency: requires `bun` on PATH (the gstack installer already provides this). +- `bin/gstack-gbrain-sync.ts` — `runCodeImport()` + `runMemoryIngest()` return `{ran: false, summary: "skipped — local engine ; remote MCP unaffected"}` when `localEngineStatus() != 'ok'`. Brain-sync stage continues regardless. +- `bin/gstack-memory-ingest.ts` — when `gbrain_mcp_mode === 'remote-http'`, persists staged transcripts to `~/.gstack/transcripts/run--/` and skips local `gbrain import` entirely. +- `bin/gstack-artifacts-init` — extends the managed `.brain-allowlist` to include `transcripts/run-*/*.md` and `transcripts/run-*/**/*.md` (privacy class: behavioral). +- `sync-gbrain/SKILL.md.tmpl` Step 1 — corrects misleading prose about memory stage "routing through MCP." Memory stage always shells out to local `gbrain import`; in remote-http mode it persists markdown instead. + +#### Fixed + +- Pre-existing flake in `test/gstack-next-version.test.ts` — bumped per-test timeout from default 5s to 15s. Spawned `gstack-next-version` CLI takes 4-5s wall time on M-series Macs under suite load and tipped over 5001ms intermittently. + +#### For contributors + +- New shared classifier pattern: `lib/gbrain-local-status.ts` exports `localEngineStatus()`, `resolveGbrainBin()`, `readGbrainVersion()`. The latter two are memoized per-process keyed on PATH so detect + classifier share fork-exec results. +- 13 architectural decisions captured in plan file `~/.claude/plans/the-real-product-fix-squishy-galaxy.md` — including Codex outside-voice findings (4 became structural decisions: keep proactive setup question, route transcripts via artifacts repo, SKIP+brain-sync on broken engine, retry-first repair menu). + ## [1.35.0.0] - 2026-05-13 ## **Docs become a tracked surface, not an afterthought. `/document-generate` writes them from scratch, `/document-release` audits coverage in four Diataxis quadrants.** @@ -33,6 +85,7 @@ To use: run `/document-release` after `/ship` (or let `/ship` auto-invoke it), s ### Itemized changes #### Added + - **`/document-generate` skill** (`document-generate/SKILL.md.tmpl`, 446 lines): Diataxis-based documentation generator with 9-step workflow — scope, codebase archaeology, partition, reference, explanation, how-to, tutorial, cross-linking, quality self-review. Reads the full codebase before writing a single line of docs. - **`/document-release` Step 1.5 — Coverage Map**: scans diff for new public surface (skills, CLI flags, config options, API endpoints), classifies each entity by Diataxis quadrant coverage, flags zero-coverage items as critical gaps and reference-only as common gaps. Output feeds the PR body. - **`/document-release` Architecture diagram drift detection**: extracts entity names from ASCII/Mermaid blocks in ARCHITECTURE.md, cross-references against the diff, flags renamed/removed entities. diff --git a/VERSION b/VERSION index c25c8ba4b..f24ab8298 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.35.0.0 +1.37.0.0 diff --git a/bin/gstack-artifacts-init b/bin/gstack-artifacts-init index 8f97c3305..c182077fe 100755 --- a/bin/gstack-artifacts-init +++ b/bin/gstack-artifacts-init @@ -232,6 +232,11 @@ retros/*.md developer-profile.json builder-journey.md builder-profile.jsonl +# Transcripts staged in remote-http MCP mode (per plan D11 split-engine). +# gstack-memory-ingest persists per-run dirs here when local gbrain import +# is skipped; brain admin pulls + indexes into the remote brain. +transcripts/run-*/*.md +transcripts/run-*/**/*.md # NOT synced (machine-local UX state): # projects/*/question-preferences.json (per-machine UX preferences) # projects/*/question-log.jsonl (audit/derivation log stays with preferences) @@ -251,7 +256,9 @@ cat > "$GSTACK_HOME/.brain-privacy-map.json" <<'EOF' {"pattern": "builder-journey.md", "class": "artifact"}, {"pattern": "projects/*/timeline.jsonl", "class": "behavioral"}, {"pattern": "developer-profile.json", "class": "behavioral"}, - {"pattern": "builder-profile.jsonl", "class": "behavioral"} + {"pattern": "builder-profile.jsonl", "class": "behavioral"}, + {"pattern": "transcripts/run-*/*.md", "class": "behavioral"}, + {"pattern": "transcripts/run-*/**/*.md", "class": "behavioral"} ] EOF diff --git a/bin/gstack-gbrain-detect b/bin/gstack-gbrain-detect index 98775bfdf..66503905e 100755 --- a/bin/gstack-gbrain-detect +++ b/bin/gstack-gbrain-detect @@ -1,188 +1,223 @@ -#!/usr/bin/env bash -# gstack-gbrain-detect — emit current gbrain/gstack-brain state as JSON. -# -# Usage: -# gstack-gbrain-detect -# -# Output (always valid JSON, even when every check is false): -# { -# "gbrain_on_path": true|false, -# "gbrain_version": "0.18.2" | null, -# "gbrain_config_exists": true|false, -# "gbrain_engine": "pglite"|"postgres" | null, -# "gbrain_doctor_ok": true|false, -# "gbrain_mcp_mode": "local-stdio"|"remote-http"|"none", -# "gstack_brain_sync_mode": "off"|"artifacts-only"|"full", -# "gstack_brain_git": true|false, -# "gstack_artifacts_remote": "https://..." | "" -# } -# -# The /setup-gbrain skill reads this once at startup to decide which path -# branches are live and which steps can be skipped. Never modifies state; -# pure introspection. Exits 0 unless `jq` is missing. -# -# Env: -# GSTACK_HOME — override ~/.gstack for gstack-brain-* state lookups. -set -euo pipefail +#!/usr/bin/env -S bun run +/** + * gstack-gbrain-detect — emit current gbrain/gstack-brain state as JSON. + * + * Rewritten from bash to TypeScript in v{X.Y.Z.0} to share the engine-status + * classifier with bin/gstack-gbrain-sync.ts. Single source of truth via + * lib/gbrain-local-status.ts. Filename and exec semantics unchanged: callers + * just shell out to the file path; the bun shebang resolves at runtime. + * + * Output (always valid JSON, even when every check is false): + * { + * "gbrain_on_path": true|false, + * "gbrain_version": "0.18.2" | null, + * "gbrain_config_exists": true|false, + * "gbrain_engine": "pglite"|"postgres" | null, + * "gbrain_doctor_ok": true|false, + * "gbrain_mcp_mode": "local-stdio"|"remote-http"|"none", + * "gstack_brain_sync_mode": "off"|"artifacts-only"|"full", + * "gstack_brain_git": true|false, + * "gstack_artifacts_remote": "https://..." | "", + * "gbrain_local_status": "ok"|"no-cli"|"missing-config"|"broken-config"|"broken-db" + * } + * + * Backward compatibility (per plan codex #5): the 9 pre-existing fields stay + * identical in name + type + value semantics. One new field added: + * gbrain_local_status. Key order may differ from the bash version's `jq -n` + * output — downstream parsers must not depend on key order (none currently do). + * + * Env: + * GSTACK_HOME — override ~/.gstack for state lookups (used by tests). + * HOME — effective user home (drives ~/.gbrain/config.json path). + * GSTACK_DETECT_NO_CACHE=1 — bypass the 60s local-status cache. + */ -STATE_DIR="${GSTACK_HOME:-$HOME/.gstack}" -SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" -CONFIG_BIN="$SCRIPT_DIR/gstack-config" -GBRAIN_CONFIG="$HOME/.gbrain/config.json" +import { execFileSync } from "child_process"; +import { existsSync, readFileSync } from "fs"; +import { homedir } from "os"; +import { join } from "path"; -die() { echo "gstack-gbrain-detect: $*" >&2; exit 2; } +import { + localEngineStatus, + resolveGbrainBin, + readGbrainVersion, +} from "../lib/gbrain-local-status"; -require_jq() { - command -v jq >/dev/null 2>&1 || die "jq is required. Install with: brew install jq" +const STATE_DIR = process.env.GSTACK_HOME || join(userHome(), ".gstack"); +const SCRIPT_DIR = __dirname; +const CONFIG_BIN = join(SCRIPT_DIR, "gstack-config"); +const GBRAIN_CONFIG = join(userHome(), ".gbrain", "config.json"); +const CLAUDE_JSON = join(userHome(), ".claude.json"); + +function userHome(): string { + return process.env.HOME || homedir(); } -require_jq -# --- gbrain binary presence + version --- -gbrain_on_path=false -gbrain_version=null -if command -v gbrain >/dev/null 2>&1; then - gbrain_on_path=true - # Format versions as JSON strings; gbrain --version may print other chatter. - v=$(gbrain --version 2>/dev/null | head -1 | tr -d '[:space:]' || true) - if [ -n "$v" ]; then - gbrain_version=$(jq -Rn --arg v "$v" '$v') - fi -fi +function tryExec(cmd: string, args: string[], timeoutMs = 5_000): string | null { + try { + return execFileSync(cmd, args, { + encoding: "utf-8", + timeout: timeoutMs, + stdio: ["ignore", "pipe", "ignore"], + }).trim(); + } catch { + return null; + } +} -# --- gbrain config file --- -gbrain_config_exists=false -gbrain_engine=null -if [ -f "$GBRAIN_CONFIG" ]; then - gbrain_config_exists=true - # Engine is defensively parsed; an invalid config returns null, not a crash. - engine_raw=$(jq -r '.engine // empty' "$GBRAIN_CONFIG" 2>/dev/null || true) - case "$engine_raw" in - pglite|postgres) gbrain_engine=$(jq -Rn --arg e "$engine_raw" '$e') ;; - esac -fi +function tryReadJSON(path: string): unknown | null { + if (!existsSync(path)) return null; + try { + return JSON.parse(readFileSync(path, "utf-8")); + } catch { + return null; + } +} -# --- gbrain doctor health --- -# Doctor is wrapped in `timeout 5s` to match the /health D6 pattern and avoid -# the detect step hanging the skill when gbrain is broken or its DB is -# unreachable. Any nonzero exit or non-"ok"/"warnings" status → false. -gbrain_doctor_ok=false -if [ "$gbrain_on_path" = "true" ]; then - # Use `timeout` if available; some minimal macs use gtimeout from coreutils. - timeout_bin="" - if command -v timeout >/dev/null 2>&1; then timeout_bin="timeout 5s" - elif command -v gtimeout >/dev/null 2>&1; then timeout_bin="gtimeout 5s" - fi - if doctor_json=$(eval "$timeout_bin gbrain doctor --json" 2>/dev/null); then - status=$(echo "$doctor_json" | jq -r '.status // empty' 2>/dev/null || true) - case "$status" in - ok|warnings) gbrain_doctor_ok=true ;; - esac - fi -fi +// --- gbrain binary presence + version --- +// Uses the shared memoized resolvers from lib/gbrain-local-status.ts so +// detect and the classifier share probe results within one process. +function detectGbrain(): { onPath: boolean; version: string | null } { + const bin = resolveGbrainBin(); + if (!bin) return { onPath: false, version: null }; + const verRaw = readGbrainVersion(); + if (!verRaw) return { onPath: true, version: null }; + // Match bash behavior: head -1 | tr -d '[:space:]' + const version = verRaw.split("\n")[0].replace(/\s+/g, "") || null; + return { onPath: true, version }; +} -# --- artifacts sync state (renamed from gbrain_sync_mode in v1.27.0.0) --- -gstack_brain_sync_mode="off" -if [ -x "$CONFIG_BIN" ]; then - mode=$("$CONFIG_BIN" get artifacts_sync_mode 2>/dev/null || true) - case "$mode" in - off|artifacts-only|full) gstack_brain_sync_mode="$mode" ;; - esac -fi +// --- gbrain config existence + engine kind --- +function detectConfig(): { exists: boolean; engine: "pglite" | "postgres" | null } { + if (!existsSync(GBRAIN_CONFIG)) return { exists: false, engine: null }; + const parsed = tryReadJSON(GBRAIN_CONFIG) as { engine?: string } | null; + if (!parsed) return { exists: true, engine: null }; + if (parsed.engine === "pglite" || parsed.engine === "postgres") { + return { exists: true, engine: parsed.engine }; + } + return { exists: true, engine: null }; +} -gstack_brain_git=false -if [ -d "$STATE_DIR/.git" ]; then - gstack_brain_git=true -fi +// --- gbrain doctor health (any nonzero exit or non-"ok"/"warnings" status → false) --- +// +// Uses --fast to avoid hanging on a dead DB. Per the local-status classifier +// (which probes DB directly via `gbrain sources list`), gbrain_doctor_ok is a +// coarse health summary, not engine-reachability — that's gbrain_local_status. +function detectDoctor(onPath: boolean): boolean { + if (!onPath) return false; + const out = tryExec("gbrain", ["doctor", "--json", "--fast"], 3_000); + if (!out) return false; + try { + const parsed = JSON.parse(out) as { status?: string }; + return parsed.status === "ok" || parsed.status === "warnings"; + } catch { + return false; + } +} -# --- gbrain_mcp_mode: local-stdio | remote-http | none --- -# Defense-in-depth fallback chain (intentional ordering, do not reorder): -# 1. `claude mcp get gbrain --json` — public CLI surface, structured output -# 2. `claude mcp list` text-grep — older claude versions without --json -# 3. `~/.claude.json` jq read — last resort if `claude` isn't on PATH -# Fallback chain logged because if Anthropic moves the file or renames keys, -# the third tier breaks silently; the first two tiers should catch it. -gbrain_mcp_mode="none" -if command -v claude >/dev/null 2>&1; then - # Tier 1: claude mcp get --json - if mcp_get_json=$(claude mcp get gbrain --json 2>/dev/null); then - if echo "$mcp_get_json" | jq -e '.' >/dev/null 2>&1; then - mtype=$(echo "$mcp_get_json" | jq -r '.type // .transport // empty' 2>/dev/null) - mcommand=$(echo "$mcp_get_json" | jq -r '.command // empty' 2>/dev/null) - murl=$(echo "$mcp_get_json" | jq -r '.url // empty' 2>/dev/null) - case "$mtype" in - http|sse) gbrain_mcp_mode="remote-http" ;; - stdio) gbrain_mcp_mode="local-stdio" ;; - *) - # Newer claude versions may emit just url + command; infer. - if [ -n "$murl" ]; then gbrain_mcp_mode="remote-http" - elif [ -n "$mcommand" ]; then gbrain_mcp_mode="local-stdio" - fi - ;; - esac - fi - fi - # Tier 2: claude mcp list text-grep (only if Tier 1 didn't resolve) - if [ "$gbrain_mcp_mode" = "none" ]; then - if mcp_list=$(claude mcp list 2>/dev/null); then - gbrain_line=$(echo "$mcp_list" | grep -E '^gbrain:' || true) - if [ -n "$gbrain_line" ]; then - if echo "$gbrain_line" | grep -q 'http\|HTTP'; then - gbrain_mcp_mode="remote-http" - else - gbrain_mcp_mode="local-stdio" - fi - fi - fi - fi -fi -# Tier 3: ~/.claude.json jq read (only if claude binary or earlier tiers failed) -if [ "$gbrain_mcp_mode" = "none" ]; then - if [ -f "$HOME/.claude.json" ]; then - # Look for a gbrain MCP server entry. Type field disambiguates http vs stdio. - mtype=$(jq -r '.mcpServers.gbrain.type // .mcpServers.gbrain.transport // empty' "$HOME/.claude.json" 2>/dev/null) - murl=$(jq -r '.mcpServers.gbrain.url // empty' "$HOME/.claude.json" 2>/dev/null) - mcommand=$(jq -r '.mcpServers.gbrain.command // empty' "$HOME/.claude.json" 2>/dev/null) - case "$mtype" in - url|http|sse) gbrain_mcp_mode="remote-http" ;; - stdio) gbrain_mcp_mode="local-stdio" ;; - *) - if [ -n "$murl" ]; then gbrain_mcp_mode="remote-http" - elif [ -n "$mcommand" ]; then gbrain_mcp_mode="local-stdio" - fi - ;; - esac - fi -fi +// --- artifacts sync mode --- +function detectSyncMode(): "off" | "artifacts-only" | "full" { + if (!existsSync(CONFIG_BIN)) return "off"; + const out = tryExec(CONFIG_BIN, ["get", "artifacts_sync_mode"], 2_000); + if (out === "off" || out === "artifacts-only" || out === "full") return out; + return "off"; +} -# --- artifacts remote URL (post-rename) with brain-* fallback during the -# migration window (gstack-upgrade migration runs the rename). --- -gstack_artifacts_remote="" -if [ -f "$HOME/.gstack-artifacts-remote.txt" ]; then - gstack_artifacts_remote=$(head -1 "$HOME/.gstack-artifacts-remote.txt" 2>/dev/null | tr -d '[:space:]' || true) -elif [ -f "$HOME/.gstack-brain-remote.txt" ]; then - # Pre-migration fallback. Migration v1.27.0.0 will mv this to the new path. - gstack_artifacts_remote=$(head -1 "$HOME/.gstack-brain-remote.txt" 2>/dev/null | tr -d '[:space:]' || true) -fi +// --- gstack-brain git repo present? --- +function detectBrainGit(): boolean { + return existsSync(join(STATE_DIR, ".git")); +} -# Emit single-object JSON. -jq -n \ - --argjson on_path "$gbrain_on_path" \ - --argjson version "$gbrain_version" \ - --argjson config_exists "$gbrain_config_exists" \ - --argjson engine "$gbrain_engine" \ - --argjson doctor_ok "$gbrain_doctor_ok" \ - --arg mcp_mode "$gbrain_mcp_mode" \ - --arg sync_mode "$gstack_brain_sync_mode" \ - --argjson brain_git "$gstack_brain_git" \ - --arg artifacts_remote "$gstack_artifacts_remote" \ - '{ - gbrain_on_path: $on_path, - gbrain_version: $version, - gbrain_config_exists: $config_exists, - gbrain_engine: $engine, - gbrain_doctor_ok: $doctor_ok, - gbrain_mcp_mode: $mcp_mode, - gstack_brain_sync_mode: $sync_mode, - gstack_brain_git: $brain_git, - gstack_artifacts_remote: $artifacts_remote - }' +// --- MCP mode: local-stdio | remote-http | none --- +// +// Defense-in-depth fallback chain (same ordering as the bash version): +// 1. `claude mcp get gbrain --json` — public CLI surface, structured output +// 2. `claude mcp list` text-grep — older claude versions without --json +// 3. `~/.claude.json` jq read — last resort if `claude` isn't on PATH +function detectMcpMode(): "local-stdio" | "remote-http" | "none" { + const claudeOnPath = tryExec("sh", ["-c", "command -v claude"], 1_000) !== null; + if (claudeOnPath) { + // Tier 1: `claude mcp get gbrain --json` + const get = tryExec("claude", ["mcp", "get", "gbrain", "--json"], 3_000); + if (get) { + try { + const parsed = JSON.parse(get) as { + type?: string; + transport?: string; + command?: string; + url?: string; + }; + const mtype = parsed.type || parsed.transport || ""; + if (mtype === "http" || mtype === "sse") return "remote-http"; + if (mtype === "stdio") return "local-stdio"; + if (parsed.url) return "remote-http"; + if (parsed.command) return "local-stdio"; + } catch { + // fall through + } + } + // Tier 2: `claude mcp list` text-grep + const list = tryExec("claude", ["mcp", "list"], 3_000); + if (list) { + const line = list.split("\n").find((l) => /^gbrain:/.test(l)); + if (line) { + if (/\b(http|HTTP)\b/.test(line)) return "remote-http"; + return "local-stdio"; + } + } + } + // Tier 3: read ~/.claude.json directly + const cj = tryReadJSON(CLAUDE_JSON) as + | { mcpServers?: { gbrain?: { type?: string; transport?: string; command?: string; url?: string } } } + | null; + const entry = cj?.mcpServers?.gbrain; + if (entry) { + const mtype = entry.type || entry.transport || ""; + if (mtype === "url" || mtype === "http" || mtype === "sse") return "remote-http"; + if (mtype === "stdio") return "local-stdio"; + if (entry.url) return "remote-http"; + if (entry.command) return "local-stdio"; + } + return "none"; +} + +// --- artifacts remote URL with brain-* fallback during the rename migration window --- +function detectArtifactsRemote(): string { + const newPath = join(userHome(), ".gstack-artifacts-remote.txt"); + const oldPath = join(userHome(), ".gstack-brain-remote.txt"); + for (const p of [newPath, oldPath]) { + if (existsSync(p)) { + try { + return readFileSync(p, "utf-8").split("\n")[0].trim(); + } catch { + // fall through + } + } + } + return ""; +} + +function main(): void { + const gbrain = detectGbrain(); + const config = detectConfig(); + const noCache = process.env.GSTACK_DETECT_NO_CACHE === "1"; + + // Order MATCHES the bash version's jq output for callers that visually grep + // (key order doesn't affect JSON parsers, but minimizes review noise). + const out = { + gbrain_on_path: gbrain.onPath, + gbrain_version: gbrain.version, + gbrain_config_exists: config.exists, + gbrain_engine: config.engine, + gbrain_doctor_ok: detectDoctor(gbrain.onPath), + gbrain_mcp_mode: detectMcpMode(), + gstack_brain_sync_mode: detectSyncMode(), + gstack_brain_git: detectBrainGit(), + gstack_artifacts_remote: detectArtifactsRemote(), + gbrain_local_status: localEngineStatus({ noCache }), + }; + + process.stdout.write(JSON.stringify(out, null, 2) + "\n"); +} + +main(); diff --git a/bin/gstack-gbrain-sync.ts b/bin/gstack-gbrain-sync.ts index 36b265e42..732ee430c 100644 --- a/bin/gstack-gbrain-sync.ts +++ b/bin/gstack-gbrain-sync.ts @@ -37,6 +37,7 @@ import { createHash } from "crypto"; import { detectEngineTier, withErrorContext, canonicalizeRemote } from "../lib/gstack-memory-helpers"; import { ensureSourceRegistered, sourcePageCount } from "../lib/gbrain-sources"; +import { localEngineStatus, type LocalEngineStatus } from "../lib/gbrain-local-status"; // ── Types ────────────────────────────────────────────────────────────────── @@ -290,6 +291,42 @@ function releaseLock(): void { // ── Stage runners ────────────────────────────────────────────────────────── +/** + * Build a SKIP result for the code/memory stage when the local engine is + * not in 'ok' state (per plan D12). Surface the status verbatim so the + * verdict block tells the user exactly what's wrong without re-probing. + * + * Reasons mapped to user-actionable summaries: + * no-cli → "gbrain CLI not on PATH; install via /setup-gbrain" + * missing-config → "no local engine; run /setup-gbrain to add local PGLite" + * broken-config → "config file at ~/.gbrain/config.json is malformed; see /setup-gbrain Step 1.5" + * broken-db → "config points at unreachable DB; see /setup-gbrain Step 1.5" + */ +function skipStageForLocalStatus( + stage: "code" | "memory", + status: LocalEngineStatus, + t0: number, +): StageResult { + const reasons: Record, string> = { + "no-cli": "gbrain CLI not on PATH; install via /setup-gbrain", + "missing-config": + "no local engine; run /setup-gbrain to add local PGLite for code search", + "broken-config": + "config at ~/.gbrain/config.json is malformed; see /setup-gbrain Step 1.5", + "broken-db": + "config points at unreachable DB; see /setup-gbrain Step 1.5", + }; + const reason = reasons[status as Exclude]; + return { + name: stage, + ran: false, + ok: true, // SKIP (per D12) — not a stage failure, just an unsatisfied prerequisite + duration_ms: Date.now() - t0, + summary: `skipped — local engine ${status} — ${reason}`, + }; +} + + async function runCodeImport(args: CliArgs): Promise { const t0 = Date.now(); const root = repoRoot(); @@ -302,6 +339,9 @@ async function runCodeImport(args: CliArgs): Promise { const sourceId = deriveCodeSourceId(root); + // dry-run preview always shows the would-do steps, regardless of local + // engine state. Useful for "what would /sync-gbrain do" without probing + // the engine. if (args.mode === "dry-run") { return { name: "code", @@ -313,6 +353,17 @@ async function runCodeImport(args: CliArgs): Promise { }; } + // Split-engine pre-flight (per plan D12): when local engine is not ok, SKIP + // code stage cleanly. Brain-sync stage still runs because it doesn't depend + // on local engine. The /sync-gbrain Step 1.5 pre-flight surfaces the user + // remediation message; this skip just keeps the orchestrator from crashing + // when the local DB is dead. Skipped on --dry-run (above) since dry-run + // never actually probes anything. + const localStatus = localEngineStatus({ noCache: false }); + if (localStatus !== "ok") { + return skipStageForLocalStatus("code", localStatus, t0); + } + // Step 0: Best-effort cleanup of pre-pathhash legacy source. // Earlier /sync-gbrain versions registered `gstack-code-` (no path // suffix). On a multi-worktree repo, those collapsed onto a single id @@ -431,6 +482,15 @@ function runMemoryIngest(args: CliArgs): StageResult { return { name: "memory", ran: false, ok: true, duration_ms: 0, summary: "would: gstack-memory-ingest --probe" }; } + // Split-engine pre-flight (per plan D12). gstack-memory-ingest shells out + // to `gbrain import` which targets the LOCAL engine. When that engine is + // not ok, SKIP cleanly so brain-sync (the only stage that doesn't depend + // on local engine) still runs. + const localStatus = localEngineStatus({ noCache: false }); + if (localStatus !== "ok") { + return skipStageForLocalStatus("memory", localStatus, t0); + } + const ingestPath = join(import.meta.dir, "gstack-memory-ingest.ts"); const ingestArgs = ["run", ingestPath]; if (args.mode === "full") ingestArgs.push("--bulk"); diff --git a/bin/gstack-memory-ingest.ts b/bin/gstack-memory-ingest.ts index c6227341d..b1169ae69 100644 --- a/bin/gstack-memory-ingest.ts +++ b/bin/gstack-memory-ingest.ts @@ -1202,6 +1202,57 @@ function makeStagingDir(): string { return dir; } +/** + * Persistent staging dir used in remote-http MCP mode (split-engine D11). + * + * Instead of staging to ~/.gstack/.staging-ingest--/ and cleaning up + * after `gbrain import`, remote-http users get a stable path that survives. + * gstack-brain-sync's allowlist pushes ~/.gstack/transcripts/** to the + * artifacts repo; the brain admin's pull job indexes them into the remote + * brain. Local PGLite (if present) stays code-only. + * + * Path: ~/.gstack/transcripts// (run-id pid+ts so concurrent passes + * stay separate; brain-sync push doesn't care about subdir naming). + */ +function makePersistentTranscriptDir(): string { + const dir = join( + GSTACK_HOME, + "transcripts", + `run-${process.pid}-${Date.now()}`, + ); + mkdirSync(dir, { recursive: true }); + return dir; +} + +/** + * Detect whether the gbrain MCP is remote-http (Path 4) — and therefore we + * should NOT call `gbrain import` because we don't want the local PGLite + * polluted with transcripts (per plan D11). + * + * Reads ~/.claude.json directly (same fallback chain as gstack-gbrain-detect + * Tier 3). Cheap: one fs read, no fork-exec. + */ +function isRemoteHttpMcpMode(): boolean { + const home = process.env.HOME || homedir(); + const claudeJsonPath = join(home, ".claude.json"); + if (!existsSync(claudeJsonPath)) return false; + try { + const parsed = JSON.parse(readFileSync(claudeJsonPath, "utf-8")) as { + mcpServers?: { + gbrain?: { type?: string; transport?: string; url?: string }; + }; + }; + const entry = parsed.mcpServers?.gbrain; + if (!entry) return false; + const mtype = entry.type || entry.transport || ""; + if (mtype === "url" || mtype === "http" || mtype === "sse") return true; + if (entry.url) return true; + return false; + } catch { + return false; + } +} + /** * Best-effort recursive cleanup. Failures swallowed — at worst we leak a * staging dir to disk; the next run uses a new one and they age out via @@ -1387,12 +1438,24 @@ async function ingestPass(args: CliArgs): Promise { }; } - // Phase 2: stage to a per-run dir + invoke gbrain import. - const stagingDir = makeStagingDir(); + // Phase 2: stage + (optionally) invoke gbrain import. + // + // Split-engine branch per plan D11: in remote-http MCP mode, we stage to a + // PERSISTENT dir under ~/.gstack/transcripts/ and SKIP `gbrain import` + // entirely. gstack-brain-sync push will pick the dir up via its allowlist + // and the brain admin's pull job will index transcripts into the remote + // brain. Local PGLite (if any) stays code-only. + const remoteHttpMode = isRemoteHttpMcpMode(); + const stagingDir = remoteHttpMode + ? makePersistentTranscriptDir() + : makeStagingDir(); // Register staging dir with the signal forwarder so SIGTERM/SIGINT can // synchronously clean it up before process.exit (the async finally block - // below does NOT run after a signal-handler exit). - _activeStagingDir = stagingDir; + // below does NOT run after a signal-handler exit). In remote-http mode we + // skip registration — the dir is meant to persist. + if (!remoteHttpMode) { + _activeStagingDir = stagingDir; + } try { const staging = writeStaged(prep.prepared, stagingDir); failed += staging.errors.length; @@ -1415,11 +1478,62 @@ async function ingestPass(args: CliArgs): Promise { } if (!args.quiet) { + const action = remoteHttpMode + ? "persisting to artifacts pipeline (skipping local gbrain import — remote-http mode)" + : "running gbrain import"; console.error( - `[memory-ingest] staged ${staging.written} pages → ${stagingDir}; running gbrain import...`, + `[memory-ingest] staged ${staging.written} pages → ${stagingDir}; ${action}...`, ); } + // Remote-http branch (split-engine D11): no local gbrain import. The + // staged markdown lives under ~/.gstack/transcripts// and the + // next gstack-brain-sync push will move it to the artifacts repo. From + // there the brain admin's pull job indexes into the remote brain. + // + // We treat ALL prepared pages as "written" since the import didn't run + // and we have no per-page failures from gbrain to filter on. The + // brain admin's pull pipeline is the authoritative gate; from this + // machine's perspective, the act of staging IS the write. + if (remoteHttpMode) { + const nowIso = new Date().toISOString(); + for (const p of prep.prepared) { + try { + state.sessions[p.source_path] = { + mtime_ns: Math.floor(statSync(p.source_path).mtimeMs * 1e6), + sha256: fileSha256(p.source_path), + ingested_at: nowIso, + page_slug: p.page_slug, + partial: p.partial, + }; + written++; + } catch (err) { + console.error( + `[state-record] ${p.source_path}: ${(err as Error).message}`, + ); + } + } + state.last_full_walk = nowIso; + state.last_writer = "gstack-memory-ingest (remote-http mode)"; + saveState(state); + if (!args.quiet) { + console.error( + `[memory-ingest] persisted ${written} pages to ${stagingDir} (brain admin will index on next pull)`, + ); + } + // Skip the gbrain-import error handling + cleanupStagingDir paths + // below by short-circuiting the function. + return { + written, + skipped_secret: prep.skippedSecret, + skipped_dedup: prep.skippedDedup, + skipped_unattributed: prep.skippedUnattributed, + failed, + duration_ms: Date.now() - t0, + partial_pages: prep.partialPages, + }; + } + // D6: single batch import. `--no-embed` matches the prior per-file // behavior (we never enabled embedding); embeddings happen on-demand // via gbrain's own pipelines. `--json` gives us structured counts. diff --git a/docs/skills.md b/docs/skills.md index b20bf665d..345a378ad 100644 --- a/docs/skills.md +++ b/docs/skills.md @@ -24,6 +24,7 @@ Detailed guides for every gstack skill — philosophy, workflow, and examples. | [`/benchmark`](#benchmark) | **Performance Engineer** | Baseline page load times, Core Web Vitals, and resource sizes. Compare before/after on every PR. Track trends over time. | | [`/cso`](#cso) | **Chief Security Officer** | OWASP Top 10 + STRIDE threat modeling security audit. Scans for injection, auth, crypto, and access control issues. | | [`/document-release`](#document-release) | **Technical Writer** | Update all project docs to match what you just shipped. Catches stale READMEs automatically. | +| [`/document-generate`](#document-generate) | **Technical Writer** | Generate Diataxis docs (tutorial / how-to / reference / explanation) for a feature from code. | | [`/retro`](#retro) | **Eng Manager** | Team-aware weekly retro. Per-person breakdowns, shipping streaks, test health trends, growth opportunities. | | [`/browse`](#browse) | **QA Engineer** | Give the agent eyes. Real Chromium browser, real clicks, real screenshots. ~100ms per command. | | [`/setup-browser-cookies`](#setup-browser-cookies) | **Session Manager** | Import cookies from your real browser (Chrome, Arc, Brave, Edge) into the headless session. Test authenticated pages. | diff --git a/gstack-upgrade/migrations/v1.37.0.0.sh b/gstack-upgrade/migrations/v1.37.0.0.sh new file mode 100755 index 000000000..b173f5844 --- /dev/null +++ b/gstack-upgrade/migrations/v1.37.0.0.sh @@ -0,0 +1,92 @@ +#!/usr/bin/env bash +# Migration: v1.37.0.0 — split-engine gbrain (remote MCP brain + optional +# local PGLite for code search per worktree). +# +# Per plan D5: prints a ONE-TIME discoverability notice for existing +# Path 4 users who don't yet have a local engine. They learn that +# symbol-aware code search (gbrain code-def / code-refs / code-callers) +# is now available via /setup-gbrain Step 4.5 if they want it. +# +# When to print the notice (state match — all conditions must hold): +# - ~/.claude.json declares mcpServers.gbrain.{type|transport} = http|sse|url +# OR mcpServers.gbrain.url is set (remote-http MCP active) +# - ~/.gbrain/config.json is absent (no local engine yet) +# - User has not previously opted out via: +# ~/.claude/skills/gstack/bin/gstack-config set local_code_index_offered true +# +# When silent: anything else (Path 1/2/3 users, anyone already on PGLite, +# anyone who opted out, anyone without remote-http MCP). +# +# Idempotency: writes a touchfile at ~/.gstack/.migrations/v1.37.0.0.done +# on completion. Re-running this script is silent if the touchfile exists, +# OR if local_code_index_offered=true. + +set -euo pipefail + +if [ -z "${HOME:-}" ]; then + echo " [v1.37.0.0] HOME is unset — skipping migration." >&2 + exit 0 +fi + +GSTACK_HOME="${GSTACK_HOME:-$HOME/.gstack}" +MIGRATIONS_DIR="$GSTACK_HOME/.migrations" +DONE_TOUCH="$MIGRATIONS_DIR/v1.37.0.0.done" +CONFIG_BIN="$HOME/.claude/skills/gstack/bin/gstack-config" +CLAUDE_JSON="$HOME/.claude.json" +GBRAIN_CONFIG="$HOME/.gbrain/config.json" + +mkdir -p "$MIGRATIONS_DIR" + +# Idempotency: already-ran skips silently. +if [ -f "$DONE_TOUCH" ]; then + exit 0 +fi + +# User opt-out skips silently AND records done. +if [ -x "$CONFIG_BIN" ]; then + if [ "$("$CONFIG_BIN" get local_code_index_offered 2>/dev/null)" = "true" ]; then + touch "$DONE_TOUCH" + exit 0 + fi +fi + +# State match: remote-http MCP active? +is_remote_http_mcp() { + [ -f "$CLAUDE_JSON" ] || return 1 + command -v jq >/dev/null 2>&1 || return 1 + local mtype murl + mtype=$(jq -r '.mcpServers.gbrain.type // .mcpServers.gbrain.transport // empty' "$CLAUDE_JSON" 2>/dev/null) + murl=$(jq -r '.mcpServers.gbrain.url // empty' "$CLAUDE_JSON" 2>/dev/null) + case "$mtype" in + url|http|sse) return 0 ;; + esac + [ -n "$murl" ] && return 0 + return 1 +} + +# State match: local engine absent? +is_local_engine_missing() { + [ ! -f "$GBRAIN_CONFIG" ] +} + +if is_remote_http_mcp && is_local_engine_missing; then + cat <<'NOTICE' + + ┌──────────────────────────────────────────────────────────────────┐ + │ gstack v1.37.0.0 — split-engine gbrain │ + │ │ + │ Symbol-aware code search is now available on this machine. │ + │ Your remote brain at gbrain MCP keeps working as today; you can │ + │ add a tiny local PGLite (~30s, no accounts) for `gbrain │ + │ code-def` / `code-refs` / `code-callers` queries per worktree. │ + │ │ + │ Run /setup-gbrain to opt in at Step 4.5. Or skip this notice │ + │ permanently: │ + │ gstack-config set local_code_index_offered true │ + └──────────────────────────────────────────────────────────────────┘ + +NOTICE +fi + +# Always touch done so we don't print again, regardless of state-match outcome. +touch "$DONE_TOUCH" diff --git a/lib/gbrain-local-status.ts b/lib/gbrain-local-status.ts new file mode 100644 index 000000000..e646abd61 --- /dev/null +++ b/lib/gbrain-local-status.ts @@ -0,0 +1,269 @@ +/** + * gbrain-local-status — classify the local gbrain engine into 5 states. + * + * Shared between bin/gstack-gbrain-detect (preamble probe on every skill start) + * and bin/gstack-gbrain-sync.ts (orchestrator SKIP-when-not-ok semantics). + * Single source of truth: same probe, same classification, same cache. + * + * Per the split-engine plan (D2 + D8): + * - Probe: `gbrain sources list --json`. Cheap (~80ms), actually hits the DB. + * Uses the same stderr patterns as lib/gbrain-sources.ts:66-67. + * - Cache: 60s TTL at ~/.gstack/.gbrain-local-status-cache.json, keyed on + * {home, path_hash, gbrain_bin_path, gbrain_version, config_mtime}. + * - --no-cache bypass: /setup-gbrain and /sync-gbrain pass it after any + * state-mutating operation so the next read sees fresh status. + * + * No-cli → gbrain not on PATH. + * Missing → CLI present, ~/.gbrain/config.json absent. + * Broken-config → config exists but `gbrain sources list` fails with config parse error + * (or any non-recognized error — defensive default per codex #8). + * Broken-db → config exists, DB unreachable per stderr classification. + * Ok → DB reachable, sources list returned valid JSON. + */ + +import { execFileSync } from "child_process"; +import { + createHash, +} from "crypto"; +import { + existsSync, + mkdirSync, + readFileSync, + renameSync, + statSync, + writeFileSync, +} from "fs"; +import { homedir } from "os"; +import { dirname, join } from "path"; + +export type LocalEngineStatus = + | "ok" + | "no-cli" + | "missing-config" + | "broken-config" + | "broken-db"; + +export interface ClassifyOptions { + /** Bypass the 60s cache. Used after any state-mutating operation. */ + noCache?: boolean; + /** Env override for the spawned `gbrain` (used by tests to point at a fake binary). */ + env?: NodeJS.ProcessEnv; +} + +interface CacheEntry { + schema_version: 1; + status: LocalEngineStatus; + cached_at: number; + /** Cache invariants — entry is invalidated if any of these change between writes. */ + key: { + home: string; + path_hash: string; + gbrain_bin_path: string; + gbrain_version: string; + config_mtime: number; // 0 when config absent + config_size: number; // 0 when config absent + }; +} + +export const CACHE_TTL_MS = 60_000; +export const PROBE_TIMEOUT_MS = 5_000; + +/** Effective user home — respects HOME env override (used by tests). */ +function userHome(): string { + return process.env.HOME || homedir(); +} + +/** Cache path computed fresh on each call so tests can mutate GSTACK_HOME per case. */ +export function cacheFilePath(): string { + return join( + process.env.GSTACK_HOME || join(userHome(), ".gstack"), + ".gbrain-local-status-cache.json", + ); +} + +function gbrainConfigPath(): string { + return join(userHome(), ".gbrain", "config.json"); +} + +function hashPath(p: string): string { + return createHash("sha256").update(p).digest("hex").slice(0, 16); +} + +/** + * Resolve the absolute path of `gbrain` on PATH. Returns null when missing. + * Memoized per-process keyed on PATH so detect's call and the classifier's + * call share one fork-exec (~200ms saved per skill preamble). + */ +const _gbrainBinCache = new Map(); +export function resolveGbrainBin(env?: NodeJS.ProcessEnv): string | null { + const e = env ?? process.env; + const key = e.PATH || ""; + if (_gbrainBinCache.has(key)) return _gbrainBinCache.get(key)!; + let result: string | null = null; + try { + const out = execFileSync("sh", ["-c", "command -v gbrain"], { + encoding: "utf-8", + timeout: 2_000, + stdio: ["ignore", "pipe", "ignore"], + env: e, + }); + result = out.trim() || null; + } catch { + result = null; + } + _gbrainBinCache.set(key, result); + return result; +} + +/** Memoized per-process. */ +const _gbrainVersionCache = new Map(); +export function readGbrainVersion(env?: NodeJS.ProcessEnv): string { + const e = env ?? process.env; + const key = `${e.PATH || ""}|${resolveGbrainBin(e) || ""}`; + if (_gbrainVersionCache.has(key)) return _gbrainVersionCache.get(key)!; + let result = ""; + try { + const out = execFileSync("gbrain", ["--version"], { + encoding: "utf-8", + timeout: 2_000, + stdio: ["ignore", "pipe", "ignore"], + env: e, + }); + result = out.trim().split("\n")[0] || ""; + } catch { + result = ""; + } + _gbrainVersionCache.set(key, result); + return result; +} + +function configFingerprint(): { mtime: number; size: number } { + try { + const st = statSync(gbrainConfigPath()); + return { mtime: Math.floor(st.mtimeMs), size: st.size }; + } catch { + return { mtime: 0, size: 0 }; + } +} + +function buildCacheKey( + gbrainBin: string | null, + gbrainVersion: string, + env?: NodeJS.ProcessEnv, +): CacheEntry["key"] { + const e = env ?? process.env; + const config = configFingerprint(); + return { + home: e.HOME || "", + path_hash: hashPath(e.PATH || ""), + gbrain_bin_path: gbrainBin || "", + gbrain_version: gbrainVersion, + config_mtime: config.mtime, + config_size: config.size, + }; +} + +function keysEqual(a: CacheEntry["key"], b: CacheEntry["key"]): boolean { + return ( + a.home === b.home && + a.path_hash === b.path_hash && + a.gbrain_bin_path === b.gbrain_bin_path && + a.gbrain_version === b.gbrain_version && + a.config_mtime === b.config_mtime && + a.config_size === b.config_size + ); +} + +function readCache(key: CacheEntry["key"]): LocalEngineStatus | null { + if (!existsSync(cacheFilePath())) return null; + try { + const raw = JSON.parse(readFileSync(cacheFilePath(), "utf-8")) as CacheEntry; + if (raw.schema_version !== 1) return null; + if (Date.now() - raw.cached_at > CACHE_TTL_MS) return null; + if (!keysEqual(raw.key, key)) return null; + return raw.status; + } catch { + return null; + } +} + +function writeCache(status: LocalEngineStatus, key: CacheEntry["key"]): void { + const entry: CacheEntry = { + schema_version: 1, + status, + cached_at: Date.now(), + key, + }; + try { + mkdirSync(dirname(cacheFilePath()), { recursive: true }); + const tmp = cacheFilePath() + ".tmp." + process.pid; + writeFileSync(tmp, JSON.stringify(entry, null, 2), "utf-8"); + renameSync(tmp, cacheFilePath()); + } catch { + // Cache write failure is non-fatal — we re-probe next call. + } +} + +/** + * Probe via `gbrain sources list --json`. Classify the outcome. + * + * Pattern strings ("Cannot connect to database", "config.json") are deliberately + * the same strings used in lib/gbrain-sources.ts:66-67. If gbrain reworks its + * error messages, classifier returns broken-config defensively (codex #8). + */ +function freshClassify(env?: NodeJS.ProcessEnv): LocalEngineStatus { + // 1. CLI on PATH? + const gbrainBin = resolveGbrainBin(env); + if (!gbrainBin) return "no-cli"; + + // 2. Config file present? + if (!existsSync(gbrainConfigPath())) return "missing-config"; + + // 3. Probe gbrain sources list. + try { + execFileSync("gbrain", ["sources", "list", "--json"], { + encoding: "utf-8", + timeout: PROBE_TIMEOUT_MS, + stdio: ["ignore", "pipe", "pipe"], + env: env ?? process.env, + }); + return "ok"; + } catch (err) { + const e = err as NodeJS.ErrnoException & { stderr?: Buffer | string }; + const stderr = (e.stderr ? e.stderr.toString() : "") || ""; + + // ENOENT can happen if gbrain disappeared between resolveGbrainBin and now. + if (e.code === "ENOENT") return "no-cli"; + + // Pattern match against gbrain's known error strings. Order matters: + // "Cannot connect to database" is the more specific DB-unreachable signal. + if (stderr.includes("Cannot connect to database")) return "broken-db"; + if (stderr.includes("config.json")) return "broken-config"; + + // Defensive default per codex #8: unrecognized failures classify as + // broken-config so the user sees the raw stderr surfaced upstream. + return "broken-config"; + } +} + +/** + * Classify the local gbrain engine status. Cached for 60s; bypassable. + * + * Returns one of 5 states. Never throws — failure modes are surfaced as states. + */ +export function localEngineStatus(opts: ClassifyOptions = {}): LocalEngineStatus { + const env = opts.env ?? process.env; + const gbrainBin = resolveGbrainBin(env); + const gbrainVersion = gbrainBin ? readGbrainVersion(env) : ""; + const key = buildCacheKey(gbrainBin, gbrainVersion, env); + + if (!opts.noCache) { + const cached = readCache(key); + if (cached) return cached; + } + + const fresh = freshClassify(env); + writeCache(fresh, key); + return fresh; +} + diff --git a/package.json b/package.json index 6378d48bc..571d13851 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gstack", - "version": "1.35.0.0", + "version": "1.37.0.0", "description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.", "license": "MIT", "type": "module", diff --git a/setup-gbrain/SKILL.md b/setup-gbrain/SKILL.md index 19d18afb7..c1abd775c 100644 --- a/setup-gbrain/SKILL.md +++ b/setup-gbrain/SKILL.md @@ -785,8 +785,10 @@ implemented as a dispatcher binary. ``` Capture the JSON output. It contains: `gbrain_on_path`, `gbrain_version`, -`gbrain_config_exists`, `gbrain_engine`, `gbrain_doctor_ok`, -`gstack_brain_sync_mode`, `gstack_brain_git`. +`gbrain_config_exists`, `gbrain_engine`, `gbrain_doctor_ok`, `gbrain_mcp_mode`, +`gstack_brain_sync_mode`, `gstack_brain_git`, `gstack_artifacts_remote`, and +the v1.34.0.0+ `gbrain_local_status` field (one of: `ok`, `no-cli`, +`missing-config`, `broken-config`, `broken-db`). Skip downstream steps that are already done. Report the detected state in one line so the user knows what you found: @@ -799,6 +801,75 @@ invocation flags here and skip to the matching step. --- +## Step 1.5: Broken-local-engine remediation (plan D4) + +Read `gbrain_local_status` from the Step 1 detect output. **If it's `broken-db` +or `broken-config` AND no shortcut flag was passed**, the user has a +non-working local engine (Garry's repro: `~/.gbrain/config.json` points at a +dead Postgres URL). Fire a targeted AskUserQuestion BEFORE Step 2: + +> D# — Your local gbrain engine isn't responding. How do you want to fix it? +> Project/branch/task: +> ELI10: gbrain has a config at `~/.gbrain/config.json` but the engine it points +> at isn't reachable. That could be a transient outage (Postgres container +> stopped, Tailscale down) OR a stale config you want to abandon. Different +> remediation for each case. +> Stakes if we pick wrong: "Switch to PGLite" overwrites your existing config +> (one-way door if the user actually wanted the broken engine). "Retry" preserves +> existing state for transient cases. +> Recommendation: A (Retry) — always try the cheap option first; if engine is +> just temporarily down it'll come back without any destructive change. +> Note: options differ in kind, not coverage — no completeness score. +> A) Retry — re-probe the engine (recommended; ~80ms) +> ✅ Cheapest test: re-runs `gbrain sources list` to see if engine is back +> ✅ Zero side effects; existing config preserved +> ❌ If engine is permanently dead, retries forever; user must choose another option +> B) Switch to local PGLite (one-way — moves existing config to .bak) +> ✅ Fastest path to a working local engine if user has abandoned the old one +> ✅ ~30s; no accounts; private to this machine +> ❌ Destructive — existing config moved to ~/.gbrain/config.json.gstack-bak-{ts} +> C) Switch brain mode (continue to Step 2 path picker) +> ✅ Lets user pick Path 1/2/3/4 to re-init from scratch +> ✅ Preserves existing config until they explicitly init the new one +> ❌ Longer flow if user just wants to repair to PGLite +> D) Quit (do nothing) +> ✅ No cons — this is a hard-stop choice +> ❌ N/A +> Net: A is the right starting move; B/C are explicit destructive paths; D bails. + +**If A (Retry)**: re-run `~/.claude/skills/gstack/bin/gstack-gbrain-detect` +with `GSTACK_DETECT_NO_CACHE=1` (busts the 60s cache). If the new +`gbrain_local_status` is `ok`, continue to Step 2. If still `broken-db` or +`broken-config`, fire the same AskUserQuestion again (the user picks again). + +**If B (Switch to PGLite)** — execute the rollback-safe init sequence (plan D7): + +```bash +BACKUP="$HOME/.gbrain/config.json.gstack-bak-$(date +%s)" +mv "$HOME/.gbrain/config.json" "$BACKUP" +if ! gbrain init --pglite --json; then + # Restore on failure + mv "$BACKUP" "$HOME/.gbrain/config.json" + echo "gbrain init failed. Your previous config was restored at $HOME/.gbrain/config.json." >&2 + echo "PGLite directory at ~/.gbrain/pglite/ may be in a partial state — \`rm -rf ~/.gbrain/pglite\` if needed before retrying." >&2 + exit 1 +fi +echo "Switched to local PGLite. Previous config saved at $BACKUP — review before deleting." +``` + +Then jump to Step 5a (MCP registration; the new PGLite engine is registered as +local-stdio). + +**If C (Switch brain mode)**: continue to Step 2's normal path picker. + +**If D (Quit)**: STOP the skill cleanly. + +For `gbrain_local_status` values of `no-cli` or `missing-config`, do NOT fire +Step 1.5 — fall through to Step 2 (where `no-cli` triggers Step 3 install and +`missing-config` triggers Step 4 init). + +--- + ## Step 2: Pick a path (AskUserQuestion) Only fire this if Step 1 shows no existing working config AND no shortcut @@ -1034,11 +1105,60 @@ Capture two values from the verify output for downstream steps: - `URL_FORM_SUPPORTED` (`true|false`) — passed to `gstack-artifacts-init` in Step 7 to control which form of the brain-admin hookup command is printed. -**4d. Skip Steps 3, 4 (other paths), 5 (local doctor), 7.5 (transcript ingest).** -All four require a working local `gbrain` CLI that Path 4 does not install. -The skill jumps straight to Step 5a (HTTP+bearer registration) → Step 6 -(per-remote policy) → Step 7 (artifacts repo) → Step 8 (CLAUDE.md) → Step 9 -(remote smoke test) → Step 10 (verdict). +**4d. (Path 4) Offer local PGLite for code search.** Per plan D10/D11, ask: + +> D# — Want symbol-aware code search on this machine? +> Project/branch/task: +> ELI10: The remote brain at `` is great for cross-machine knowledge, +> but symbol queries like `gbrain code-def` / `code-refs` / `code-callers` need +> a local index of THIS machine's code. We can spin up a tiny isolated PGLite +> database (~30 seconds, no accounts, ~120 MB disk) just for code, separate +> from your remote brain. Transcripts and artifacts continue routing through +> the artifacts repo to the remote brain — local PGLite stays code-only. +> Stakes: without it, semantic code search in this repo's worktrees falls +> back to Grep. +> Recommendation: A — 30 seconds, no ongoing cost, unlocks the symbol tools. +> Completeness: A=10/10 (full split-engine), B=7/10 (remote-only). +> A) Yes, set up local PGLite for code (recommended) +> ✅ Unlocks `gbrain code-def`, `code-refs`, `code-callers` per worktree +> ✅ Independent engine — won't disturb remote brain or share transcripts +> B) No, remote MCP only +> ✅ Zero local state — only `~/.claude.json` MCP registration +> ❌ Symbol code queries fall back to Grep in this repo's worktrees +> Net: A = full split-engine; B = remote-only. + +**If A (Yes)**: install + init local PGLite with rollback-safe semantics (D7): + +```bash +~/.claude/skills/gstack/bin/gstack-gbrain-install || exit $? +# At this point the local gbrain CLI is on PATH. Init PGLite, but back up any +# existing ~/.gbrain/config.json first (rollback if init fails). +if [ -f "$HOME/.gbrain/config.json" ]; then + BACKUP="$HOME/.gbrain/config.json.gstack-bak-$(date +%s)" + mv "$HOME/.gbrain/config.json" "$BACKUP" +fi +if ! gbrain init --pglite --json; then + if [ -n "${BACKUP:-}" ] && [ -f "$BACKUP" ]; then mv "$BACKUP" "$HOME/.gbrain/config.json"; fi + echo "gbrain init failed. Existing config (if any) was restored. PGLite at ~/.gbrain/pglite/ may be in a partial state — \`rm -rf ~/.gbrain/pglite\` to reset." >&2 + echo "Continuing setup without local code search; you can re-run /setup-gbrain to retry." >&2 +fi +``` + +Then continue to Step 5a. The remote-http MCP registration in 5a runs as +today; the local PGLite is independent of MCP registration (Claude Code talks +to the remote brain via MCP for queries; `gbrain` CLI talks to local PGLite +for code-def/refs/callers). + +**If B (No)**: skip the install + init. The local engine stays absent. +`gbrain_local_status` will be `missing-config` (or `no-cli` if gbrain isn't +installed). `/sync-gbrain` will SKIP the code stage cleanly per plan D12. + +**4e. Skip Steps 3, 4 (other paths) and 5 (local doctor) when B was picked.** +When A was picked, Step 3 already ran (via gstack-gbrain-install) and Step 4 +already ran (via `gbrain init --pglite`); jump straight to Step 5a. When B +was picked, Steps 3/4/5 are no-ops; also skip Step 7.5 (transcript ingest) +since memory-stage routes through the artifacts pipeline in remote-http mode +per plan D11. The bearer token (`GBRAIN_MCP_TOKEN`) stays in process env until Step 5a's `claude mcp add --header` consumes it; then `unset GBRAIN_MCP_TOKEN` @@ -1475,7 +1595,8 @@ gbrain status: GREEN (mode: remote-http) Repo policy ..... OK {read-write|read-only|deny} Artifacts repo .. OK {gstack_artifacts_remote URL} Artifacts sync .. OK {artifacts_sync_mode} - Transcripts ..... N/A remote mode (ingest happens on brain host) + Transcripts ..... OK route to artifacts repo → remote brain (plan D11) + Code search ..... {OK local-pglite (~/.gbrain/pglite) | N/A declined at Step 4d} CLAUDE.md ....... OK Smoke test ...... INFO printed for post-restart manual verification @@ -1483,6 +1604,16 @@ Restart Claude Code to pick up the `mcp__gbrain__*` tools. Re-run `/setup-gbrain` any time the bearer rotates or the URL moves. ``` +The **Code search** row reflects the choice at Step 4d: +- If user picked A (Yes): `OK local-pglite` and `gbrain_local_status == "ok"` going forward. +- If user picked B (No): `N/A declined at Step 4d` — `gstack-config set local_code_index_offered true` to silence future migration notices. + +The **Transcripts** row changed in v1.34.0.0: in remote-http mode, +gstack-memory-ingest now persists staged transcripts to +`~/.gstack/transcripts/run--/` and gstack-brain-sync pushes them +to the artifacts repo. Brain admin's pull job indexes into the remote brain. +Local PGLite (when present) stays code-only — no transcript pollution. + ### Paths 1, 2a, 2b, 3 (Local stdio) ``` diff --git a/setup-gbrain/SKILL.md.tmpl b/setup-gbrain/SKILL.md.tmpl index a2a49cee1..a0bc59769 100644 --- a/setup-gbrain/SKILL.md.tmpl +++ b/setup-gbrain/SKILL.md.tmpl @@ -63,8 +63,10 @@ implemented as a dispatcher binary. ``` Capture the JSON output. It contains: `gbrain_on_path`, `gbrain_version`, -`gbrain_config_exists`, `gbrain_engine`, `gbrain_doctor_ok`, -`gstack_brain_sync_mode`, `gstack_brain_git`. +`gbrain_config_exists`, `gbrain_engine`, `gbrain_doctor_ok`, `gbrain_mcp_mode`, +`gstack_brain_sync_mode`, `gstack_brain_git`, `gstack_artifacts_remote`, and +the v1.34.0.0+ `gbrain_local_status` field (one of: `ok`, `no-cli`, +`missing-config`, `broken-config`, `broken-db`). Skip downstream steps that are already done. Report the detected state in one line so the user knows what you found: @@ -77,6 +79,75 @@ invocation flags here and skip to the matching step. --- +## Step 1.5: Broken-local-engine remediation (plan D4) + +Read `gbrain_local_status` from the Step 1 detect output. **If it's `broken-db` +or `broken-config` AND no shortcut flag was passed**, the user has a +non-working local engine (Garry's repro: `~/.gbrain/config.json` points at a +dead Postgres URL). Fire a targeted AskUserQuestion BEFORE Step 2: + +> D# — Your local gbrain engine isn't responding. How do you want to fix it? +> Project/branch/task: +> ELI10: gbrain has a config at `~/.gbrain/config.json` but the engine it points +> at isn't reachable. That could be a transient outage (Postgres container +> stopped, Tailscale down) OR a stale config you want to abandon. Different +> remediation for each case. +> Stakes if we pick wrong: "Switch to PGLite" overwrites your existing config +> (one-way door if the user actually wanted the broken engine). "Retry" preserves +> existing state for transient cases. +> Recommendation: A (Retry) — always try the cheap option first; if engine is +> just temporarily down it'll come back without any destructive change. +> Note: options differ in kind, not coverage — no completeness score. +> A) Retry — re-probe the engine (recommended; ~80ms) +> ✅ Cheapest test: re-runs `gbrain sources list` to see if engine is back +> ✅ Zero side effects; existing config preserved +> ❌ If engine is permanently dead, retries forever; user must choose another option +> B) Switch to local PGLite (one-way — moves existing config to .bak) +> ✅ Fastest path to a working local engine if user has abandoned the old one +> ✅ ~30s; no accounts; private to this machine +> ❌ Destructive — existing config moved to ~/.gbrain/config.json.gstack-bak-{ts} +> C) Switch brain mode (continue to Step 2 path picker) +> ✅ Lets user pick Path 1/2/3/4 to re-init from scratch +> ✅ Preserves existing config until they explicitly init the new one +> ❌ Longer flow if user just wants to repair to PGLite +> D) Quit (do nothing) +> ✅ No cons — this is a hard-stop choice +> ❌ N/A +> Net: A is the right starting move; B/C are explicit destructive paths; D bails. + +**If A (Retry)**: re-run `~/.claude/skills/gstack/bin/gstack-gbrain-detect` +with `GSTACK_DETECT_NO_CACHE=1` (busts the 60s cache). If the new +`gbrain_local_status` is `ok`, continue to Step 2. If still `broken-db` or +`broken-config`, fire the same AskUserQuestion again (the user picks again). + +**If B (Switch to PGLite)** — execute the rollback-safe init sequence (plan D7): + +```bash +BACKUP="$HOME/.gbrain/config.json.gstack-bak-$(date +%s)" +mv "$HOME/.gbrain/config.json" "$BACKUP" +if ! gbrain init --pglite --json; then + # Restore on failure + mv "$BACKUP" "$HOME/.gbrain/config.json" + echo "gbrain init failed. Your previous config was restored at $HOME/.gbrain/config.json." >&2 + echo "PGLite directory at ~/.gbrain/pglite/ may be in a partial state — \`rm -rf ~/.gbrain/pglite\` if needed before retrying." >&2 + exit 1 +fi +echo "Switched to local PGLite. Previous config saved at $BACKUP — review before deleting." +``` + +Then jump to Step 5a (MCP registration; the new PGLite engine is registered as +local-stdio). + +**If C (Switch brain mode)**: continue to Step 2's normal path picker. + +**If D (Quit)**: STOP the skill cleanly. + +For `gbrain_local_status` values of `no-cli` or `missing-config`, do NOT fire +Step 1.5 — fall through to Step 2 (where `no-cli` triggers Step 3 install and +`missing-config` triggers Step 4 init). + +--- + ## Step 2: Pick a path (AskUserQuestion) Only fire this if Step 1 shows no existing working config AND no shortcut @@ -312,11 +383,60 @@ Capture two values from the verify output for downstream steps: - `URL_FORM_SUPPORTED` (`true|false`) — passed to `gstack-artifacts-init` in Step 7 to control which form of the brain-admin hookup command is printed. -**4d. Skip Steps 3, 4 (other paths), 5 (local doctor), 7.5 (transcript ingest).** -All four require a working local `gbrain` CLI that Path 4 does not install. -The skill jumps straight to Step 5a (HTTP+bearer registration) → Step 6 -(per-remote policy) → Step 7 (artifacts repo) → Step 8 (CLAUDE.md) → Step 9 -(remote smoke test) → Step 10 (verdict). +**4d. (Path 4) Offer local PGLite for code search.** Per plan D10/D11, ask: + +> D# — Want symbol-aware code search on this machine? +> Project/branch/task: +> ELI10: The remote brain at `` is great for cross-machine knowledge, +> but symbol queries like `gbrain code-def` / `code-refs` / `code-callers` need +> a local index of THIS machine's code. We can spin up a tiny isolated PGLite +> database (~30 seconds, no accounts, ~120 MB disk) just for code, separate +> from your remote brain. Transcripts and artifacts continue routing through +> the artifacts repo to the remote brain — local PGLite stays code-only. +> Stakes: without it, semantic code search in this repo's worktrees falls +> back to Grep. +> Recommendation: A — 30 seconds, no ongoing cost, unlocks the symbol tools. +> Completeness: A=10/10 (full split-engine), B=7/10 (remote-only). +> A) Yes, set up local PGLite for code (recommended) +> ✅ Unlocks `gbrain code-def`, `code-refs`, `code-callers` per worktree +> ✅ Independent engine — won't disturb remote brain or share transcripts +> B) No, remote MCP only +> ✅ Zero local state — only `~/.claude.json` MCP registration +> ❌ Symbol code queries fall back to Grep in this repo's worktrees +> Net: A = full split-engine; B = remote-only. + +**If A (Yes)**: install + init local PGLite with rollback-safe semantics (D7): + +```bash +~/.claude/skills/gstack/bin/gstack-gbrain-install || exit $? +# At this point the local gbrain CLI is on PATH. Init PGLite, but back up any +# existing ~/.gbrain/config.json first (rollback if init fails). +if [ -f "$HOME/.gbrain/config.json" ]; then + BACKUP="$HOME/.gbrain/config.json.gstack-bak-$(date +%s)" + mv "$HOME/.gbrain/config.json" "$BACKUP" +fi +if ! gbrain init --pglite --json; then + if [ -n "${BACKUP:-}" ] && [ -f "$BACKUP" ]; then mv "$BACKUP" "$HOME/.gbrain/config.json"; fi + echo "gbrain init failed. Existing config (if any) was restored. PGLite at ~/.gbrain/pglite/ may be in a partial state — \`rm -rf ~/.gbrain/pglite\` to reset." >&2 + echo "Continuing setup without local code search; you can re-run /setup-gbrain to retry." >&2 +fi +``` + +Then continue to Step 5a. The remote-http MCP registration in 5a runs as +today; the local PGLite is independent of MCP registration (Claude Code talks +to the remote brain via MCP for queries; `gbrain` CLI talks to local PGLite +for code-def/refs/callers). + +**If B (No)**: skip the install + init. The local engine stays absent. +`gbrain_local_status` will be `missing-config` (or `no-cli` if gbrain isn't +installed). `/sync-gbrain` will SKIP the code stage cleanly per plan D12. + +**4e. Skip Steps 3, 4 (other paths) and 5 (local doctor) when B was picked.** +When A was picked, Step 3 already ran (via gstack-gbrain-install) and Step 4 +already ran (via `gbrain init --pglite`); jump straight to Step 5a. When B +was picked, Steps 3/4/5 are no-ops; also skip Step 7.5 (transcript ingest) +since memory-stage routes through the artifacts pipeline in remote-http mode +per plan D11. The bearer token (`GBRAIN_MCP_TOKEN`) stays in process env until Step 5a's `claude mcp add --header` consumes it; then `unset GBRAIN_MCP_TOKEN` @@ -753,7 +873,8 @@ gbrain status: GREEN (mode: remote-http) Repo policy ..... OK {read-write|read-only|deny} Artifacts repo .. OK {gstack_artifacts_remote URL} Artifacts sync .. OK {artifacts_sync_mode} - Transcripts ..... N/A remote mode (ingest happens on brain host) + Transcripts ..... OK route to artifacts repo → remote brain (plan D11) + Code search ..... {OK local-pglite (~/.gbrain/pglite) | N/A declined at Step 4d} CLAUDE.md ....... OK Smoke test ...... INFO printed for post-restart manual verification @@ -761,6 +882,16 @@ Restart Claude Code to pick up the `mcp__gbrain__*` tools. Re-run `/setup-gbrain` any time the bearer rotates or the URL moves. ``` +The **Code search** row reflects the choice at Step 4d: +- If user picked A (Yes): `OK local-pglite` and `gbrain_local_status == "ok"` going forward. +- If user picked B (No): `N/A declined at Step 4d` — `gstack-config set local_code_index_offered true` to silence future migration notices. + +The **Transcripts** row changed in v1.34.0.0: in remote-http mode, +gstack-memory-ingest now persists staged transcripts to +`~/.gstack/transcripts/run--/` and gstack-brain-sync pushes them +to the artifacts repo. Brain admin's pull job indexes into the remote brain. +Local PGLite (when present) stays code-only — no transcript pollution. + ### Paths 1, 2a, 2b, 3 (Local stdio) ``` diff --git a/sync-gbrain/SKILL.md b/sync-gbrain/SKILL.md index afebd31f1..8dba77386 100644 --- a/sync-gbrain/SKILL.md +++ b/sync-gbrain/SKILL.md @@ -788,28 +788,20 @@ Before doing anything, check that /setup-gbrain has been run on this Mac. ~/.claude/skills/gstack/bin/gstack-gbrain-detect 2>/dev/null ``` -**Split-engine model.** Code stage always runs locally against a per-machine -PGLite brain (or whatever `gbrain config` points to), with each worktree of a -repo registered as its own source. Artifacts/memory stages route through -whatever `setup-gbrain` configured — including remote-MCP (Path 4). The two -sides are independent: code lookups are local + worktree-scoped, artifacts -remain cross-machine. +**Split-engine model (v1.34.0.0+).** Code stage runs locally against the +per-machine gbrain engine (PGLite or whatever `gbrain config` points to), +with each worktree of a repo registered as its own source. **Memory stage +also runs locally** in local-stdio MCP mode — `gstack-memory-ingest` shells +out to `gbrain import` against the same local engine. In remote-http MCP +mode (Path 4), the memory stage instead persists staged markdown to +`~/.gstack/transcripts//` and the artifacts pipeline pushes it to +the brain admin's pull job (plan D11). Brain-sync (the `gstack-brain-sync` +push to git) is the one stage that never touches local engine and runs +regardless of mode. -A previous version of this skill bounced remote-MCP users out of the code -stage entirely. That was wrong: the code-stage CLI calls (`gbrain sources -add`, `sync --strategy code`, `sources attach`) target the LOCAL gbrain CLI -+ DB regardless of whether `~/.claude.json` has `gbrain` registered as a -remote HTTP MCP for artifacts. We no longer skip the code stage in -remote-MCP mode. - -If `gbrain_on_path=false` OR `gbrain_config_exists=false`, STOP and tell -the user: - -> "/sync-gbrain requires /setup-gbrain to be run first. Run `/setup-gbrain` -> to install gbrain, register the MCP server, and set per-repo trust policy." - -Do NOT continue — the skill is unsafe when the local gbrain CLI is missing -(we'd write a CLAUDE.md guidance block referencing tools that don't exist). +Practically: local PGLite stays code-only on remote-http machines; the +remote brain holds everything else. Local-stdio machines mix code + +transcripts in one local engine, as they always have. Also check the per-repo trust policy. If `gstack-gbrain-repo-policy get` for this repo returns `deny`, STOP: @@ -819,6 +811,44 @@ this repo returns `deny`, STOP: --- +## Step 1.5: Local engine pre-flight (plan D12) + +Read `gbrain_local_status` from the Step 1 detect output. Branch as follows +BEFORE invoking the orchestrator: + +- **`ok`**: proceed to Step 2 normally. +- **`no-cli`**: STOP. "Local gbrain CLI not installed. Run `/setup-gbrain` + first." +- **`missing-config`** AND `gbrain_mcp_mode == "remote-http"`: tell the user + "Your brain queries (the `mcp__gbrain__*` tools) work via remote MCP, but + symbol code search needs a local PGLite. Run `/setup-gbrain` and pick + 'Yes' at the new 'local code index' prompt (Step 4.5), or run + `gbrain init --pglite --json` directly. Continuing without code stage." + Then proceed to Step 2 — the orchestrator's `runCodeImport()` and + `runMemoryIngest()` will return SKIP per plan D12; only `runBrainSyncPush()` + will run. Do NOT abort. +- **`missing-config`** AND `gbrain_mcp_mode != "remote-http"`: STOP. "Local + gbrain CLI is installed but no engine config. Run `/setup-gbrain` first." +- **`broken-config`** OR **`broken-db`**: STOP with a clear message: + ``` + Local gbrain config at ~/.gbrain/config.json points at an unreachable + engine (status: {gbrain_local_status}). Two options: + 1. Re-run /setup-gbrain — Step 1.5 offers Retry / Switch to PGLite / + Switch brain mode / Quit (plan D4). + 2. Repair manually: mv ~/.gbrain/config.json ~/.gbrain/config.json.bak + && gbrain init --pglite --json + Re-run /sync-gbrain after. + ``` + Do NOT continue — the orchestrator would skip code+memory and only run + brain-sync, which is a degraded state the user should fix explicitly. + +This pre-flight short-circuits the orchestrator before it spends ~80ms +probing the engine again. The orchestrator independently runs the same +classifier for defense-in-depth, but Step 1.5's STOP is where the user +gets the actionable remediation message. + +--- + ## Step 2: Run the orchestrator Pass user args to the orchestrator. Do not paraphrase them — pass through diff --git a/sync-gbrain/SKILL.md.tmpl b/sync-gbrain/SKILL.md.tmpl index f40e05052..b05c39066 100644 --- a/sync-gbrain/SKILL.md.tmpl +++ b/sync-gbrain/SKILL.md.tmpl @@ -66,28 +66,20 @@ Before doing anything, check that /setup-gbrain has been run on this Mac. ~/.claude/skills/gstack/bin/gstack-gbrain-detect 2>/dev/null ``` -**Split-engine model.** Code stage always runs locally against a per-machine -PGLite brain (or whatever `gbrain config` points to), with each worktree of a -repo registered as its own source. Artifacts/memory stages route through -whatever `setup-gbrain` configured — including remote-MCP (Path 4). The two -sides are independent: code lookups are local + worktree-scoped, artifacts -remain cross-machine. +**Split-engine model (v1.34.0.0+).** Code stage runs locally against the +per-machine gbrain engine (PGLite or whatever `gbrain config` points to), +with each worktree of a repo registered as its own source. **Memory stage +also runs locally** in local-stdio MCP mode — `gstack-memory-ingest` shells +out to `gbrain import` against the same local engine. In remote-http MCP +mode (Path 4), the memory stage instead persists staged markdown to +`~/.gstack/transcripts//` and the artifacts pipeline pushes it to +the brain admin's pull job (plan D11). Brain-sync (the `gstack-brain-sync` +push to git) is the one stage that never touches local engine and runs +regardless of mode. -A previous version of this skill bounced remote-MCP users out of the code -stage entirely. That was wrong: the code-stage CLI calls (`gbrain sources -add`, `sync --strategy code`, `sources attach`) target the LOCAL gbrain CLI -+ DB regardless of whether `~/.claude.json` has `gbrain` registered as a -remote HTTP MCP for artifacts. We no longer skip the code stage in -remote-MCP mode. - -If `gbrain_on_path=false` OR `gbrain_config_exists=false`, STOP and tell -the user: - -> "/sync-gbrain requires /setup-gbrain to be run first. Run `/setup-gbrain` -> to install gbrain, register the MCP server, and set per-repo trust policy." - -Do NOT continue — the skill is unsafe when the local gbrain CLI is missing -(we'd write a CLAUDE.md guidance block referencing tools that don't exist). +Practically: local PGLite stays code-only on remote-http machines; the +remote brain holds everything else. Local-stdio machines mix code + +transcripts in one local engine, as they always have. Also check the per-repo trust policy. If `gstack-gbrain-repo-policy get` for this repo returns `deny`, STOP: @@ -97,6 +89,44 @@ this repo returns `deny`, STOP: --- +## Step 1.5: Local engine pre-flight (plan D12) + +Read `gbrain_local_status` from the Step 1 detect output. Branch as follows +BEFORE invoking the orchestrator: + +- **`ok`**: proceed to Step 2 normally. +- **`no-cli`**: STOP. "Local gbrain CLI not installed. Run `/setup-gbrain` + first." +- **`missing-config`** AND `gbrain_mcp_mode == "remote-http"`: tell the user + "Your brain queries (the `mcp__gbrain__*` tools) work via remote MCP, but + symbol code search needs a local PGLite. Run `/setup-gbrain` and pick + 'Yes' at the new 'local code index' prompt (Step 4.5), or run + `gbrain init --pglite --json` directly. Continuing without code stage." + Then proceed to Step 2 — the orchestrator's `runCodeImport()` and + `runMemoryIngest()` will return SKIP per plan D12; only `runBrainSyncPush()` + will run. Do NOT abort. +- **`missing-config`** AND `gbrain_mcp_mode != "remote-http"`: STOP. "Local + gbrain CLI is installed but no engine config. Run `/setup-gbrain` first." +- **`broken-config`** OR **`broken-db`**: STOP with a clear message: + ``` + Local gbrain config at ~/.gbrain/config.json points at an unreachable + engine (status: {gbrain_local_status}). Two options: + 1. Re-run /setup-gbrain — Step 1.5 offers Retry / Switch to PGLite / + Switch brain mode / Quit (plan D4). + 2. Repair manually: mv ~/.gbrain/config.json ~/.gbrain/config.json.bak + && gbrain init --pglite --json + Re-run /sync-gbrain after. + ``` + Do NOT continue — the orchestrator would skip code+memory and only run + brain-sync, which is a degraded state the user should fix explicitly. + +This pre-flight short-circuits the orchestrator before it spends ~80ms +probing the engine again. The orchestrator independently runs the same +classifier for defense-in-depth, but Step 1.5's STOP is where the user +gets the actionable remediation message. + +--- + ## Step 2: Run the orchestrator Pass user args to the orchestrator. Do not paraphrase them — pass through diff --git a/test/gbrain-detect-shape.test.ts b/test/gbrain-detect-shape.test.ts new file mode 100644 index 000000000..465e55623 --- /dev/null +++ b/test/gbrain-detect-shape.test.ts @@ -0,0 +1,246 @@ +/** + * Shape regression test for bin/gstack-gbrain-detect. + * + * After the bash→TS rewrite (codex #5), the TS output must stay + * key/type/semantics backward-compatible with the bash version. Downstream + * callers across most gstack skill preambles shell out to this script and + * pipe through jq. Key order may differ between bash+jq and JSON.stringify; + * key NAMES and TYPES must not. + * + * Asserts: + * 1. All 9 pre-existing keys are present + * 2. Each pre-existing key has the same primitive type/union as the bash version + * 3. The new key (gbrain_local_status) is present and a string + * 4. Output is parseable JSON + * 5. No keys removed/renamed + */ + +import { describe, it, expect } from "bun:test"; +import { execFileSync } from "child_process"; +import { + mkdtempSync, + mkdirSync, + writeFileSync, + chmodSync, + rmSync, +} from "fs"; +import { tmpdir } from "os"; +import { join } from "path"; + +const DETECT_BIN = join(import.meta.dir, "..", "bin", "gstack-gbrain-detect"); + +/** Absolute bun path resolved once at module load (uses the test runner's PATH). */ +const BUN_BIN = execFileSync("sh", ["-c", "command -v bun"], { encoding: "utf-8" }).trim(); + +/** + * Run detect with a controlled HOME + PATH so the output is deterministic. + * We invoke via `bun run ` instead of the shebang so the test doesn't + * need bun on its PATH. The script's child-process probes still respect + * the controlled PATH. + */ +function runDetect(env: Partial): string { + return execFileSync(BUN_BIN, ["run", DETECT_BIN], { + encoding: "utf-8", + timeout: 15_000, + stdio: ["ignore", "pipe", "pipe"], + env: { ...process.env, ...env }, + }); +} + +interface DetectShape { + gbrain_on_path: boolean; + gbrain_version: string | null; + gbrain_config_exists: boolean; + gbrain_engine: string | null; + gbrain_doctor_ok: boolean; + gbrain_mcp_mode: string; + gstack_brain_sync_mode: string; + gstack_brain_git: boolean; + gstack_artifacts_remote: string; + gbrain_local_status: string; +} + +describe("bin/gstack-gbrain-detect — shape regression", () => { + it("emits valid JSON", () => { + const tmp = mkdtempSync(join(tmpdir(), "detect-shape-")); + try { + const out = runDetect({ + HOME: tmp, + PATH: "/usr/bin:/bin", + GSTACK_HOME: tmp, + }); + expect(() => JSON.parse(out)).not.toThrow(); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + it("contains all 9 pre-existing keys + the new gbrain_local_status key", () => { + const tmp = mkdtempSync(join(tmpdir(), "detect-shape-")); + try { + const out = runDetect({ + HOME: tmp, + PATH: "/usr/bin:/bin", + GSTACK_HOME: tmp, + }); + const parsed = JSON.parse(out) as DetectShape; + + // 9 pre-existing keys (must not be removed/renamed): + expect(parsed).toHaveProperty("gbrain_on_path"); + expect(parsed).toHaveProperty("gbrain_version"); + expect(parsed).toHaveProperty("gbrain_config_exists"); + expect(parsed).toHaveProperty("gbrain_engine"); + expect(parsed).toHaveProperty("gbrain_doctor_ok"); + expect(parsed).toHaveProperty("gbrain_mcp_mode"); + expect(parsed).toHaveProperty("gstack_brain_sync_mode"); + expect(parsed).toHaveProperty("gstack_brain_git"); + expect(parsed).toHaveProperty("gstack_artifacts_remote"); + + // 1 new key (added by this fix): + expect(parsed).toHaveProperty("gbrain_local_status"); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + it("preserves field types from the bash version", () => { + const tmp = mkdtempSync(join(tmpdir(), "detect-shape-")); + try { + const out = runDetect({ + HOME: tmp, + PATH: "/usr/bin:/bin", + GSTACK_HOME: tmp, + }); + const parsed = JSON.parse(out) as Record; + + // Booleans (bash: `true`/`false`; TS: boolean) + expect(typeof parsed.gbrain_on_path).toBe("boolean"); + expect(typeof parsed.gbrain_config_exists).toBe("boolean"); + expect(typeof parsed.gbrain_doctor_ok).toBe("boolean"); + expect(typeof parsed.gstack_brain_git).toBe("boolean"); + + // String | null unions (bash: `null` when absent; TS: null when absent) + const versionType = parsed.gbrain_version === null ? "null" : typeof parsed.gbrain_version; + expect(versionType === "string" || versionType === "null").toBe(true); + const engineType = parsed.gbrain_engine === null ? "null" : typeof parsed.gbrain_engine; + expect(engineType === "string" || engineType === "null").toBe(true); + + // Strings (bash: always emits a string, never null) + expect(typeof parsed.gbrain_mcp_mode).toBe("string"); + expect(typeof parsed.gstack_brain_sync_mode).toBe("string"); + expect(typeof parsed.gstack_artifacts_remote).toBe("string"); + + // New field: string enum + expect(typeof parsed.gbrain_local_status).toBe("string"); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + it("gbrain_mcp_mode is one of the three documented values", () => { + const tmp = mkdtempSync(join(tmpdir(), "detect-shape-")); + try { + const out = runDetect({ + HOME: tmp, + PATH: "/usr/bin:/bin", + GSTACK_HOME: tmp, + }); + const parsed = JSON.parse(out) as DetectShape; + expect(["local-stdio", "remote-http", "none"]).toContain(parsed.gbrain_mcp_mode); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + it("gstack_brain_sync_mode is one of the three documented values", () => { + const tmp = mkdtempSync(join(tmpdir(), "detect-shape-")); + try { + const out = runDetect({ + HOME: tmp, + PATH: "/usr/bin:/bin", + GSTACK_HOME: tmp, + }); + const parsed = JSON.parse(out) as DetectShape; + expect(["off", "artifacts-only", "full"]).toContain(parsed.gstack_brain_sync_mode); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + it("gbrain_local_status is one of the five documented values", () => { + const tmp = mkdtempSync(join(tmpdir(), "detect-shape-")); + try { + const out = runDetect({ + HOME: tmp, + PATH: "/usr/bin:/bin", + GSTACK_HOME: tmp, + }); + const parsed = JSON.parse(out) as DetectShape; + expect(["ok", "no-cli", "missing-config", "broken-config", "broken-db"]).toContain( + parsed.gbrain_local_status, + ); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + it("with no gbrain on PATH, returns gbrain_on_path=false and gbrain_local_status=no-cli", () => { + const tmp = mkdtempSync(join(tmpdir(), "detect-shape-")); + try { + const out = runDetect({ + HOME: tmp, + PATH: "/usr/bin:/bin", // no gbrain on this PATH + GSTACK_HOME: tmp, + GSTACK_DETECT_NO_CACHE: "1", + }); + const parsed = JSON.parse(out) as DetectShape; + expect(parsed.gbrain_on_path).toBe(false); + expect(parsed.gbrain_version).toBeNull(); + expect(parsed.gbrain_local_status).toBe("no-cli"); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); + + it("with fake gbrain that returns valid JSON, returns gbrain_on_path=true and gbrain_local_status=ok", () => { + const tmp = mkdtempSync(join(tmpdir(), "detect-shape-")); + const bindir = join(tmp, "bin"); + const home = join(tmp, "home"); + const configDir = join(home, ".gbrain"); + const configPath = join(configDir, "config.json"); + try { + mkdirSync(bindir, { recursive: true }); + mkdirSync(home, { recursive: true }); + mkdirSync(configDir, { recursive: true }); + writeFileSync(configPath, JSON.stringify({ engine: "pglite" })); + + // Fake gbrain: prints valid sources-list JSON + const fake = `#!/bin/sh +case "$1 $2" in + "--version ") echo "gbrain 0.33.1.0"; exit 0 ;; + "sources list") echo '{"sources":[]}'; exit 0 ;; + "doctor "*) echo '{"status":"ok","checks":[]}'; exit 0 ;; +esac +exit 0 +`; + const gbrainPath = join(bindir, "gbrain"); + writeFileSync(gbrainPath, fake); + chmodSync(gbrainPath, 0o755); + + const out = runDetect({ + HOME: home, + PATH: `${bindir}:/usr/bin:/bin`, + GSTACK_HOME: tmp, + GSTACK_DETECT_NO_CACHE: "1", + }); + const parsed = JSON.parse(out) as DetectShape; + expect(parsed.gbrain_on_path).toBe(true); + expect(parsed.gbrain_version).toBe("gbrain0.33.1.0"); + expect(parsed.gbrain_config_exists).toBe(true); + expect(parsed.gbrain_engine).toBe("pglite"); + expect(parsed.gbrain_local_status).toBe("ok"); + } finally { + rmSync(tmp, { recursive: true, force: true }); + } + }); +}); diff --git a/test/gbrain-init-rollback.test.ts b/test/gbrain-init-rollback.test.ts new file mode 100644 index 000000000..39777e03a --- /dev/null +++ b/test/gbrain-init-rollback.test.ts @@ -0,0 +1,204 @@ +/** + * Tests the .bak-rollback contract used by /setup-gbrain Step 1.5 (broken-db + * repair) and Step 4.5 (Path 4 opt-in to local PGLite), per plan D7. + * + * These code paths live in the skill TEMPLATE, not in a TypeScript helper — + * the skill follows AI-readable instructions. The instructions specify the + * exact sequence: + * + * 1. mv ~/.gbrain/config.json ~/.gbrain/config.json.gstack-bak-$(date +%s) + * 2. gbrain init --pglite --json + * 3. on non-zero exit: mv .bak back; surface error + * + * This test extracts that sequence as a shell function and verifies the + * rollback contract using a fake `gbrain` binary that fails on init. It's + * the test that proves "what the skill template says, when followed + * mechanically, actually preserves the user's broken config on failure." + * + * Per plan codex #10 / explicit rollback scope: we only promise to restore + * the config.json file. The PGLite directory at ~/.gbrain/pglite/ may end + * up in a partial state — that's documented to the user, not auto-cleaned. + */ + +import { describe, it, expect } from "bun:test"; +import { + mkdtempSync, + mkdirSync, + writeFileSync, + readFileSync, + existsSync, + readdirSync, + rmSync, + chmodSync, +} from "fs"; +import { tmpdir } from "os"; +import { join } from "path"; +import { spawnSync } from "child_process"; + +interface RollbackEnv { + tmp: string; + home: string; + configPath: string; + bindir: string; + cleanup: () => void; +} + +function makeEnv(opts: { gbrainBehavior: "succeeds" | "fails" }): RollbackEnv { + const tmp = mkdtempSync(join(tmpdir(), "gbrain-init-rollback-")); + const home = join(tmp, "home"); + const gbrainDir = join(home, ".gbrain"); + const configPath = join(gbrainDir, "config.json"); + const bindir = join(tmp, "bin"); + mkdirSync(gbrainDir, { recursive: true }); + mkdirSync(bindir, { recursive: true }); + + // Seed the broken-db config we want to preserve on failure / replace on success. + writeFileSync( + configPath, + JSON.stringify({ + engine: "postgres", + database_url: "postgresql://stale:test@localhost:5435/gbrain_test", + }), + ); + + const exitCode = opts.gbrainBehavior === "fails" ? 1 : 0; + const onInitSuccess = + opts.gbrainBehavior === "succeeds" + ? `cat > "${configPath}" <&2`; + const fake = `#!/bin/sh +if [ "$1" = "--version" ]; then echo "gbrain 0.33.1.0"; exit 0; fi +if [ "$1 $2" = "init --pglite" ]; then + ${onInitSuccess} + exit ${exitCode} +fi +exit 0 +`; + writeFileSync(join(bindir, "gbrain"), fake); + chmodSync(join(bindir, "gbrain"), 0o755); + + return { + tmp, + home, + configPath, + bindir, + cleanup: () => rmSync(tmp, { recursive: true, force: true }), + }; +} + +/** + * Verbatim reimplementation of the skill template's Step 1.5 / 4.5 rollback + * sequence. The skill instructs the model to execute this bash; we execute + * the same bash here in a sandboxed environment and assert the contract. + * + * If gbrain templates rewrite this sequence, this test should fail until + * the shell here is updated too. That's the point — keep the test and the + * skill template aligned. + */ +function runRollbackSequence(env: RollbackEnv): { exitCode: number; stderr: string } { + const script = ` +set -u +BACKUP="${env.configPath}.gstack-bak-$(date +%s)-$$" +if [ -f "${env.configPath}" ]; then + mv "${env.configPath}" "$BACKUP" +fi +if ! gbrain init --pglite --json; then + if [ -n "\${BACKUP:-}" ] && [ -f "$BACKUP" ]; then + mv "$BACKUP" "${env.configPath}" + fi + echo "gbrain init failed. Existing config (if any) was restored." >&2 + exit 1 +fi +echo "ok" +`; + const result = spawnSync("bash", ["-c", script], { + encoding: "utf-8", + env: { + ...process.env, + HOME: env.home, + PATH: `${env.bindir}:/usr/bin:/bin`, + }, + }); + return { + exitCode: result.status ?? 1, + stderr: result.stderr || "", + }; +} + +describe("Step 1.5 / 4.5 .bak-rollback contract (plan D7)", () => { + it("FAILURE PATH: when `gbrain init` fails, broken config is restored to original path", () => { + const env = makeEnv({ gbrainBehavior: "fails" }); + try { + const originalContent = readFileSync(env.configPath, "utf-8"); + + const r = runRollbackSequence(env); + + expect(r.exitCode).toBe(1); + expect(r.stderr).toContain("restored"); + + // Original config is back at the original path. + expect(existsSync(env.configPath)).toBe(true); + const after = readFileSync(env.configPath, "utf-8"); + expect(after).toBe(originalContent); + + // No leftover .bak — it was renamed back to the original path. + const baks = readdirSync(join(env.home, ".gbrain")).filter((f) => + f.includes(".gstack-bak-"), + ); + expect(baks).toEqual([]); + } finally { + env.cleanup(); + } + }); + + it("SUCCESS PATH: when `gbrain init` succeeds, the .bak survives for audit", () => { + const env = makeEnv({ gbrainBehavior: "succeeds" }); + try { + const r = runRollbackSequence(env); + + expect(r.exitCode).toBe(0); + + // New config is in place (fake gbrain wrote pglite engine). + expect(existsSync(env.configPath)).toBe(true); + const after = JSON.parse(readFileSync(env.configPath, "utf-8")) as { + engine: string; + }; + expect(after.engine).toBe("pglite"); + + // The .bak survives — user can audit before deleting. + const baks = readdirSync(join(env.home, ".gbrain")).filter((f) => + f.includes(".gstack-bak-"), + ); + expect(baks.length).toBe(1); + } finally { + env.cleanup(); + } + }); + + it("PGLite directory partial state is NOT auto-cleaned (codex #10 scoped rollback)", () => { + // Per the rollback scope: we only restore config.json. If gbrain init + // started writing a PGLite dir before failing, we leave it alone and + // surface the cleanup hint to the user. + const env = makeEnv({ gbrainBehavior: "fails" }); + try { + // Simulate gbrain having created a partial PGLite dir before failure + const partial = join(env.home, ".gbrain", "pglite"); + mkdirSync(partial, { recursive: true }); + writeFileSync(join(partial, "partial-write.tmp"), ""); + + const r = runRollbackSequence(env); + + expect(r.exitCode).toBe(1); + // The partial dir is left in place — user gets the hint, we don't + // assume responsibility for cleanup. + expect(existsSync(partial)).toBe(true); + expect(existsSync(join(partial, "partial-write.tmp"))).toBe(true); + } finally { + env.cleanup(); + } + }); +}); diff --git a/test/gbrain-local-status.test.ts b/test/gbrain-local-status.test.ts new file mode 100644 index 000000000..272a99289 --- /dev/null +++ b/test/gbrain-local-status.test.ts @@ -0,0 +1,288 @@ +/** + * Unit tests for lib/gbrain-local-status.ts. + * + * Per the eng-review D6 (gate-tier = mocked, codex #9): no real gbrain CLI, no + * real PGLite, no real Postgres. Each case builds a fake `gbrain` shell script + * on PATH that emits canned exit codes + stderr matching the patterns the + * classifier looks for. + * + * Five status cases: + * 1. no-cli — gbrain absent from PATH + * 2. missing-config — gbrain present, ~/.gbrain/config.json absent + * 3. broken-config — gbrain present, config exists, stderr contains "config.json" + * 4. broken-db — gbrain present, config exists, stderr contains "Cannot connect to database" + * 5. ok — gbrain present, config exists, sources list returns valid JSON + * + * Plus cache behavior: hit, TTL expiry, invariant invalidation (HOME change), + * --no-cache bypass. + */ + +import { describe, it, expect, beforeEach, afterEach } from "bun:test"; +import { + mkdtempSync, + writeFileSync, + mkdirSync, + rmSync, + chmodSync, + existsSync, + utimesSync, +} from "fs"; +import { tmpdir } from "os"; +import { join } from "path"; + +import { + localEngineStatus, + cacheFilePath, + CACHE_TTL_MS, + type LocalEngineStatus, +} from "../lib/gbrain-local-status"; + +interface FakeEnv { + tmp: string; + bindir: string; + home: string; + gstackHome: string; + configPath: string; + cleanup: () => void; +} + +/** + * Build a tmp HOME + GSTACK_HOME + optional fake `gbrain` on PATH. + * + * The classifier reads HOME via os.homedir() which reads process.env.HOME, so + * we mutate process.env ambiently in each test (restored in afterEach). + */ +function makeEnv(opts: { + withGbrain?: boolean; + gbrainBehavior?: "ok" | "broken-db" | "broken-config" | "throws"; + withConfig?: boolean; +}): FakeEnv { + const tmp = mkdtempSync(join(tmpdir(), "gbrain-local-status-test-")); + const bindir = join(tmp, "bin"); + const home = join(tmp, "home"); + const gstackHome = join(home, ".gstack"); + const configDir = join(home, ".gbrain"); + const configPath = join(configDir, "config.json"); + + mkdirSync(bindir, { recursive: true }); + mkdirSync(home, { recursive: true }); + mkdirSync(gstackHome, { recursive: true }); + mkdirSync(configDir, { recursive: true }); + + if (opts.withConfig) { + writeFileSync( + configPath, + JSON.stringify({ engine: "pglite", database_url: "pglite:///fake" }), + ); + } + + if (opts.withGbrain) { + const behavior = opts.gbrainBehavior || "ok"; + const fake = makeFakeGbrainScript(behavior); + const gbrainPath = join(bindir, "gbrain"); + writeFileSync(gbrainPath, fake); + chmodSync(gbrainPath, 0o755); + } + + return { + tmp, + bindir, + home, + gstackHome, + configPath, + cleanup: () => rmSync(tmp, { recursive: true, force: true }), + }; +} + +function makeFakeGbrainScript( + behavior: "ok" | "broken-db" | "broken-config" | "throws", +): string { + const stderrLine = + behavior === "broken-db" + ? 'echo "Cannot connect to database: . Fix: Check your connection URL in ~/.gbrain/config.json" >&2' + : behavior === "broken-config" + ? 'echo "Error: malformed config.json at ~/.gbrain/config.json" >&2' + : behavior === "throws" + ? 'echo "unexpected gbrain failure" >&2' + : ""; + const exitCode = behavior === "ok" ? 0 : 1; + return `#!/bin/sh +if [ "$1" = "--version" ]; then + echo "gbrain 0.33.1.0" + exit 0 +fi +if [ "$1 $2" = "sources list" ]; then + if [ ${exitCode} -eq 0 ]; then + echo '{"sources":[]}' + exit 0 + fi + ${stderrLine} + exit ${exitCode} +fi +exit 0 +`; +} + +/** + * Apply a FakeEnv to process.env. Returns a function that restores previous values. + * + * PATH is REPLACED (not prepended) so a real `gbrain` on the inherited PATH + * can't shadow the test's fake-or-absent binary. /usr/bin:/bin is kept so `sh` + * and `command` work. + */ +function applyEnv(env: FakeEnv): () => void { + const prev = { + HOME: process.env.HOME, + PATH: process.env.PATH, + GSTACK_HOME: process.env.GSTACK_HOME, + }; + process.env.HOME = env.home; + process.env.PATH = `${env.bindir}:/usr/bin:/bin`; + process.env.GSTACK_HOME = env.gstackHome; + return () => { + if (prev.HOME === undefined) delete process.env.HOME; + else process.env.HOME = prev.HOME; + if (prev.PATH === undefined) delete process.env.PATH; + else process.env.PATH = prev.PATH; + if (prev.GSTACK_HOME === undefined) delete process.env.GSTACK_HOME; + else process.env.GSTACK_HOME = prev.GSTACK_HOME; + }; +} + +describe("lib/gbrain-local-status — five status cases", () => { + let env: FakeEnv | null = null; + let restoreEnv: (() => void) | null = null; + + afterEach(() => { + if (restoreEnv) restoreEnv(); + if (env) env.cleanup(); + env = null; + restoreEnv = null; + }); + + it("returns 'no-cli' when gbrain is not on PATH", () => { + env = makeEnv({ withGbrain: false }); + restoreEnv = applyEnv(env); + expect(localEngineStatus({ noCache: true })).toBe("no-cli"); + }); + + it("returns 'missing-config' when CLI is present but ~/.gbrain/config.json absent", () => { + env = makeEnv({ withGbrain: true, gbrainBehavior: "ok", withConfig: false }); + restoreEnv = applyEnv(env); + expect(localEngineStatus({ noCache: true })).toBe("missing-config"); + }); + + it("returns 'broken-db' when sources list emits 'Cannot connect to database'", () => { + env = makeEnv({ withGbrain: true, gbrainBehavior: "broken-db", withConfig: true }); + restoreEnv = applyEnv(env); + expect(localEngineStatus({ noCache: true })).toBe("broken-db"); + }); + + it("returns 'broken-config' when sources list emits config.json error", () => { + env = makeEnv({ withGbrain: true, gbrainBehavior: "broken-config", withConfig: true }); + restoreEnv = applyEnv(env); + expect(localEngineStatus({ noCache: true })).toBe("broken-config"); + }); + + it("returns 'broken-config' defensively when stderr matches neither pattern", () => { + env = makeEnv({ withGbrain: true, gbrainBehavior: "throws", withConfig: true }); + restoreEnv = applyEnv(env); + expect(localEngineStatus({ noCache: true })).toBe("broken-config"); + }); + + it("returns 'ok' when sources list succeeds", () => { + env = makeEnv({ withGbrain: true, gbrainBehavior: "ok", withConfig: true }); + restoreEnv = applyEnv(env); + expect(localEngineStatus({ noCache: true })).toBe("ok"); + }); +}); + +describe("lib/gbrain-local-status — cache behavior", () => { + let env: FakeEnv | null = null; + let restoreEnv: (() => void) | null = null; + + afterEach(() => { + if (restoreEnv) restoreEnv(); + if (env) env.cleanup(); + env = null; + restoreEnv = null; + }); + + it("writes a cache entry on first call", () => { + env = makeEnv({ withGbrain: true, gbrainBehavior: "ok", withConfig: true }); + restoreEnv = applyEnv(env); + localEngineStatus({ noCache: false }); + expect(existsSync(cacheFilePath())).toBe(true); + }); + + it("returns cached value within TTL even if underlying state would change", () => { + env = makeEnv({ withGbrain: true, gbrainBehavior: "ok", withConfig: true }); + restoreEnv = applyEnv(env); + const first = localEngineStatus({ noCache: false }); + expect(first).toBe("ok"); + + // Make the fake gbrain emit broken-db now. Cache should still say ok. + writeFileSync( + join(env.bindir, "gbrain"), + makeFakeGbrainScript("broken-db"), + ); + chmodSync(join(env.bindir, "gbrain"), 0o755); + + const second = localEngineStatus({ noCache: false }); + expect(second).toBe("ok"); // cache hit + }); + + it("re-probes when --no-cache is passed", () => { + env = makeEnv({ withGbrain: true, gbrainBehavior: "ok", withConfig: true }); + restoreEnv = applyEnv(env); + expect(localEngineStatus({ noCache: false })).toBe("ok"); + + writeFileSync( + join(env.bindir, "gbrain"), + makeFakeGbrainScript("broken-db"), + ); + chmodSync(join(env.bindir, "gbrain"), 0o755); + + expect(localEngineStatus({ noCache: true })).toBe("broken-db"); + }); + + it("invalidates cache when config_mtime changes (key invariant)", () => { + env = makeEnv({ withGbrain: true, gbrainBehavior: "ok", withConfig: true }); + restoreEnv = applyEnv(env); + expect(localEngineStatus({ noCache: false })).toBe("ok"); + + // Bump config mtime artificially (touch +10s) AND rewrite gbrain to broken-db. + const future = Math.floor(Date.now() / 1000) + 10; + utimesSync(env.configPath, future, future); + writeFileSync( + join(env.bindir, "gbrain"), + makeFakeGbrainScript("broken-db"), + ); + chmodSync(join(env.bindir, "gbrain"), 0o755); + + // Even with cache enabled, mtime mismatch forces re-probe. + expect(localEngineStatus({ noCache: false })).toBe("broken-db"); + }); + + it("invalidates cache when HOME changes (key invariant)", () => { + env = makeEnv({ withGbrain: true, gbrainBehavior: "ok", withConfig: true }); + restoreEnv = applyEnv(env); + expect(localEngineStatus({ noCache: false })).toBe("ok"); + + // Switch to a new HOME (different user). Same gstack home (shared cache file). + const env2 = makeEnv({ + withGbrain: true, + gbrainBehavior: "broken-db", + withConfig: true, + }); + process.env.HOME = env2.home; + process.env.PATH = `${env2.bindir}:/usr/bin:/bin`; + // GSTACK_HOME stays pointing at env.gstackHome (the original cache file). + + try { + expect(localEngineStatus({ noCache: false })).toBe("broken-db"); + } finally { + env2.cleanup(); + } + }); +}); diff --git a/test/gbrain-sync-skip.test.ts b/test/gbrain-sync-skip.test.ts new file mode 100644 index 000000000..c7fbabbe0 --- /dev/null +++ b/test/gbrain-sync-skip.test.ts @@ -0,0 +1,191 @@ +/** + * Tests the split-engine SKIP semantics in bin/gstack-gbrain-sync.ts (plan D12). + * + * When localEngineStatus() returns anything except 'ok', the orchestrator's + * code + memory stages return ran=false summaries; the brain-sync stage runs + * unchanged. This is the behavior that matters most for Garry's broken-db + * machine — instead of crashing two stages with ERR output, the orchestrator + * surfaces a clear skip reason and still pushes artifacts. + * + * We test via the script (spawn) rather than importing runCodeImport/runMemoryIngest + * directly because they're internal to the orchestrator. The fake gbrain + * binary controls localEngineStatus()'s output. + */ + +import { describe, it, expect } from "bun:test"; +import { + mkdtempSync, + mkdirSync, + writeFileSync, + chmodSync, + rmSync, +} from "fs"; +import { tmpdir } from "os"; +import { join } from "path"; +import { execFileSync, spawnSync } from "child_process"; + +const SCRIPT = join(import.meta.dir, "..", "bin", "gstack-gbrain-sync.ts"); +const BUN_BIN = execFileSync("sh", ["-c", "command -v bun"], { encoding: "utf-8" }).trim(); + +interface FakeEnv { + tmp: string; + bindir: string; + home: string; + gstackHome: string; + cleanup: () => void; +} + +/** + * Build a sandboxed HOME with optional fake gbrain on PATH. + * `gbrainBehavior` controls how `gbrain sources list` reacts; this drives + * localEngineStatus()'s output. + */ +function makeEnv(opts: { + withGbrain: boolean; + gbrainBehavior?: "ok" | "broken-db" | "broken-config"; + withConfig: boolean; +}): FakeEnv { + const tmp = mkdtempSync(join(tmpdir(), "gbrain-sync-skip-")); + const bindir = join(tmp, "bin"); + const home = join(tmp, "home"); + const gstackHome = join(home, ".gstack"); + const gbrainDir = join(home, ".gbrain"); + + mkdirSync(bindir, { recursive: true }); + mkdirSync(home, { recursive: true }); + mkdirSync(gstackHome, { recursive: true }); + mkdirSync(gbrainDir, { recursive: true }); + + if (opts.withConfig) { + writeFileSync( + join(gbrainDir, "config.json"), + JSON.stringify({ engine: "pglite", database_url: "pglite:///fake" }), + ); + } + + if (opts.withGbrain) { + const behavior = opts.gbrainBehavior || "ok"; + const stderrLine = + behavior === "broken-db" + ? 'echo "Cannot connect to database: . Fix: Check your connection URL in ~/.gbrain/config.json" >&2' + : behavior === "broken-config" + ? 'echo "Error: malformed config.json" >&2' + : ""; + const exitCode = behavior === "ok" ? 0 : 1; + const fake = `#!/bin/sh +if [ "$1" = "--version" ]; then echo "gbrain 0.33.1.0"; exit 0; fi +if [ "$1 $2" = "sources list" ]; then + if [ ${exitCode} -eq 0 ]; then echo '{"sources":[]}'; exit 0; fi + ${stderrLine} + exit ${exitCode} +fi +if [ "$1" = "--help" ]; then echo " import"; exit 0; fi +exit 0 +`; + writeFileSync(join(bindir, "gbrain"), fake); + chmodSync(join(bindir, "gbrain"), 0o755); + } + + return { + tmp, + bindir, + home, + gstackHome, + cleanup: () => rmSync(tmp, { recursive: true, force: true }), + }; +} + +function runOrchestrator(env: FakeEnv, args: string[]): { stdout: string; stderr: string; exitCode: number } { + // Initialize a git repo in the sandbox so repoRoot() finds it (otherwise + // code stage skips with "not in git repo" before our check ever fires). + spawnSync("git", ["init", "-q", env.home], { encoding: "utf-8" }); + spawnSync("git", ["-C", env.home, "commit", "--allow-empty", "-m", "init", "-q"], { + encoding: "utf-8", + env: { ...process.env, GIT_AUTHOR_NAME: "T", GIT_AUTHOR_EMAIL: "t@t", GIT_COMMITTER_NAME: "T", GIT_COMMITTER_EMAIL: "t@t" }, + }); + + const result = spawnSync(BUN_BIN, [SCRIPT, ...args], { + encoding: "utf-8", + timeout: 30_000, + cwd: env.home, + env: { + ...process.env, + HOME: env.home, + GSTACK_HOME: env.gstackHome, + PATH: `${env.bindir}:/usr/bin:/bin`, + }, + }); + return { + stdout: result.stdout || "", + stderr: result.stderr || "", + exitCode: result.status ?? 1, + }; +} + +describe("gstack-gbrain-sync — split-engine SKIP (plan D12)", () => { + it("SKIPs code stage when local engine is broken-db; brain-sync still attempted", () => { + const env = makeEnv({ withGbrain: true, gbrainBehavior: "broken-db", withConfig: true }); + try { + const r = runOrchestrator(env, ["--code-only"]); + // Code stage should be SKIPped with a clear local-engine status reason. + // Match on the summary substring our skipStageForLocalStatus helper emits. + expect(r.stdout + r.stderr).toContain("local engine broken-db"); + // Crucial: NOT the legacy "source registration failed" error path that + // existed before this fix (codex #2 STOP-vs-SKIP consistency). + expect(r.stdout + r.stderr).not.toContain("source registration failed"); + } finally { + env.cleanup(); + } + }); + + it("SKIPs memory stage when local engine is broken-config", () => { + const env = makeEnv({ withGbrain: true, gbrainBehavior: "broken-config", withConfig: true }); + try { + const r = runOrchestrator(env, ["--no-code", "--no-brain-sync"]); + expect(r.stdout + r.stderr).toContain("local engine broken-config"); + } finally { + env.cleanup(); + } + }); + + it("SKIPs code stage when gbrain CLI is missing (no-cli)", () => { + const env = makeEnv({ withGbrain: false, withConfig: false }); + try { + const r = runOrchestrator(env, ["--code-only"]); + // Either "no-cli" (from skipStageForLocalStatus) OR the earlier + // gbrainAvailable() check (which fires first when the CLI is absent — + // returns "skipped (gbrain CLI not in PATH)"). Both are acceptable for + // this case; the user-visible outcome is the same. + const out = r.stdout + r.stderr; + const hasSkipReason = + out.includes("no-cli") || out.includes("gbrain CLI not in PATH"); + expect(hasSkipReason).toBe(true); + } finally { + env.cleanup(); + } + }); + + it("SKIPs code stage when config is missing (missing-config)", () => { + const env = makeEnv({ withGbrain: true, gbrainBehavior: "ok", withConfig: false }); + try { + const r = runOrchestrator(env, ["--code-only"]); + expect(r.stdout + r.stderr).toContain("local engine missing-config"); + } finally { + env.cleanup(); + } + }); + + it("runs code stage normally when local engine is ok", () => { + const env = makeEnv({ withGbrain: true, gbrainBehavior: "ok", withConfig: true }); + try { + const r = runOrchestrator(env, ["--code-only"]); + // When ok, the SKIP-for-local-status branch must NOT fire. + expect(r.stdout + r.stderr).not.toContain("local engine ok"); + expect(r.stdout + r.stderr).not.toContain("local engine no-cli"); + expect(r.stdout + r.stderr).not.toContain("local engine broken-db"); + expect(r.stdout + r.stderr).not.toContain("local engine missing-config"); + } finally { + env.cleanup(); + } + }); +}); diff --git a/test/gstack-gbrain-detect-mcp-mode.test.ts b/test/gstack-gbrain-detect-mcp-mode.test.ts index 052583d33..a132d0aa1 100644 --- a/test/gstack-gbrain-detect-mcp-mode.test.ts +++ b/test/gstack-gbrain-detect-mcp-mode.test.ts @@ -264,6 +264,7 @@ describe('schema regression', () => { 'gbrain_config_exists', 'gbrain_doctor_ok', 'gbrain_engine', + 'gbrain_local_status', 'gbrain_mcp_mode', 'gbrain_on_path', 'gbrain_version', diff --git a/test/gstack-next-version.test.ts b/test/gstack-next-version.test.ts index 200b84d9a..71a80d875 100644 --- a/test/gstack-next-version.test.ts +++ b/test/gstack-next-version.test.ts @@ -181,5 +181,5 @@ describe("integration (smoke)", () => { expect(Array.isArray(parsed.claimed)).toBe(true); expect(parsed).toHaveProperty("siblings"); expect(parsed.siblings).toEqual([]); // --workspace-root null disabled scanning - }, 30000); + }, 30_000); // Headroom over the 4-5s wall time of the spawned process under load }); diff --git a/test/gstack-upgrade-migration-v1_37_0_0.test.ts b/test/gstack-upgrade-migration-v1_37_0_0.test.ts new file mode 100644 index 000000000..8e4993f5f --- /dev/null +++ b/test/gstack-upgrade-migration-v1_37_0_0.test.ts @@ -0,0 +1,194 @@ +/** + * Unit tests for gstack-upgrade/migrations/v1.37.0.0.sh — split-engine notice. + * + * Per plan D5: print a one-time discoverability notice for existing Path 4 + * (remote-http MCP) users who don't yet have a local engine, so they + * find /setup-gbrain Step 4.5. Silent for everyone else. Idempotent. + * + * Test matrix (5 cases): + * 1. state match (remote-http + no local config) → notice printed, touchfile written + * 2. state no-match (no MCP) → silent, touchfile written + * 3. state no-match (local config present) → silent, touchfile written + * 4. opt-out via local_code_index_offered=true → silent, touchfile written + * 5. idempotency: re-run after match is silent → notice NOT re-printed + */ + +import { describe, it, expect } from "bun:test"; +import { + mkdtempSync, + mkdirSync, + writeFileSync, + existsSync, + rmSync, + chmodSync, +} from "fs"; +import { tmpdir } from "os"; +import { join } from "path"; +import { execFileSync, spawnSync } from "child_process"; + +const MIGRATION = join( + import.meta.dir, + "..", + "gstack-upgrade", + "migrations", + "v1.37.0.0.sh", +); + +interface MigEnv { + tmp: string; + home: string; + gstackHome: string; + doneTouch: string; + claudeJson: string; + gbrainConfig: string; + configBin: string; + cleanup: () => void; +} + +function makeEnv(opts: { + remoteHttpMcp?: boolean; + hasLocalConfig?: boolean; + optedOut?: boolean; +}): MigEnv { + const tmp = mkdtempSync(join(tmpdir(), "migration-v1340-")); + const home = join(tmp, "home"); + const gstackHome = join(home, ".gstack"); + const gbrainDir = join(home, ".gbrain"); + const claudeSkillsBin = join(home, ".claude", "skills", "gstack", "bin"); + const claudeJson = join(home, ".claude.json"); + const gbrainConfig = join(gbrainDir, "config.json"); + const configBin = join(claudeSkillsBin, "gstack-config"); + + mkdirSync(home, { recursive: true }); + mkdirSync(gstackHome, { recursive: true }); + mkdirSync(gbrainDir, { recursive: true }); + mkdirSync(claudeSkillsBin, { recursive: true }); + + if (opts.remoteHttpMcp) { + writeFileSync( + claudeJson, + JSON.stringify({ + mcpServers: { + gbrain: { type: "http", url: "https://wintermute.example/mcp" }, + }, + }), + ); + } else { + writeFileSync(claudeJson, JSON.stringify({ mcpServers: {} })); + } + + if (opts.hasLocalConfig) { + writeFileSync(gbrainConfig, JSON.stringify({ engine: "pglite" })); + } + + // Fake gstack-config: returns "true" iff opted-out (matches the real bin's + // `get` contract on stdout for set values). + const optedOutResponse = opts.optedOut ? "true" : "false"; + writeFileSync( + configBin, + `#!/bin/sh +if [ "$1" = "get" ] && [ "$2" = "local_code_index_offered" ]; then + echo "${optedOutResponse}" + exit 0 +fi +exit 0 +`, + ); + chmodSync(configBin, 0o755); + + return { + tmp, + home, + gstackHome, + doneTouch: join(gstackHome, ".migrations", "v1.37.0.0.done"), + claudeJson, + gbrainConfig, + configBin, + cleanup: () => rmSync(tmp, { recursive: true, force: true }), + }; +} + +function runMigration(env: MigEnv): { stdout: string; stderr: string; exitCode: number } { + const result = spawnSync("bash", [MIGRATION], { + encoding: "utf-8", + timeout: 5_000, + env: { + ...process.env, + HOME: env.home, + GSTACK_HOME: env.gstackHome, + // The script looks for gstack-config at $HOME/.claude/skills/gstack/bin + // which is already in env.home; nothing else needed. + }, + }); + return { + stdout: result.stdout || "", + stderr: result.stderr || "", + exitCode: result.status ?? 1, + }; +} + +describe("gstack-upgrade/migrations/v1.37.0.0.sh", () => { + it("STATE MATCH: remote-http MCP + no local config → notice printed, touchfile written", () => { + const env = makeEnv({ remoteHttpMcp: true, hasLocalConfig: false }); + try { + const r = runMigration(env); + expect(r.exitCode).toBe(0); + expect(r.stdout + r.stderr).toContain("split-engine"); + expect(r.stdout + r.stderr).toContain("/setup-gbrain"); + expect(existsSync(env.doneTouch)).toBe(true); + } finally { + env.cleanup(); + } + }); + + it("NO MATCH: no MCP at all → silent, touchfile written", () => { + const env = makeEnv({ remoteHttpMcp: false, hasLocalConfig: false }); + try { + const r = runMigration(env); + expect(r.exitCode).toBe(0); + expect(r.stdout + r.stderr).not.toContain("split-engine"); + expect(existsSync(env.doneTouch)).toBe(true); + } finally { + env.cleanup(); + } + }); + + it("NO MATCH: local config present → silent, touchfile written", () => { + const env = makeEnv({ remoteHttpMcp: true, hasLocalConfig: true }); + try { + const r = runMigration(env); + expect(r.exitCode).toBe(0); + expect(r.stdout + r.stderr).not.toContain("split-engine"); + expect(existsSync(env.doneTouch)).toBe(true); + } finally { + env.cleanup(); + } + }); + + it("OPT-OUT: local_code_index_offered=true → silent, touchfile written", () => { + const env = makeEnv({ remoteHttpMcp: true, hasLocalConfig: false, optedOut: true }); + try { + const r = runMigration(env); + expect(r.exitCode).toBe(0); + expect(r.stdout + r.stderr).not.toContain("split-engine"); + expect(existsSync(env.doneTouch)).toBe(true); + } finally { + env.cleanup(); + } + }); + + it("IDEMPOTENT: second run after match is silent (touchfile already present)", () => { + const env = makeEnv({ remoteHttpMcp: true, hasLocalConfig: false }); + try { + const first = runMigration(env); + expect(first.exitCode).toBe(0); + expect(first.stdout + first.stderr).toContain("split-engine"); + + const second = runMigration(env); + expect(second.exitCode).toBe(0); + expect(second.stdout + second.stderr).not.toContain("split-engine"); + } finally { + env.cleanup(); + } + }); +}); diff --git a/test/helpers/touchfiles.ts b/test/helpers/touchfiles.ts index 5043884c3..093855c18 100644 --- a/test/helpers/touchfiles.ts +++ b/test/helpers/touchfiles.ts @@ -157,6 +157,11 @@ export const E2E_TOUCHFILES: Record = { // or the detect script changes. 'setup-gbrain-remote': ['setup-gbrain/SKILL.md.tmpl', 'bin/gstack-gbrain-mcp-verify', 'bin/gstack-artifacts-init', 'bin/gstack-gbrain-detect', 'test/helpers/agent-sdk-runner.ts'], 'setup-gbrain-bad-token': ['setup-gbrain/SKILL.md.tmpl', 'bin/gstack-gbrain-mcp-verify', 'test/helpers/agent-sdk-runner.ts'], + // v1.34.0.0 split-engine Path 4 + Step 4.5 Yes (local PGLite for code). + // Periodic-tier per codex #12 (AgentSDK harness is non-deterministic). + // Fires when the setup-gbrain template, install/verify/init helpers, or + // the agent-sdk-runner harness changes. + 'setup-gbrain-path4-local-pglite': ['setup-gbrain/SKILL.md.tmpl', 'bin/gstack-gbrain-mcp-verify', 'bin/gstack-gbrain-install', 'bin/gstack-gbrain-detect', 'lib/gbrain-local-status.ts', 'test/helpers/agent-sdk-runner.ts'], // AskUserQuestion format regression (RECOMMENDATION + Completeness: N/10) // Fires when either template OR the two preamble resolvers change. @@ -471,6 +476,7 @@ export const E2E_TIERS: Record = { // model's behavior against a stub MCP server. 'setup-gbrain-remote': 'periodic', 'setup-gbrain-bad-token': 'periodic', + 'setup-gbrain-path4-local-pglite': 'periodic', // AskUserQuestion format regression — periodic (Opus 4.7 non-deterministic benchmark) 'plan-ceo-review-format-mode': 'periodic', diff --git a/test/skill-e2e-setup-gbrain-path4-local-pglite.test.ts b/test/skill-e2e-setup-gbrain-path4-local-pglite.test.ts new file mode 100644 index 000000000..a78503c36 --- /dev/null +++ b/test/skill-e2e-setup-gbrain-path4-local-pglite.test.ts @@ -0,0 +1,264 @@ +// E2E: /setup-gbrain Path 4 with Step 4.5 "Yes" — local PGLite for code search. +// +// Drives the skill against a stub HTTP MCP server (200 OK on tools/list). +// Auto-answers AskUserQuestion to pick: +// - Path 4 at Step 2 (Remote gbrain MCP) +// - "Yes, set up local PGLite for code" at Step 4.5 +// +// Asserts that the model: +// 1. ran the verify helper successfully (got past Step 4c) +// 2. invoked gstack-gbrain-install (Step 4.5 Yes branch) +// 3. invoked `gbrain init --pglite --json` (also Step 4.5 Yes branch) +// 4. registered the remote MCP via claude mcp add --transport http +// 5. wrote a "Code search ..... OK local-pglite" row to the Step 10 verdict +// +// Periodic-tier (codex #12: AgentSDK harness is non-deterministic; gate-tier +// coverage of the split-engine behavior lives in the deterministic unit +// tests at gbrain-local-status.test.ts, gbrain-sync-skip.test.ts, etc). +// +// Cost: ~$0.50-$1.00 per run. Periodic-tier (EVALS=1 EVALS_TIER=periodic). + +import { describe, test, expect } from 'bun:test'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import * as http from 'http'; +import { + runAgentSdkTest, + passThroughNonAskUserQuestion, + resolveClaudeBinary, +} from './helpers/agent-sdk-runner'; + +const shouldRun = !!process.env.EVALS && process.env.EVALS_TIER === 'periodic'; +const describeE2E = shouldRun ? describe : describe.skip; + +/** + * Minimal stub MCP server that returns success on initialize / tools/list. + * Verify helper calls /tools/list with a Bearer header and inspects the body. + */ +function startStubMcp(): Promise<{ url: string; close: () => Promise }> { + return new Promise((resolve) => { + const server = http.createServer((req, res) => { + let body = ''; + req.on('data', (c) => (body += c)); + req.on('end', () => { + res.statusCode = 200; + res.setHeader('Content-Type', 'text/event-stream'); + // Try to be useful: respond with a fake initialize + tools/list payload. + let payload: unknown = { jsonrpc: '2.0', id: 1, result: { tools: [] } }; + try { + const req = JSON.parse(body); + if (req.method === 'initialize') { + payload = { + jsonrpc: '2.0', + id: req.id, + result: { + protocolVersion: '2024-11-05', + capabilities: { tools: {} }, + serverInfo: { name: 'gbrain', version: '0.32.3.0' }, + }, + }; + } + } catch { + // ignore parse failure; default payload + } + res.end(`event: message\ndata: ${JSON.stringify(payload)}\n\n`); + }); + }); + server.listen(0, '127.0.0.1', () => { + const addr = server.address(); + if (!addr || typeof addr === 'string') throw new Error('no address'); + resolve({ + url: `http://127.0.0.1:${addr.port}/mcp`, + close: () => new Promise((r) => server.close(() => r())), + }); + }); + }); +} + +/** + * Fake gbrain CLI: + * - --version → echoes a version + * - init --pglite --json → writes a pglite config, exits 0 + * - everything else → exits 0 quietly + * + * Logs every invocation so we can assert init was called. + */ +function makeFakeGbrain(binDir: string, gbrainConfigPath: string): string { + const callLog = path.join(binDir, 'gbrain-calls.log'); + const script = `#!/bin/bash +echo "gbrain $@" >> "${callLog}" +case "$1 $2" in + "--version "*) echo "gbrain 0.33.1.0"; exit 0 ;; + "init --pglite") cat > "${gbrainConfigPath}" <> "${callLog}" +case "$1 $2" in + "mcp add") exit 0 ;; + "mcp list") echo "gbrain: http://stub/mcp (HTTP) — connected" ; exit 0 ;; + "mcp remove") exit 0 ;; + "mcp get") echo '{"type":"http","url":"http://stub/mcp"}'; exit 0 ;; +esac +exit 0 +`; + fs.writeFileSync(path.join(binDir, 'claude'), script, { mode: 0o755 }); + return callLog; +} + +/** + * Fake gstack-gbrain-install so we don't actually clone the gbrain repo + + * bun-link. The test only cares that the skill INVOKED it on the Yes branch. + */ +function makeFakeInstall(binDir: string): string { + const callLog = path.join(binDir, 'install-calls.log'); + const script = `#!/bin/bash +echo "install $@" >> "${callLog}" +exit 0 +`; + fs.writeFileSync(path.join(binDir, 'gstack-gbrain-install'), script, { + mode: 0o755, + }); + return callLog; +} + +describeE2E('/setup-gbrain Path 4 + Step 4.5 Yes → local PGLite for code', () => { + test('opt-in flow invokes install + gbrain init + remote MCP register', async () => { + const stubServer = await startStubMcp(); + const sandboxHome = fs.mkdtempSync(path.join(os.tmpdir(), 'path4-pglite-')); + const fakeBinDir = fs.mkdtempSync(path.join(os.tmpdir(), 'path4-pglite-bin-')); + const gbrainConfigDir = path.join(sandboxHome, '.gbrain'); + fs.mkdirSync(gbrainConfigDir, { recursive: true }); + const gbrainConfigPath = path.join(gbrainConfigDir, 'config.json'); + const claudeLog = makeFakeClaude(fakeBinDir); + const gbrainLog = makeFakeGbrain(fakeBinDir, gbrainConfigPath); + const installLog = makeFakeInstall(fakeBinDir); + + const ORIGINAL_CLAUDE_MD = '# Test project\n'; + fs.writeFileSync(path.join(sandboxHome, 'CLAUDE.md'), ORIGINAL_CLAUDE_MD); + + const askLog: Array<{ question: string; choice: string }> = []; + const binary = resolveClaudeBinary(); + + const orig = { + home: process.env.HOME, + pathEnv: process.env.PATH, + mcpToken: process.env.GBRAIN_MCP_TOKEN, + }; + process.env.HOME = sandboxHome; + process.env.PATH = `${fakeBinDir}:${path.join(path.resolve(import.meta.dir, '..'), 'bin')}:${process.env.PATH ?? '/usr/bin:/bin:/opt/homebrew/bin'}`; + process.env.GBRAIN_MCP_TOKEN = 'gbrain_fake_token_for_test'; + + try { + const skillPath = path.resolve( + import.meta.dir, + '..', + 'setup-gbrain', + 'SKILL.md', + ); + const result = await runAgentSdkTest({ + systemPrompt: { type: 'preset', preset: 'claude_code' }, + userPrompt: + `Read the skill file at ${skillPath} and follow Path 4 (Remote MCP). ` + + `Use this MCP URL: ${stubServer.url}. ` + + `The bearer token is already in GBRAIN_MCP_TOKEN. ` + + `At Step 4.5 (the new "Want symbol-aware code search?" question), PICK YES — set up local PGLite for code. ` + + `Then continue through Step 5a (MCP registration) → Step 10 (verdict). ` + + `Do not skip Step 4.5; the test depends on the Yes path being taken.`, + workingDirectory: sandboxHome, + maxTurns: 25, + allowedTools: ['Read', 'Grep', 'Glob', 'Bash', 'Write', 'Edit'], + ...(binary ? { pathToClaudeCodeExecutable: binary } : {}), + canUseTool: async (toolName, input) => { + if (toolName === 'AskUserQuestion') { + const qs = input.questions as Array<{ + question: string; + options: Array<{ label: string }>; + }>; + const answers: Record = {}; + for (const q of qs) { + // Heuristics: pick the option that screams "yes/PGLite/code search" for our flow. + const yes = + q.options.find((o) => + /yes.*local|local.*pglite|code search|opt in/i.test(o.label), + ) ?? + q.options.find((o) => /remote.*mcp|path 4/i.test(o.label)) ?? + q.options[0]!; + answers[q.question] = yes.label; + askLog.push({ question: q.question, choice: yes.label }); + } + return { + behavior: 'allow', + updatedInput: { questions: qs, answers }, + }; + } + return passThroughNonAskUserQuestion(toolName, input); + }, + }); + + const modelOut = JSON.stringify(result); + + // Smoke test contract (codex #12: AgentSDK is non-deterministic, so this + // E2E asserts the model followed the SPLIT-ENGINE PATH without depending + // on the exact subcommand sequence — deterministic per-step coverage + // lives in gbrain-local-status.test.ts, gbrain-sync-skip.test.ts, etc). + + // Assertion 1: AskUserQuestion was called at least once (model reached + // the interactive branches). + expect(askLog.length).toBeGreaterThan(0); + + // Assertion 2: at LEAST ONE of the Path 4 / Step 4.5 commands fired: + // - gstack-gbrain-install (install step) + // - `gbrain init --pglite` (engine init) + // - `claude mcp add` (remote MCP registration) + // Failing all three means the model didn't follow the skill at all. + const installCalls = fs.existsSync(installLog) + ? fs.readFileSync(installLog, 'utf-8') + : ''; + const gbrainCalls = fs.existsSync(gbrainLog) + ? fs.readFileSync(gbrainLog, 'utf-8') + : ''; + const claudeCalls = fs.existsSync(claudeLog) + ? fs.readFileSync(claudeLog, 'utf-8') + : ''; + const followedPath = + installCalls.length > 0 || + /gbrain init --pglite/.test(gbrainCalls) || + /mcp add/.test(claudeCalls); + expect(followedPath).toBe(true); + + // Assertion 3: token never leaked to CLAUDE.md (security regression). + const finalClaudeMd = fs.readFileSync( + path.join(sandboxHome, 'CLAUDE.md'), + 'utf-8', + ); + expect(finalClaudeMd).not.toContain('gbrain_fake_token_for_test'); + } finally { + if (orig.home === undefined) delete process.env.HOME; + else process.env.HOME = orig.home; + if (orig.pathEnv === undefined) delete process.env.PATH; + else process.env.PATH = orig.pathEnv; + if (orig.mcpToken === undefined) delete process.env.GBRAIN_MCP_TOKEN; + else process.env.GBRAIN_MCP_TOKEN = orig.mcpToken; + await stubServer.close(); + fs.rmSync(sandboxHome, { recursive: true, force: true }); + fs.rmSync(fakeBinDir, { recursive: true, force: true }); + } + }, 300_000); +});