mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-18 10:31:30 +08:00
* fix(gbrain-sync): fold hostname into code-source id hash + migration (#1414) Cherry-picked from #1468 by 0xDevNinja and extended with the hostname-fold migration that codex review surfaced. Pre-fix `deriveCodeSourceId` hashed the absolute repo path alone, so two machines with identical home-dir layouts (chezmoi-managed dotfiles, ansible-provisioned VMs) derived the same id and clobbered each other's `local_path` in a federated brain. Last-writer-wins, with cryptic "Not a git repository" errors on the loser. Hash key is now `\${hostname}::\${path}`. Conductor worktrees on a single host stay distinct (path entropy unchanged within a host); cross-machine federations stop colliding. Migration (D1=B + codex refinements): every existing user has a pre-#1468 path-only-hash source id in their brain that no longer matches what `deriveCodeSourceId` produces. Without migration, the next sync registers a fresh source and orphans the old one. This commit adds: - \`derivePathOnlyHashLegacyId\` — separate helper for the pre-#1468 form. Distinct from \`deriveLegacyCodeSourceId\` (pre-pathhash v1.x form); both probes run. - \`planHostnameFoldMigration\` — feature-checks \`gbrain sources rename <old> <new>\` (exact argument shape, not just \`--help\`), gates on path-drift (skip migration if old source's \`local_path\` differs from current repo root), and falls back to register-new + sync-OK + remove-old when rename is unsupported. As of gbrain 0.35.0.0 the rename subcommand does not exist, so users go through the cleanup path; the rename path stays dormant until gbrain ships it. - \`removeOrphanedSource\` — called only AFTER new-source sync verifies page_count > 0. Closes the data-loss window codex flagged where "register new, remove old before sync" can wipe pages if sync fails. - \`sourceLocalPath\` — looks up a source's \`local_path\` from \`gbrain sources list --json\` for the drift gate. - Helpers accept an optional \`env\` parameter so tests can inject a gbrain shim via PATH without process-wide PATH mutation (Bun's spawnSync doesn't pick up runtime PATH changes). Pre-positions for commit 4's centralized gbrain-exec helper. - \`if (import.meta.main)\` guard around \`main()\` so the helpers can be imported for in-process unit tests. Tests cover: pure derivation, ids-match degenerate case, no-legacy short-circuit, path-drift skip path, rename path with shim, cleanup fallback when rename unsupported, cleanup fallback when rename call itself fails, source-lookup happy/missing/error paths. \`GSTACK_HOSTNAME\` env var is a test-only knob; production uses \`os.hostname()\`. Fixes #1414 Co-Authored-By: Claude <noreply@anthropic.com> * fix(gbrain-sync): cut source-id slugs on hyphen boundaries (+ #1357) Cherry-picked from #1481 by drummerms and extended with the explicit HTTPS-remote regression case for #1357 (decision D2=A). `constrainSourceId` truncated the slug with `slug.slice(-tailBudget)`, which cut mid-word when the boundary fell inside a token. For a repo where the combined `prefix-org-repo-pathhash` exceeded 32 chars, this produced embarrassing artifacts like `gstack-code-kill-270c0001-c32152` (from `drummerms-av-sow-wiz-skill-270c0001`). Two changes carried from #1481, adapted for the #1468 hostpathhash: 1. `constrainSourceId` now walks hyphen-separated tokens from the right, accumulating whole tokens until adding the next would exceed `tailBudget`. When no token fits, falls through to the existing `${prefix}-${hash}` form. 2. `deriveCodeSourceId` now retries with `repo-only-hostpathhash` (dropping the org segment) when the full `org-repo-hostpathhash` triggers truncation. Keeps the repo name readable when it fits at all. Plus a new test asserting the source id is period-free for the exact HTTPS-with-.git remote shape from #1357 (`https://github.com/foo/bar.git`). canonicalizeRemote strips `.git`; the sanitizer strips any residual non-alnum. The test closes #1357 by pinning the property. Closes #1357 Co-Authored-By: Claude <noreply@anthropic.com> * fix(gbrain): probe CLI without command builtin * fix(gbrain-sync): centralize gbrain spawn surface + seed DATABASE_URL Cherry-picked from #1508 by jasshultz, restructured per codex review #4 and #7 to widen scope and centralize the spawn surface. The bug: gbrain auto-loads .env.local from cwd via dotenv. When /sync-gbrain runs inside a Next.js / Prisma / Rails project whose .env.local defines its own DATABASE_URL (pointing at the app's local DB), gbrain reads that value instead of its own ~/.gbrain/config.json — auth fails, code + memory stages crash. This commit: - Adds lib/gbrain-exec.ts: buildGbrainEnv, spawnGbrain, execGbrainJson, execGbrainText, spawnGbrainAsync (the last one for memory-ingest's streaming gbrain import call). buildGbrainEnv seeds DATABASE_URL from ${GBRAIN_HOME:-$HOME/.gbrain}/config.json, returns a fresh env object (never the caller's by identity — codex review #11), and honors the GSTACK_RESPECT_ENV_DATABASE_URL=1 escape hatch. - Routes every gbrain spawn in bin/gstack-gbrain-sync.ts and bin/gstack-memory-ingest.ts through the helpers. Both files now own zero direct spawnSync("gbrain"|spawn("gbrain"|execFileSync("gbrain" call sites. - Threads buildGbrainEnv into the spawnSync("bun", [memory-ingest], ...) grandchild in runMemoryIngest (codex review #7). Without this, the parent fix is half-baked — the bun child inherits a clean env but needs DATABASE_URL pre-seeded too. spawnGbrainAsync inside memory-ingest provides defense in depth for standalone invocations. - Adds GBRAIN_HOME support — aligns with detectEngineTier (already honors GBRAIN_HOME) so all gstack-side gbrain calls agree on which config file matters. Resolves baseEnv.HOME first, then homedir(), so test injection works without process-wide HOME mutation. - Adds test/build-gbrain-env.test.ts: 10 unit tests covering all five env-seeding branches (seed from config / override caller / GSTACK_RESPECT escape hatch / missing config / unparseable config / no database_url field / GBRAIN_HOME path / object-identity guard / unrelated-vars preservation / idempotent-when-matches). - Adds test/gbrain-exec-invariant.test.ts: static-source check that greps both bin/gstack-gbrain-sync.ts and bin/gstack-memory-ingest.ts for direct spawnSync("gbrain"|spawn("gbrain"|execFileSync("gbrain"| execSync(...gbrain matches and fails the build if any are found. Refactor-proof against future contributors adding a new gbrain spawn without env threading. The invariant is intentionally narrow — only the two files where the DATABASE_URL bug actually hurts users are guarded. Migrating the spawn sites in lib/gbrain-local-status.ts, lib/gstack-memory-helpers.ts, and bin/gstack-brain-context-load.ts is a follow-up. Co-Authored-By: Jason Shultz <jasshultz@gmail.com> Co-Authored-By: Claude <noreply@anthropic.com> * fix(gbrain-sync): add .gbrain-source to consumer repo .gitignore (#1384) The v1.29.0.0 changelog promised .gbrain-source would be added to the consuming repo's .gitignore so the per-worktree pin stays local, but the change actually only added it to gstack's own .gitignore. Without the consumer-side entry, the pin gets committed and Conductor sibling worktrees of the same repo + branch step on each other's pin every time anyone commits. Add ensureGbrainSourceGitignored after a successful gbrain sources attach in runCodeImport. Idempotent on repeat runs (line-trim match), creates .gitignore if missing, logs a warning and continues on permission errors so a read-only checkout doesn't fail the sync. Gate the top-level main() call behind import.meta.main so tests can import the helper without triggering a full sync run on module load. Tests in test/gbrain-source-gitignore.test.ts cover: create-when-missing, append-without-trailing-newline, append-with-trailing-newline, idempotent on repeat, recognize whitespace-surrounded entry, no-throw on read-only file. 6 pass. * fix(gbrain-sources): bump gbrain sources list --json timeout 10s → 30s Supabase free-tier cold-starts can push `gbrain sources list --json` past 10s (observed 14.5s in the wild), causing probeSource() to throw ETIMEDOUT during /sync-gbrain code stage even though the underlying CLI was healthy. Matches the 30s ceiling already used by `sources add` / `sources remove` in the same file. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(brain-allowlist): sync project-root eng-review-test-plan artifacts (#1452) Cherry-picked from #1465 by genisis0x and extended with the v1.40.0.0 upgrade migration that codex review #5 surfaced. #1465 alone only patches bin/gstack-artifacts-init, which means fresh installs and re-inits pick up the new pattern. But existing users who already ran v1.38.1.0 have a `.migrations/v1.38.1.0.done` marker — that migration won't re-run no matter what we change. So their installed `.brain-allowlist`, `.brain-privacy-map.json`, and `.gitattributes` stay without the new pattern, and `/plan-eng-review` artifacts continue to silently drop out of their federation queue. This commit: - bin/gstack-artifacts-init: adds projects/*/*-eng-review-test-plan-*.md to the three managed blocks. v1.38.1.0 covered design + test-plan; this completes the set for /plan-eng-review. - gstack-upgrade/migrations/v1.40.0.0.sh: targeted in-place repair for existing installs. Same idempotent jq-based shape as v1.38.1.0. Adds the new pattern to .brain-allowlist (before the USER ADDITIONS marker), .brain-privacy-map.json (as class=artifact), and .gitattributes (as merge=union). NEVER commits + pushes — the user controls when the patches ship to their federated artifacts repo. - test/artifacts-init-migration.test.ts: 5 new tests covering the v1.40.0.0 migration applied on top of a post-v1.38.1.0 state, jq patching, gitattributes append, idempotent re-run, and done-marker write when files are missing entirely. Co-Authored-By: Claude <noreply@anthropic.com> * fix(gbrain-install): skip postinstall on Windows MSYS/MINGW + post-install probe Cherry-picked from #1487 by genisis0x and extended with the post-install subcommand probe per T6 / codex review #19. `bun install` in $INSTALL_DIR fails on Windows MSYS/MINGW/Cygwin shells because gbrain's native postinstall script mis-parses path arguments and aborts with a non-zero exit, breaking gstack-gbrain-install for Windows users running git-bash/MSYS2. The package installs cleanly without scripts. This commit: - Adds Windows shell detection via `uname -s` matching MINGW*/MSYS*/CYGWIN*/Windows_NT (#1487's case statement already covers all four — codex review #18 confirmed MINGW* is included). Windows paths get `bun install --ignore-scripts`; macOS and Linux unchanged. - Adds a post-install probe of `gbrain sources --help`. `gbrain --version` already runs (D19 PATH-shadowing validation), but version success doesn't prove the subcommand surface is reachable — and `--ignore-scripts` may have skipped artifacts that subcommands need. Probe failure logs a clear warning (with Windows-specific remediation pointing at re-running `bun install` outside MSYS) but does NOT exit non-zero; users may still get value from gbrain even if the probe fails transiently. Refs #1271 Co-Authored-By: Claude <noreply@anthropic.com> * chore: v1.40.0.0 — gbrain sync hardening wave Bumps VERSION 1.39.2.0 → 1.40.0.0 (MINOR — substantial gbrain capability hardening across sync pipeline, install path, federation allowlist; ~600 net LOC added across 8 community PRs + plan-review refinements). CHANGELOG entry follows the release-summary format: two-line headline, lead paragraph, "numbers that matter" with before/after table across 8 user-visible surfaces, "what this means for builders" closer, itemized Added/Changed/Fixed/NOT fixed/For contributors sections. Per-commit contributor credits: 0xDevNinja, drummerms, Jayesh Betala, Jason Shultz, genisis0x. Also names NikhileshNanduri and realcarsonterry in the wave's "Fixed" section for independent submissions of the .gbrain-source gitignore bug. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: 0xDevNinja <manmit0x@gmail.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: drummerms <mike@av2o.com> Co-authored-by: Jayesh Betala <jayesh.betala7@gmail.com> Co-authored-by: Jason Shultz <jasshultz@gmail.com> Co-authored-by: genisis0x <manietdavv@gmail.com>
269 lines
8.3 KiB
TypeScript
269 lines
8.3 KiB
TypeScript
/**
|
|
* gbrain-local-status — classify the local gbrain engine into 5 states.
|
|
*
|
|
* Shared between bin/gstack-gbrain-detect (preamble probe on every skill start)
|
|
* and bin/gstack-gbrain-sync.ts (orchestrator SKIP-when-not-ok semantics).
|
|
* Single source of truth: same probe, same classification, same cache.
|
|
*
|
|
* Per the split-engine plan (D2 + D8):
|
|
* - Probe: `gbrain sources list --json`. Cheap (~80ms), actually hits the DB.
|
|
* Uses the same stderr patterns as lib/gbrain-sources.ts:66-67.
|
|
* - Cache: 60s TTL at ~/.gstack/.gbrain-local-status-cache.json, keyed on
|
|
* {home, path_hash, gbrain_bin_path, gbrain_version, config_mtime}.
|
|
* - --no-cache bypass: /setup-gbrain and /sync-gbrain pass it after any
|
|
* state-mutating operation so the next read sees fresh status.
|
|
*
|
|
* No-cli → gbrain not on PATH.
|
|
* Missing → CLI present, ~/.gbrain/config.json absent.
|
|
* Broken-config → config exists but `gbrain sources list` fails with config parse error
|
|
* (or any non-recognized error — defensive default per codex #8).
|
|
* Broken-db → config exists, DB unreachable per stderr classification.
|
|
* Ok → DB reachable, sources list returned valid JSON.
|
|
*/
|
|
|
|
import { execFileSync } from "child_process";
|
|
import {
|
|
createHash,
|
|
} from "crypto";
|
|
import {
|
|
existsSync,
|
|
mkdirSync,
|
|
readFileSync,
|
|
renameSync,
|
|
statSync,
|
|
writeFileSync,
|
|
} from "fs";
|
|
import { homedir } from "os";
|
|
import { dirname, join } from "path";
|
|
|
|
export type LocalEngineStatus =
|
|
| "ok"
|
|
| "no-cli"
|
|
| "missing-config"
|
|
| "broken-config"
|
|
| "broken-db";
|
|
|
|
export interface ClassifyOptions {
|
|
/** Bypass the 60s cache. Used after any state-mutating operation. */
|
|
noCache?: boolean;
|
|
/** Env override for the spawned `gbrain` (used by tests to point at a fake binary). */
|
|
env?: NodeJS.ProcessEnv;
|
|
}
|
|
|
|
interface CacheEntry {
|
|
schema_version: 1;
|
|
status: LocalEngineStatus;
|
|
cached_at: number;
|
|
/** Cache invariants — entry is invalidated if any of these change between writes. */
|
|
key: {
|
|
home: string;
|
|
path_hash: string;
|
|
gbrain_bin_path: string;
|
|
gbrain_version: string;
|
|
config_mtime: number; // 0 when config absent
|
|
config_size: number; // 0 when config absent
|
|
};
|
|
}
|
|
|
|
export const CACHE_TTL_MS = 60_000;
|
|
export const PROBE_TIMEOUT_MS = 5_000;
|
|
|
|
/** Effective user home — respects HOME env override (used by tests). */
|
|
function userHome(): string {
|
|
return process.env.HOME || homedir();
|
|
}
|
|
|
|
/** Cache path computed fresh on each call so tests can mutate GSTACK_HOME per case. */
|
|
export function cacheFilePath(): string {
|
|
return join(
|
|
process.env.GSTACK_HOME || join(userHome(), ".gstack"),
|
|
".gbrain-local-status-cache.json",
|
|
);
|
|
}
|
|
|
|
function gbrainConfigPath(): string {
|
|
return join(userHome(), ".gbrain", "config.json");
|
|
}
|
|
|
|
function hashPath(p: string): string {
|
|
return createHash("sha256").update(p).digest("hex").slice(0, 16);
|
|
}
|
|
|
|
/**
|
|
* Resolve the absolute path of `gbrain` on PATH. Returns null when missing.
|
|
* Memoized per-process keyed on PATH so detect's call and the classifier's
|
|
* call share one fork-exec (~200ms saved per skill preamble).
|
|
*/
|
|
const _gbrainBinCache = new Map<string, string | null>();
|
|
export function resolveGbrainBin(env?: NodeJS.ProcessEnv): string | null {
|
|
const e = env ?? process.env;
|
|
const key = e.PATH || "";
|
|
if (_gbrainBinCache.has(key)) return _gbrainBinCache.get(key)!;
|
|
let result: string | null = null;
|
|
try {
|
|
execFileSync("gbrain", ["--version"], {
|
|
encoding: "utf-8",
|
|
timeout: 2_000,
|
|
stdio: ["ignore", "ignore", "ignore"],
|
|
env: e,
|
|
});
|
|
result = "gbrain";
|
|
} catch {
|
|
result = null;
|
|
}
|
|
_gbrainBinCache.set(key, result);
|
|
return result;
|
|
}
|
|
|
|
/** Memoized per-process. */
|
|
const _gbrainVersionCache = new Map<string, string>();
|
|
export function readGbrainVersion(env?: NodeJS.ProcessEnv): string {
|
|
const e = env ?? process.env;
|
|
const key = `${e.PATH || ""}|${resolveGbrainBin(e) || ""}`;
|
|
if (_gbrainVersionCache.has(key)) return _gbrainVersionCache.get(key)!;
|
|
let result = "";
|
|
try {
|
|
const out = execFileSync("gbrain", ["--version"], {
|
|
encoding: "utf-8",
|
|
timeout: 2_000,
|
|
stdio: ["ignore", "pipe", "ignore"],
|
|
env: e,
|
|
});
|
|
result = out.trim().split("\n")[0] || "";
|
|
} catch {
|
|
result = "";
|
|
}
|
|
_gbrainVersionCache.set(key, result);
|
|
return result;
|
|
}
|
|
|
|
function configFingerprint(): { mtime: number; size: number } {
|
|
try {
|
|
const st = statSync(gbrainConfigPath());
|
|
return { mtime: Math.floor(st.mtimeMs), size: st.size };
|
|
} catch {
|
|
return { mtime: 0, size: 0 };
|
|
}
|
|
}
|
|
|
|
function buildCacheKey(
|
|
gbrainBin: string | null,
|
|
gbrainVersion: string,
|
|
env?: NodeJS.ProcessEnv,
|
|
): CacheEntry["key"] {
|
|
const e = env ?? process.env;
|
|
const config = configFingerprint();
|
|
return {
|
|
home: e.HOME || "",
|
|
path_hash: hashPath(e.PATH || ""),
|
|
gbrain_bin_path: gbrainBin || "",
|
|
gbrain_version: gbrainVersion,
|
|
config_mtime: config.mtime,
|
|
config_size: config.size,
|
|
};
|
|
}
|
|
|
|
function keysEqual(a: CacheEntry["key"], b: CacheEntry["key"]): boolean {
|
|
return (
|
|
a.home === b.home &&
|
|
a.path_hash === b.path_hash &&
|
|
a.gbrain_bin_path === b.gbrain_bin_path &&
|
|
a.gbrain_version === b.gbrain_version &&
|
|
a.config_mtime === b.config_mtime &&
|
|
a.config_size === b.config_size
|
|
);
|
|
}
|
|
|
|
function readCache(key: CacheEntry["key"]): LocalEngineStatus | null {
|
|
if (!existsSync(cacheFilePath())) return null;
|
|
try {
|
|
const raw = JSON.parse(readFileSync(cacheFilePath(), "utf-8")) as CacheEntry;
|
|
if (raw.schema_version !== 1) return null;
|
|
if (Date.now() - raw.cached_at > CACHE_TTL_MS) return null;
|
|
if (!keysEqual(raw.key, key)) return null;
|
|
return raw.status;
|
|
} catch {
|
|
return null;
|
|
}
|
|
}
|
|
|
|
function writeCache(status: LocalEngineStatus, key: CacheEntry["key"]): void {
|
|
const entry: CacheEntry = {
|
|
schema_version: 1,
|
|
status,
|
|
cached_at: Date.now(),
|
|
key,
|
|
};
|
|
try {
|
|
mkdirSync(dirname(cacheFilePath()), { recursive: true });
|
|
const tmp = cacheFilePath() + ".tmp." + process.pid;
|
|
writeFileSync(tmp, JSON.stringify(entry, null, 2), "utf-8");
|
|
renameSync(tmp, cacheFilePath());
|
|
} catch {
|
|
// Cache write failure is non-fatal — we re-probe next call.
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Probe via `gbrain sources list --json`. Classify the outcome.
|
|
*
|
|
* Pattern strings ("Cannot connect to database", "config.json") are deliberately
|
|
* the same strings used in lib/gbrain-sources.ts:66-67. If gbrain reworks its
|
|
* error messages, classifier returns broken-config defensively (codex #8).
|
|
*/
|
|
function freshClassify(env?: NodeJS.ProcessEnv): LocalEngineStatus {
|
|
// 1. CLI on PATH?
|
|
const gbrainBin = resolveGbrainBin(env);
|
|
if (!gbrainBin) return "no-cli";
|
|
|
|
// 2. Config file present?
|
|
if (!existsSync(gbrainConfigPath())) return "missing-config";
|
|
|
|
// 3. Probe gbrain sources list.
|
|
try {
|
|
execFileSync("gbrain", ["sources", "list", "--json"], {
|
|
encoding: "utf-8",
|
|
timeout: PROBE_TIMEOUT_MS,
|
|
stdio: ["ignore", "pipe", "pipe"],
|
|
env: env ?? process.env,
|
|
});
|
|
return "ok";
|
|
} catch (err) {
|
|
const e = err as NodeJS.ErrnoException & { stderr?: Buffer | string };
|
|
const stderr = (e.stderr ? e.stderr.toString() : "") || "";
|
|
|
|
// ENOENT can happen if gbrain disappeared between resolveGbrainBin and now.
|
|
if (e.code === "ENOENT") return "no-cli";
|
|
|
|
// Pattern match against gbrain's known error strings. Order matters:
|
|
// "Cannot connect to database" is the more specific DB-unreachable signal.
|
|
if (stderr.includes("Cannot connect to database")) return "broken-db";
|
|
if (stderr.includes("config.json")) return "broken-config";
|
|
|
|
// Defensive default per codex #8: unrecognized failures classify as
|
|
// broken-config so the user sees the raw stderr surfaced upstream.
|
|
return "broken-config";
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Classify the local gbrain engine status. Cached for 60s; bypassable.
|
|
*
|
|
* Returns one of 5 states. Never throws — failure modes are surfaced as states.
|
|
*/
|
|
export function localEngineStatus(opts: ClassifyOptions = {}): LocalEngineStatus {
|
|
const env = opts.env ?? process.env;
|
|
const gbrainBin = resolveGbrainBin(env);
|
|
const gbrainVersion = gbrainBin ? readGbrainVersion(env) : "";
|
|
const key = buildCacheKey(gbrainBin, gbrainVersion, env);
|
|
|
|
if (!opts.noCache) {
|
|
const cached = readCache(key);
|
|
if (cached) return cached;
|
|
}
|
|
|
|
const fresh = freshClassify(env);
|
|
writeCache(fresh, key);
|
|
return fresh;
|
|
}
|