mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-21 20:28:24 +08:00
* fix(gstack-paths): guard CLAUDE_PLUGIN_DATA against cross-plugin contamination (#1569) gstack-paths previously trusted CLAUDE_PLUGIN_DATA as a fallback for GSTACK_STATE_ROOT whenever GSTACK_HOME was unset. When another plugin (e.g. Codex) persists its own CLAUDE_PLUGIN_DATA into the session env via CLAUDE_ENV_FILE, gstack picked it up and wrote checkpoints, analytics, and learnings into that plugin's directory. Anyone with the Codex plugin installed alongside gstack hit this silently. Fix: guard the CLAUDE_PLUGIN_DATA branch so it only fires when CLAUDE_PLUGIN_ROOT confirms we're running as the gstack plugin (path contains "gstack"). Skill installs fall through to \$HOME/.gstack. Contributed by @ElliotDrel via #1570. Closes #1569. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(gbrain-sync): sourceLocalPath handles wrapped {sources:[...]} shape from gbrain v0.20+ gbrain v0.20+ changed `gbrain sources list --json` to return {sources: [...]} instead of a flat array. sourceLocalPath crashed upstream with `list.find is not a function` on every /sync-gbrain invocation against modern gbrain. Accept both shapes for forward/backward compat, matching probeSource/sourcePageCount in lib/gbrain-sources.ts. Contributed by @jakehann11 via #1571. Closes #1567. Supersedes #1564 (@tonyjzhou, same fix, different shape — credit retained). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(brain-context-load): probe gbrain via execFile, not shell builtin (#1559) gbrainAvailable() used `execFileSync("command", ["-v", "gbrain"])`, which fails in any environment where the `command` builtin isn't on the spawned process's PATH (most non-interactive shells). The probe then reported gbrain as missing even when it was installed, and context-load silently skipped vector/list queries. Fix: probe `gbrain --version` directly with a 500ms timeout (matching the rest of the file's MCP_TIMEOUT_MS). Same semantics, works everywhere execFile works. Contributed by @jbetala7 via #1560. Closes #1559. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(gbrain-doctor): pin schema_version:2 doctor parse path (#1418) Adds an exec-path regression test that runs a fake gbrain shim emitting the v0.25+ doctor JSON shape (schema_version: 2, status: "warnings", exit 1 for health_score < 100, no top-level `engine` field). Confirms freshDetectEngineTier recovers stdout from the non-zero exit and falls back to GBRAIN_HOME/config.json for the engine label. The pre-existing test for #1415 only stripped gbrain from PATH; this test exercises the actual doctor parse path, closing the gap that codex's plan review flagged. Also documents the schema_version separation in lib/gbrain-local-status.ts: the local CacheEntry stays at version 1, distinct from the doctor-output schema_version which we accept across versions in gstack-memory-helpers. Closes #1418 (credit @mvanhorn for surfacing the doctor + schema_v2 collapse). The fix landed pre-emptively in v1.29.x; this commit pins it with a stronger test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(memory-ingest): pin put_page regression + scrub stale name from --help and comments (#1346) #1346 reported that gstack-memory-ingest still called the renamed gbrain put_page subcommand on gbrain v0.18+. The actual code migrated to `gbrain put` and later to batch `gbrain import <dir>` before this report landed — only documentation lag remained. This commit: - Updates the --help string ("Skip gbrain put calls (still updates state file)") so user-facing docs match the shipped subcommand - Updates two inline comments that still referenced the old name - Adds test/memory-ingest-no-put_page.test.ts: a regression pin that strips comments from bin/gstack-memory-ingest.ts and fails the build if "put_page" appears in any active code or string literal, plus a sanity check that the file still calls a supported gbrain page-write verb (put or import) Closes #1346. Reporter @kylma-code surfaced the doc lag; the original code migration credit is on the v1.27.x wave. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(resolvers): rewrite all gbrain put_page instructions to canonical put <slug> scripts/resolvers/gbrain.ts emitted user-facing copy-paste instructions using the renamed `gbrain put_page` subcommand across 10 skills (office-hours, investigate, plan-ceo-review, retro, plan-eng-review, ship, cso, design-consultation, fallback, entity-stub). Every gstack user copying those snippets hit "unknown command: put_page" on gbrain v0.18+. This commit: - Rewrites all 10 instruction templates to use `gbrain put <slug> --content "$(cat <<EOF...EOF)"` with title/tags moved into YAML frontmatter inside --content, matching the v0.18+ subcommand shape - Updates README.md and USING_GBRAIN_WITH_GSTACK.md "common commands" table to reference `gbrain put` and `gbrain get` - Adds test/resolvers-gbrain-put-rewrite.test.ts pinning two invariants: (a) resolver source ships only canonical instructions, (b) every tracked SKILL.md file is free of `gbrain put_page` CHANGELOG entries are deliberately left untouched (historical record). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(build): extract package.json build to scripts/build.sh for Windows Bun compat (#1538, #1537, #1530, #1457, #1561) Bun's Windows shell parser rejects multiple constructs the inline package.json build chain used: brace groups `{ cmd; }`, subshells with redirection `( git ... ) > path/.version`, and (in Bun 1.3.x) subshells near redirections in general. Every Windows install + every auto-upgrade since v1.34.2.0 has failed on `bun run build`. Extracts the build chain to scripts/build.sh and the .version writes to scripts/write-version-files.sh. POSIX-portable, no Bun shell parsing involved. Also adds Windows-specific bun.exe handling for non-ASCII PATHs (a separate Windows footgun where Bun's --compile fails when the binary lives under a path with non-ASCII chars). Updates test/build-script-shell-compat.test.ts to assert the new shape: no subshells with redirections anywhere in the build chain, and build delegates to scripts/build.sh which delegates .version writes. Contributed by @Charlie-El via #1544. Supersedes #1531 (@scarson, fixed in build helper), #1480 (@mikepsinn, partial overlap), #1460 (@realcarsonterry, brace-group fix subsumed) — credit retained. Closes #1538, #1537, #1530, #1457, #1561. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(windows): .exe glob in .gitignore + .exe extension resolution in find-browse (#1554) bun build --compile on Windows appends .exe to the output filename, producing browse.exe instead of browse. find-browse's existsSync probe only checked the bare path and returned null on Windows even when the binary was correctly built. .gitignore similarly only excluded the bare bin/gstack-global-discover path, leaving the .exe variant tracked. This commit: - .gitignore: changes `bin/gstack-global-discover` → `bin/gstack-global-discover*` so the Windows .exe variant is ignored - browse/src/find-browse.ts: adds isExecutable + findExecutable helpers that fall back to .exe/.cmd/.bat probing on Windows, mirroring the same helper already in make-pdf/src/browseClient.ts and pdftotext.ts Contributed by @Mike-E-Log via #1554. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(windows): add fresh-install E2E gate that runs bun run build on windows-latest Adds .github/workflows/windows-setup-e2e.yml as the gate that catches Bun shell-parser regressions in the build chain before they reach users. Triggers on PRs touching package.json, scripts/build.sh, scripts/write-version-files.sh, setup, browse cli/find-browse, or gstack-paths. What it verifies: 1. bun run build completes on Windows (the previously-broken path that #1538/#1537/#1530/#1457/#1561 reported) 2. All compiled binaries land on disk (browse.exe, find-browse.exe, design.exe, gstack-global-discover.exe) 3. find-browse resolves to the .exe variant on Windows (regression gate for #1554) 4. gstack-paths returns non-empty GSTACK_STATE_ROOT/PLAN_ROOT/TMP_ROOT on Windows (regression gate for #1570) Complements the existing windows-free-tests.yml (curated unit subset); this new workflow exercises the install path itself. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(codex): move diff scope into prompt instead of --base (Codex CLI 0.130+ argv conflict) (#1209) Codex CLI ≥ 0.130.0 rejects passing a custom prompt and --base together (mutually exclusive at argv level). Every /codex review, /review, and /ship structured Codex review call ended with an argv error before the model ran. Fix: scope the diff in prompt text using "Run git diff origin/<base>...HEAD 2>/dev/null || git diff <base>...HEAD" instead of `--base <base>`. Preserves the filesystem boundary instruction across all invocations and keeps Codex's review prompt tuning. Touches: - codex/SKILL.md.tmpl + regenerated codex/SKILL.md - scripts/resolvers/review.ts + regenerated review/SKILL.md, ship/SKILL.md - test/gen-skill-docs.test.ts: new regression that fails if any of the five known files still contain the prompt+--base shape - test/skill-validation.test.ts: corresponding negative + positive pin on the rendered SKILL.md files Contributed by @jbetala7 via #1209. Closes #1479. Supersedes #1527 (@mvanhorn — same intent, different patch shape, CONFLICTING) and #1449 (@Gujiassh — broader refactor, CONFLICTING). Credit retained in CHANGELOG. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(review): diff from git merge-base, not git diff origin/<base> (#1492) git diff origin/<base> shows everything since the common ancestor in both directions — it includes commits that landed on origin/<base> after this branch was created as deletions. That made /review and /ship's pre-landing structured review report inflated diff totals and flagged "removed" code that was actually still present in the working tree. Fix: compute DIFF_BASE via git merge-base origin/<base> HEAD and diff the working tree against that point. Same coverage of uncommitted edits, no phantom deletions from out-of-order base advancement. Applies to /review's Step 1 (diff existence check), Step 3 (get the diff), the build-on-intent scope-creep check, the structured review DIFF_INS/DIFF_DEL stats, and the Claude adversarial subagent prompt. Same change flows into ship/SKILL.md via the shared resolver. Touches: - review/SKILL.md.tmpl + regenerated review/SKILL.md, ship/SKILL.md - scripts/resolvers/review.ts - scripts/resolvers/review-army.ts Contributed by @mvanhorn via #1492. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(codex): pin filesystem-boundary preservation across all codex review surfaces (#1503, #1522) #1503 reported that the bare codex review --base path stripped the filesystem boundary instruction, letting Codex spend tokens reading .claude/skills/ and agents/. #1522 proposed adding a skill-path detector that switched to the custom-instructions route when the diff touched skill files. After C10 (#1209) restructured codex review to always carry the boundary in the prompt (the prompt+--base argv conflict forced the restructure), the skill-path detector becomes redundant — every default call already preserves the boundary. This commit pins the post-#1209 invariant with a test that fails the build if any future refactor strips the boundary from codex/SKILL.md, review/SKILL.md, or ship/SKILL.md. Closes #1503 by regression test. #1522 (@genisis0x) is superseded by #1209 (the prompt rewrite covers its safety concern); credit retained in CHANGELOG. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(skills): use command -v instead of which for codex detection (#1197) `which` is not on PATH in every shell — some Windows shells, BusyBox- only containers, and minimal CI images all fail when skills probe codex availability via `which codex`. `command -v` is a POSIX builtin and always available where the skill is running. Touched: - codex/SKILL.md.tmpl: CODEX_BIN=$(command -v codex || echo "") - scripts/resolvers/review.ts and scripts/resolvers/design.ts: 3 + 3 sites each rewritten to `command -v codex >/dev/null 2>&1` - Regenerated all 10 affected SKILL.md files (codex, review, ship, design-consultation, design-review, office-hours, plan-ceo-review, plan-design-review, plan-devex-review, plan-eng-review) - test/skill-validation.test.ts: updated pin + defensive regression test that fails if `which codex` returns to codex/SKILL.md - test/skill-e2e-plan.test.ts: updated summary regex Contributed by @mvanhorn via #1197. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(codex): surface non-zero exits so wrappers stop reading as silent stalls (#1467, #1327) When codex exits non-zero (parse errors, arg-shape breaks, model API errors that propagate as non-zero status), the calling agent previously saw an empty output and burned 30-60 minutes misdiagnosing as a silent model/API stall. The hang-detection block only caught exit 124 (the timeout-wrapper signal). Adds elif blocks in all four codex invocation sites (Review default, Challenge, Consult new-session, Consult resume) that: - Echo "[codex exit N] <stderr first line>" to stdout - Indent the first 20 stderr lines for inline context - Log codex_nonzero_exit telemetry tagged with the call site Contributed by @genisis0x via #1467. Closes #1327. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(design): disclose OpenAI key source + warn on cwd .env match (#1278, closes #1248) The design binary previously called process.env.OPENAI_API_KEY without checking where the key came from. If a user ran $D inside someone else's project that had OPENAI_API_KEY in its .env, the resulting generation billed that project's account. Silent and irreversible. Fix: resolveApiKeyInfo() returns both the key and its source. When the env-var path matches an OPENAI_API_KEY entry in the current directory's .env, .env.<NODE_ENV>, or .env.local file, we set a warning. requireApiKey() prints "Using OpenAI key from <source>" plus the warning before the run — never the key itself. Adds 6 unit tests covering: config-vs-env precedence, env-only (no match), env+cwd .env match, quoted/exported values, value-mismatch (no false positive), and the no-leak invariant for requireApiKey stderr output. Contributed by @jbetala7 via #1278. Closes #1248. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(browse): guard full-page screenshots against Anthropic vision API >2000px brick (#1214) Full-page screenshots of tall pages routinely exceeded 2000px on the longest dimension, silently bricking the agent's session: the resulting base64 reached the Anthropic vision API which rejected the oversized image, leaving the agent burning turns on a useless blob with no stderr trace from the browse side. Adds browse/src/screenshot-size-guard.ts as a shared helper: - guardScreenshotBuffer(buf) → downscales in-memory if max(w,h) > 2000 - guardScreenshotPath(path) → file-mode variant that rewrites in place - Aspect ratio preserved via sharp's resize fit:inside - Stderr diagnostic on any downscale so callers can see when it fired - Lazy sharp import so non-screenshot paths pay no startup cost Wires the guard into all three full-page callsites codex review flagged: - browse/src/snapshot.ts: annotated + heatmap fullPage captures - browse/src/meta-commands.ts: screenshot command (path + base64 fullPage modes) plus the responsive 3-viewport sweep - browse/src/write-commands.ts: prettyscreenshot fullPage path Covers seven unit cases (pass-through, downscale, aspect ratio, exactly-2000px edge, file-mode rewrite) plus a static invariant test that fails the build if any of the three callsites stops importing the guard. Closes #1214. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(security): add Node sidecar entry for L4 prompt-injection classifier (#1370) The L4 TestSavant classifier in browse/src/security-classifier.ts can't be imported into the compiled browse server (onnxruntime-node dlopen fails from Bun's compile extract dir per CLAUDE.md). The agent that used to host it (sidebar-agent.ts) was removed when the PTY proved out — leaving the classifier file shipped but with zero callers. Exactly the gap codex flagged in #1370. Adds browse/src/security-sidecar-entry.ts: a Node script that runs the classifier as a subprocess of the browse server. It reads NDJSON requests from stdin and writes id-correlated NDJSON responses to stdout, supporting: - op: "scan-page-content" — full L4 classifier scan - op: "ping" — liveness probe for the client's health check - op: "status" — classifier readiness (used by /pty-inject-scan to surface l4 { available: bool } in its response) Plus browse/src/find-security-sidecar.ts: a resolver that locates node + the bundled JS entry (browse/dist/security-sidecar.js, built in a follow-up package.json change) or falls back to the dev TS entry. Returns null cleanly when node isn't on PATH so the calling endpoint can degrade per D7 (extension WARN + user confirm). C17 of the security-stack wave. C18 adds the IPC client + lifecycle management; C19 wires the endpoint; C20 routes the extension through it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(security): sidecar IPC client with lifecycle + circuit breaker (#1370) Adds browse/src/security-sidecar-client.ts to manage the Node L4 classifier subprocess from the compiled browse server: - Lazy spawn on first scan; reuses the same process across requests - Id-correlated request/response via NDJSON over stdio - 5s default per-scan timeout; 64KB payload cap (short-circuits before spawn so oversized requests don't waste a process) - 3-in-10-minutes respawn cap → trips circuit breaker; subsequent scans throw immediately so the /pty-inject-scan endpoint can surface l4 { available: false } to the extension and degrade to WARN+confirm - process.on('exit') sends SIGTERM to the child for clean teardown - isSidecarAvailable() lets the endpoint probe before scan calls so the response shape reflects degraded mode honestly Unit tests cover the payload cap, the availability probe, and the breaker-doesn't-crash invariant under repeated rejected calls. C18 of the security-stack wave. C19 adds POST /pty-inject-scan; C20 routes the extension through it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(security): add POST /pty-inject-scan endpoint for pre-PTY-inject scans (#1370) The sidebar's gstackInjectToTerminal callers (toolbar Cleanup, Inspector "Send to Code") were piping page-derived text directly into the live claude PTY with ZERO classifier processing — the gap codex flagged in #1370. The documented sidebar security stack had a hole the size of every Cleanup-button click. Adds POST /pty-inject-scan to browse/src/server.ts: - Local-only binding (NOT in TUNNEL_PATHS — tunnel attempts get the general 404 path; never reaches the scan logic) - Root-token auth via existing validateAuth() — 401 on unauth - 64KB request cap → 413 + payload-too-large body - 5s scan timeout via sidecar client - URL-blocklist forced to BLOCK in PTY context (page-derived REPL input is higher-risk than ordinary tool output) - L4 ML classifier via the sidecar when available; degrades to WARN per D7 when sidecar is unavailable - Response goes through JSON.stringify(..., sanitizeReplacer) per v1.38.0.0 Unicode-egress hardening - Imports only from security-sidecar-client.ts, never directly from security-classifier.ts (which would brick the compiled Bun binary) Seven static-invariant tests pin the POST verb, auth gate, 64KB cap, tunnel-listener exclusion, sanitizeReplacer wrapping, l4 availability shape, and the no-direct-classifier-import rule. C19 of the security-stack wave. C20 routes the extension through it; C21 adds the invariant AST check. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(extension): route gstackInjectToTerminal through /pty-inject-scan (#1370) Closes the documented-vs-shipped gap codex flagged in #1370. The sidebar's two PTY-injection call sites (Inspector "Send to Code" and toolbar Cleanup) now pre-scan via the new /pty-inject-scan endpoint before writing to the live claude REPL. Adds window.gstackScanForPTYInject(text, origin) to extension/sidepanel-terminal.js: - Async, returns { allow, verdict, reasons, l4 } - POST to /pty-inject-scan with the existing root-token auth - WARN+confirm on scan failure (network down, sidecar absent, etc.) rather than silent PASS — D7 honest-degradation gstackInjectToTerminal stays synchronous, returns boolean. Per D6: keeping the inject sync means existing `const ok = ...?.()` callers don't break, and the invariant test in test/extension-pty-inject-invariant.test.ts can statically pin that every call goes through the scan first. extension/sidepanel.js call sites updated: - inspectorSendBtn click → await scan, BLOCK drops + WARN prompts via window.confirm, PASS injects silently - runCleanup() → same flow. Static cleanup prompt always PASSes but still routes through scan to honor the invariant. C20 of the security-stack wave. C21 adds the static invariant test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(security): invariant — extension PTY inject must be scan-gated (#1370) Static-analysis invariant test that fails the build if any extension/*.js path calls window.gstackInjectToTerminal without a preceding window.gstackScanForPTYInject in the same enclosing function. Closes the documented-vs-shipped gap codex demanded a machine check on. Rules: - Rule 1: any file that calls inject must also reference scan - Rule 2: in the enclosing function (function declaration, arrow, async (), event handler), a scan call must appear before the inject call by source position - Exemption: sidepanel-terminal.js (the file that DEFINES the inject function) is exempt from Rule 2 since the definition is not a call Plus two structural checks: - sidepanel-terminal.js defines both the inject and scan functions - inject stays SYNCHRONOUS (no `async` modifier) per D6 — async would silently break the `const ok = ...?.()` pattern at every caller C21 of the security-stack wave. The sidecar architecture (#1370) is complete: server-side L1-L3 + L4-via-sidecar (C17+C18+C19), extension pre-scan wiring (C20), and now the regression gate (C21). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(browse): opt-in extended stealth mode with 6 detection-vector patches (#1112) Rebases @garrytan's PR #1112 (Apr 2026, abandoned) onto the current browse/src/stealth.ts contract. The existing minimal "codex narrowed" stealth (webdriver-mask + AutomationControlled launch arg) stays the default. PR #1112's six additional patches are added behind an opt-in GSTACK_STEALTH=extended env flag. Extended-mode patches (applied AFTER the default mask, in order): 1. delete navigator.webdriver from prototype (not just the getter — detectors check `"webdriver" in navigator`) 2. WebGL renderer spoof to Apple M1 Pro (SwiftShader was the #1 software-GPU tell in containers) 3. navigator.plugins returns a PluginArray-prototype-passing array with MimeType objects and namedItem() 4. window.chrome populated with chrome.app, chrome.runtime, chrome.loadTimes(), chrome.csi() with realistic shapes 5. navigator.mediaDevices backfilled when headless drops it 6. CDP cdc_*-prefixed window globals cleared Why opt-in: the default mode's contract is fingerprint CONSISTENCY, which protects against detectors that flag spoofing mismatch. Extended mode actively lies about the environment; sites that reflect on these properties can break. Users who hit detection in default mode can flip GSTACK_STEALTH=extended for SannySoft 100% pass-rate. Twenty unit tests pin the env-flag semantics, all six patches' code presence, and the applyStealth wiring order. Live SannySoft pass-rate verification stays in the periodic-tier E2E suite. Contributed by @garrytan via #1112 (rebased — original PR opened before the codex-narrowed minimum landed; rebase preserves the narrowed default while adding the SannySoft-passing path as opt-in). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(fixtures): regenerate ship-SKILL.md golden baselines after C10-C13 + C16 templates Updates the three ship-SKILL.md golden baselines (claude, codex, factory hosts) to match the new shape produced by: - C10 #1209 codex argv (prompt + diff scope, no --base) - C11 #1492 merge-base diff (DIFF_BASE= preamble) - C13 #1197 command -v for codex detection - C12 + boundary preservation per regen-enforcing test Per CLAUDE.md SKILL.md workflow: edit the .tmpl, run gen:skill-docs, commit the regenerated outputs together. Goldens are part of the regen contract — without this commit, test/host-config.test.ts' golden-baseline checks fail with the diff codex review surfaced. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(release): v1.41.0.0 — Daegu wave (24 bisect commits, 14 user-facing fixes) Bumps VERSION 1.40.0.0 → 1.41.0.0. CHANGELOG entry follows the release-summary format in CLAUDE.md: two-line headline, lead paragraph, "The numbers that matter" table, "What this means for builders" closer, then itemized Added/Changed/Fixed/For contributors with inline credit to every PR author and original issue reporter. Scale-aware bump per CLAUDE.md: 24 commits, ~6000 LOC net, substantial new capability across security (PTY sidecar wiring), install (Windows build chain), compat (gbrain 0.18-0.35, Codex CLI 0.130+), and quality (screenshot guard, design key disclosure, extended stealth opt-in). MINOR is the right call. Closes for users: #1567, #1559, #1569, #1346, #1418, #1538, #1537, #1530, #1457, #1561, #1554, #1479, #1503, #1248, #1214, #1370, #1327, #1193 pattern, #1152 pattern. Credit retained inline. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(find-browse): resolve source-checkout layout <git-root>/browse/dist/browse[.exe] windows-setup-e2e.yml runs `bun browse/src/find-browse.ts` against a freshly-built repo where binaries land at browse/dist/browse.exe (no .claude/skills/gstack/ install layout). The previous markers chain only matched .codex/.agents/.claude prefixed paths, so find-browse exited "not found" even when the binary was present. Adds a source-checkout fallback after the marker scan: if no installed layout resolves but <git-root>/browse/dist/browse[.exe] exists, return that. Three real callers hit this path: - gstack repo dev workflow before `./setup` runs - windows-setup-e2e.yml CI (the breakage that surfaced this) - make-pdf consumers running from a sibling source checkout Smoke-verified: a fresh git repo with browse/dist/browse on disk now resolves through the source-checkout branch (was returning null before this commit). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(release): bump v1.41.0.0 → v1.42.0.0 to clear queue collision with #1574 The version-gate workflow flagged a collision: PR #1574 (garrytan/colombo-v3) already claims v1.41.0.0, and #1592 (fix/audit-critical-high-bugs) claims v1.41.1.0. Per CLAUDE.md's workspace-aware ship rule, queue-advancing past a claimed version within the same bump level is permitted — MINOR work landing on top of a queued MINOR still reads as MINOR relative to main. Util's suggested next slot is v1.42.0.0; taking it. CHANGELOG entry header bumped + dated 2026-05-19; entry body unchanged (same wave content, same credit list). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3144 lines
134 KiB
TypeScript
3144 lines
134 KiB
TypeScript
import { describe, test, expect } from 'bun:test';
|
|
import { COMMAND_DESCRIPTIONS } from '../browse/src/commands';
|
|
import { SNAPSHOT_FLAGS } from '../browse/src/snapshot';
|
|
import * as fs from 'fs';
|
|
import * as path from 'path';
|
|
import * as os from 'os';
|
|
|
|
const ROOT = path.resolve(import.meta.dir, '..');
|
|
const MAX_SKILL_DESCRIPTION_LENGTH = 1024;
|
|
|
|
function extractDescription(content: string): string {
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
expect(fmEnd).toBeGreaterThan(0);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
const lines = frontmatter.split('\n');
|
|
let description = '';
|
|
let inDescription = false;
|
|
const descLines: string[] = [];
|
|
|
|
for (const line of lines) {
|
|
if (line.match(/^description:\s*\|?\s*$/)) {
|
|
inDescription = true;
|
|
continue;
|
|
}
|
|
if (line.match(/^description:\s*\S/)) {
|
|
return line.replace(/^description:\s*/, '').trim();
|
|
}
|
|
if (inDescription) {
|
|
if (line === '' || line.match(/^\s/)) {
|
|
descLines.push(line.replace(/^ /, ''));
|
|
} else {
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
|
|
if (descLines.length > 0) {
|
|
description = descLines.join('\n').trim();
|
|
}
|
|
return description;
|
|
}
|
|
|
|
function extractMarkdownSection(content: string, heading: string): string {
|
|
const escaped = heading.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
|
const startMatch = content.match(new RegExp(`^${escaped}.*$`, 'm'));
|
|
expect(startMatch?.index).toBeDefined();
|
|
const start = startMatch!.index!;
|
|
const afterHeading = start + startMatch![0].length;
|
|
const nextSection = content.slice(afterHeading).match(/\n## /);
|
|
const end = nextSection?.index === undefined
|
|
? content.length
|
|
: afterHeading + nextSection.index;
|
|
return content.slice(start, end).trim();
|
|
}
|
|
|
|
function extractPreambleBeforeWorkflow(content: string, workflowMarkers: string[]): string {
|
|
const markerIndexes = workflowMarkers
|
|
.map(marker => content.indexOf(marker))
|
|
.filter(index => index >= 0);
|
|
expect(markerIndexes.length).toBeGreaterThan(0);
|
|
return content.slice(0, Math.min(...markerIndexes));
|
|
}
|
|
|
|
function isRepoRootSymlink(candidateDir: string): boolean {
|
|
try {
|
|
return fs.realpathSync(candidateDir) === fs.realpathSync(ROOT);
|
|
} catch {
|
|
return false;
|
|
}
|
|
}
|
|
|
|
// Dynamic template discovery — matches the generator's findTemplates() behavior.
|
|
// New skills automatically get test coverage without updating a static list.
|
|
const ALL_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; name: string }> = [];
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
skills.push({ dir: '.', name: 'root gstack' });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) {
|
|
skills.push({ dir: entry.name, name: entry.name });
|
|
}
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
const CLAUDE_SKIPPED_SKILL_DIRS = new Set(['claude']);
|
|
const CLAUDE_GENERATED_SKILLS = ALL_SKILLS.filter(skill => !CLAUDE_SKIPPED_SKILL_DIRS.has(skill.dir));
|
|
|
|
describe('gen-skill-docs', () => {
|
|
test('generated SKILL.md contains all command categories', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
const categories = new Set(Object.values(COMMAND_DESCRIPTIONS).map(d => d.category));
|
|
for (const cat of categories) {
|
|
expect(content).toContain(`### ${cat}`);
|
|
}
|
|
});
|
|
|
|
test('generated SKILL.md contains all commands', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
const display = meta.usage || cmd;
|
|
expect(content).toContain(display);
|
|
}
|
|
});
|
|
|
|
test('command table is sorted alphabetically within categories', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
// Extract command names from the Navigation section as a test
|
|
const navSection = content.match(/### Navigation\n\|.*\n\|.*\n([\s\S]*?)(?=\n###|\n## )/);
|
|
expect(navSection).not.toBeNull();
|
|
const rows = navSection![1].trim().split('\n');
|
|
const commands = rows.map(r => {
|
|
const match = r.match(/\| `(\w+)/);
|
|
return match ? match[1] : '';
|
|
}).filter(Boolean);
|
|
const sorted = [...commands].sort();
|
|
expect(commands).toEqual(sorted);
|
|
});
|
|
|
|
test('generated header is present in SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
});
|
|
|
|
test('generated header is present in browse/SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
});
|
|
|
|
test('snapshot flags section contains all flags', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
expect(content).toContain(flag.short);
|
|
expect(content).toContain(flag.description);
|
|
}
|
|
});
|
|
|
|
test('every skill has a SKILL.md.tmpl template', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
expect(fs.existsSync(tmplPath)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('every skill has a generated SKILL.md with auto-generated header', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const mdPath = path.join(ROOT, skill.dir, 'SKILL.md');
|
|
expect(fs.existsSync(mdPath)).toBe(true);
|
|
const content = fs.readFileSync(mdPath, 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
}
|
|
});
|
|
|
|
test('every generated SKILL.md has valid YAML frontmatter', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
expect(content.startsWith('---\n')).toBe(true);
|
|
expect(content).toContain('name:');
|
|
expect(content).toContain('description:');
|
|
}
|
|
});
|
|
|
|
test(`every generated SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const description = extractDescription(content);
|
|
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
|
|
}
|
|
});
|
|
|
|
test('Claude outside-voice skill is not generated for Claude host', () => {
|
|
expect(fs.existsSync(path.join(ROOT, 'claude', 'SKILL.md.tmpl'))).toBe(true);
|
|
expect(fs.existsSync(path.join(ROOT, 'claude', 'SKILL.md'))).toBe(false);
|
|
});
|
|
|
|
test(`every Codex SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return; // skip if not generated
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
const description = extractDescription(content);
|
|
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
|
|
}
|
|
});
|
|
|
|
test('every Codex SKILL.md description stays under 900-char warning threshold', () => {
|
|
const WARN_THRESHOLD = 900;
|
|
const agentsDir = path.join(ROOT, '.agents', 'skills');
|
|
if (!fs.existsSync(agentsDir)) return;
|
|
const violations: string[] = [];
|
|
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
const description = extractDescription(content);
|
|
if (description.length > WARN_THRESHOLD) {
|
|
violations.push(`${entry.name}: ${description.length} chars (limit ${MAX_SKILL_DESCRIPTION_LENGTH}, ${MAX_SKILL_DESCRIPTION_LENGTH - description.length} remaining)`);
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('package.json version matches VERSION file', () => {
|
|
const pkg = JSON.parse(fs.readFileSync(path.join(ROOT, 'package.json'), 'utf-8'));
|
|
const version = fs.readFileSync(path.join(ROOT, 'VERSION'), 'utf-8').trim();
|
|
expect(pkg.version).toBe(version);
|
|
});
|
|
|
|
test('generated files are fresh (match --dry-run)', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// Every skill should be FRESH
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const file = skill.dir === '.' ? 'SKILL.md' : `${skill.dir}/SKILL.md`;
|
|
expect(output).toContain(`FRESH: ${file}`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('no generated SKILL.md contains unresolved placeholders', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const unresolved = content.match(/\{\{[A-Z_]+\}\}/g);
|
|
expect(unresolved).toBeNull();
|
|
}
|
|
});
|
|
|
|
test('templates contain placeholders', () => {
|
|
const rootTmpl = fs.readFileSync(path.join(ROOT, 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(rootTmpl).toContain('{{COMMAND_REFERENCE}}');
|
|
expect(rootTmpl).toContain('{{SNAPSHOT_FLAGS}}');
|
|
expect(rootTmpl).toContain('{{PREAMBLE}}');
|
|
|
|
const browseTmpl = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(browseTmpl).toContain('{{COMMAND_REFERENCE}}');
|
|
expect(browseTmpl).toContain('{{SNAPSHOT_FLAGS}}');
|
|
expect(browseTmpl).toContain('{{PREAMBLE}}');
|
|
});
|
|
|
|
test('generated SKILL.md contains operational self-improvement (replaced contributor mode)', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('Contributor Mode');
|
|
expect(content).not.toContain('gstack_contributor');
|
|
expect(content).not.toContain('contributor-logs');
|
|
expect(content).toContain('Operational Self-Improvement');
|
|
expect(content).toContain('gstack-learnings-log');
|
|
expect(content).toContain('gstack-learnings-search --limit 3');
|
|
});
|
|
|
|
test('generated SKILL.md with LEARNINGS_LOG contains operational type', () => {
|
|
// Check a skill that has LEARNINGS_LOG (e.g., review)
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('operational');
|
|
});
|
|
|
|
test('generated SKILL.md contains session awareness', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_SESSIONS');
|
|
expect(content).toContain('RECOMMENDATION');
|
|
});
|
|
|
|
test('generated SKILL.md contains branch detection', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_BRANCH');
|
|
expect(content).toContain('git branch --show-current');
|
|
});
|
|
|
|
test('tier 2+ skills contain ELI10 simplification rules (AskUserQuestion format)', () => {
|
|
// Root SKILL.md is tier 1 (no AskUserQuestion format). Check a tier 2+ skill instead.
|
|
// v1.7.0.0 Pros/Cons format uses "ELI10 (ALWAYS)" rather than "Simplify (ELI10".
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('ELI10');
|
|
expect(content).toContain('plain English');
|
|
expect(content).toContain('not function names');
|
|
});
|
|
|
|
test('tier 1 skills do NOT contain AskUserQuestion format', () => {
|
|
// Use benchmark (tier 1) instead of root — root SKILL.md gets overwritten by Codex test setup
|
|
const content = fs.readFileSync(path.join(ROOT, 'benchmark', 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('## AskUserQuestion Format');
|
|
expect(content).not.toContain('## Completeness Principle');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry line', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('skill-usage.jsonl');
|
|
expect(content).toContain('~/.gstack/analytics');
|
|
});
|
|
|
|
test('plan-review generated preambles stay under the Option A budget', () => {
|
|
const reviewSkills = [
|
|
{
|
|
path: path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
|
|
markers: ['# Mega Plan Review Mode', '## Step 0: Detect platform and base branch'],
|
|
},
|
|
{
|
|
path: path.join(ROOT, 'plan-eng-review', 'SKILL.md'),
|
|
markers: ['# Plan Review Mode'],
|
|
},
|
|
];
|
|
|
|
// Plan skills carry the same preamble surface as other tier-≥2 skills
|
|
// (Artifacts Sync, Context Recovery, Routing Injection are load-bearing
|
|
// functionality, not optional). Budget is set to current size + small
|
|
// headroom; ratchet down if a future slim trims real bytes.
|
|
// Ratcheted from 33000 → 35000 when the gbrain context-load block was
|
|
// added (per /sync-gbrain plan §4). Ratcheted 35000 → 36500 in v1.27.0.0
|
|
// when generate-brain-sync-block.ts gained the gbrain_mcp_mode probe +
|
|
// remote-mode ARTIFACTS_SYNC status line (Path 4 of /setup-gbrain).
|
|
// Ratcheted 36500 → 39000 in the contributor wave when #1205 added the
|
|
// \\u-escape CJK rule (rule 12 + self-check item) to the AskUserQuestion
|
|
// preamble.
|
|
for (const skill of reviewSkills) {
|
|
const content = fs.readFileSync(skill.path, 'utf-8');
|
|
const preamble = extractPreambleBeforeWorkflow(content, skill.markers);
|
|
expect(Buffer.byteLength(preamble, 'utf-8')).toBeLessThan(39_000);
|
|
}
|
|
});
|
|
|
|
test('voice and writing-style preamble sections stay compact', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
|
|
const voice = extractMarkdownSection(content, '## Voice');
|
|
const writingStyle = extractMarkdownSection(content, '## Writing Style');
|
|
|
|
expect(Buffer.byteLength(voice, 'utf-8')).toBeLessThan(3_000);
|
|
expect(Buffer.byteLength(writingStyle, 'utf-8')).toBeLessThan(2_000);
|
|
});
|
|
|
|
test('slim voice section preserves the gstack voice contract', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
|
|
const voice = extractMarkdownSection(content, '## Voice');
|
|
|
|
expect(voice).toMatch(/lead with the point|direct/i);
|
|
expect(voice).toMatch(/file|function|line|command|real numbers/i);
|
|
expect(voice).toMatch(/user.*outcome|user.*experience|real user/i);
|
|
expect(voice).toMatch(/corporate|academic|PR|hype/i);
|
|
expect(voice).toMatch(/AI vocabulary|delve|crucial|robust/i);
|
|
expect(voice).toMatch(/user decides|user.*context|sovereignty|recommendation, not a decision/i);
|
|
});
|
|
|
|
test('preamble .pending-* glob is zsh-safe (uses find, not shell glob)', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
if (!content.includes('.pending-')) continue;
|
|
// Must NOT have a bare shell glob ".pending-*" outside of find's -name argument
|
|
expect(content).not.toMatch(/for _PF in [^\n]*\/\.pending-\*/);
|
|
// Must use find to avoid zsh NOMATCH error on glob expansion
|
|
expect(content).toContain("find ~/.gstack/analytics -maxdepth 1 -name '.pending-*'");
|
|
}
|
|
});
|
|
|
|
test('bash blocks with shell globs are zsh-safe (setopt guard or find)', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
const bashBlocks = [...content.matchAll(/```bash\n([\s\S]*?)```/g)].map(m => m[1]);
|
|
|
|
for (const block of bashBlocks) {
|
|
const lines = block.split('\n');
|
|
|
|
for (const line of lines) {
|
|
const trimmed = line.trimStart();
|
|
if (trimmed.startsWith('#')) continue;
|
|
if (!trimmed.includes('*')) continue;
|
|
// Skip lines where * is inside find -name, git pathspecs, or $(find)
|
|
if (/\bfind\b/.test(trimmed)) continue;
|
|
if (/\bgit\b/.test(trimmed)) continue;
|
|
if (/\$\(find\b/.test(trimmed)) continue;
|
|
|
|
// Check 1: "for VAR in <glob>" must use $(find ...) — caught above by the
|
|
// $(find check, so any surviving for-in with a glob pattern is a violation
|
|
if (/\bfor\s+\w+\s+in\b/.test(trimmed) && /\*\./.test(trimmed)) {
|
|
throw new Error(
|
|
`Unsafe for-in glob in ${skill.dir}/SKILL.md: "${trimmed}". ` +
|
|
`Use \`for f in $(find ... -name '*.ext')\` for zsh compatibility.`
|
|
);
|
|
}
|
|
|
|
// Check 2: ls/cat/rm/grep with glob file args must have setopt guard
|
|
const isGlobCmd = /\b(?:ls|cat|rm|grep)\b/.test(trimmed) &&
|
|
/(?:\/\*[a-z.*]|\*\.[a-z])/.test(trimmed);
|
|
if (isGlobCmd) {
|
|
expect(block).toContain('setopt +o nomatch');
|
|
}
|
|
}
|
|
}
|
|
}
|
|
});
|
|
|
|
test('preamble-using skills have correct skill name in telemetry', () => {
|
|
const PREAMBLE_SKILLS = [
|
|
{ dir: '.', name: 'gstack' },
|
|
{ dir: 'ship', name: 'ship' },
|
|
{ dir: 'review', name: 'review' },
|
|
{ dir: 'qa', name: 'qa' },
|
|
{ dir: 'retro', name: 'retro' },
|
|
];
|
|
for (const skill of PREAMBLE_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain(`"skill":"${skill.name}"`);
|
|
}
|
|
});
|
|
|
|
test('qa and qa-only templates use QA_METHODOLOGY placeholder', () => {
|
|
const qaTmpl = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(qaTmpl).toContain('{{QA_METHODOLOGY}}');
|
|
|
|
const qaOnlyTmpl = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(qaOnlyTmpl).toContain('{{QA_METHODOLOGY}}');
|
|
});
|
|
|
|
test('QA_METHODOLOGY appears expanded in both qa and qa-only generated files', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
|
|
|
|
// Both should contain the health score rubric
|
|
expect(qaContent).toContain('Health Score Rubric');
|
|
expect(qaOnlyContent).toContain('Health Score Rubric');
|
|
|
|
// Both should contain framework guidance
|
|
expect(qaContent).toContain('Framework-Specific Guidance');
|
|
expect(qaOnlyContent).toContain('Framework-Specific Guidance');
|
|
|
|
// Both should contain the important rules
|
|
expect(qaContent).toContain('Important Rules');
|
|
expect(qaOnlyContent).toContain('Important Rules');
|
|
|
|
// Both should contain the 6 phases
|
|
expect(qaContent).toContain('Phase 1');
|
|
expect(qaOnlyContent).toContain('Phase 1');
|
|
expect(qaContent).toContain('Phase 6');
|
|
expect(qaOnlyContent).toContain('Phase 6');
|
|
});
|
|
|
|
test('qa-only has no-fix guardrails', () => {
|
|
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
|
|
expect(qaOnlyContent).toContain('Never fix bugs');
|
|
expect(qaOnlyContent).toContain('NEVER fix anything');
|
|
// Should not have Edit, Glob, or Grep in allowed-tools.
|
|
// Scope to frontmatter (between the first two --- lines) — the body can
|
|
// legitimately mention these tool names in prose (e.g., Claude model
|
|
// overlay says "prefer Read, Edit, Write, Glob, Grep over Bash").
|
|
const fmMatch = qaOnlyContent.match(/^---\n([\s\S]*?)\n---/);
|
|
expect(fmMatch).not.toBeNull();
|
|
const frontmatter = fmMatch![1];
|
|
expect(frontmatter).toMatch(/allowed-tools:/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Edit/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Glob/);
|
|
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Grep/);
|
|
});
|
|
|
|
test('qa has fix-loop tools and phases', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
// Should have Edit, Glob, Grep in allowed-tools
|
|
expect(qaContent).toContain('Edit');
|
|
expect(qaContent).toContain('Glob');
|
|
expect(qaContent).toContain('Grep');
|
|
// Should have fix-loop phases
|
|
expect(qaContent).toContain('Phase 7');
|
|
expect(qaContent).toContain('Phase 8');
|
|
expect(qaContent).toContain('Fix Loop');
|
|
expect(qaContent).toContain('Triage');
|
|
expect(qaContent).toContain('WTF');
|
|
});
|
|
});
|
|
|
|
describe('BASE_BRANCH_DETECT resolver', () => {
|
|
// Find a generated SKILL.md that uses the placeholder (ship is guaranteed to)
|
|
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
|
|
test('resolver output contains PR base detection command', () => {
|
|
expect(shipContent).toContain('gh pr view --json baseRefName');
|
|
});
|
|
|
|
test('resolver output contains repo default branch detection command', () => {
|
|
expect(shipContent).toContain('gh repo view --json defaultBranchRef');
|
|
});
|
|
|
|
test('resolver output contains fallback to main', () => {
|
|
expect(shipContent).toMatch(/fall\s*back\s+to\s+`main`/i);
|
|
});
|
|
|
|
test('resolver output uses "the base branch" phrasing', () => {
|
|
expect(shipContent).toContain('the base branch');
|
|
});
|
|
|
|
test('resolver output contains GitLab CLI commands', () => {
|
|
expect(shipContent).toContain('glab');
|
|
});
|
|
|
|
test('resolver output contains git-native fallback', () => {
|
|
expect(shipContent).toContain('git symbolic-ref');
|
|
});
|
|
|
|
test('resolver output mentions GitLab platform', () => {
|
|
expect(shipContent).toMatch(/gitlab/i);
|
|
});
|
|
});
|
|
|
|
describe('GitLab support in generated skills', () => {
|
|
const retroContent = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
|
|
const shipSkillContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
|
|
test('retro contains GitLab MR number extraction', () => {
|
|
expect(retroContent).toContain('[#!]');
|
|
});
|
|
|
|
test('retro uses BASE_BRANCH_DETECT (contains glab)', () => {
|
|
expect(retroContent).toContain('glab');
|
|
});
|
|
|
|
test('ship contains glab mr create', () => {
|
|
expect(shipSkillContent).toContain('glab mr create');
|
|
});
|
|
|
|
test('ship checks .gitlab-ci.yml', () => {
|
|
expect(shipSkillContent).toContain('.gitlab-ci.yml');
|
|
});
|
|
});
|
|
|
|
/**
|
|
* Quality evals — catch description regressions.
|
|
*
|
|
* These test that generated output is *useful for an AI agent*,
|
|
* not just structurally valid. Each test targets a specific
|
|
* regression we actually shipped and caught in review.
|
|
*/
|
|
describe('description quality evals', () => {
|
|
// Regression: snapshot flags lost value hints (-d <N>, -s <sel>, -o <path>)
|
|
test('snapshot flags with values include value hints in output', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
if (flag.takesValue) {
|
|
expect(flag.valueHint).toBeDefined();
|
|
expect(content).toContain(`${flag.short} ${flag.valueHint}`);
|
|
}
|
|
}
|
|
});
|
|
|
|
// Regression: "is" lost the valid states enum
|
|
test('is command lists valid state values', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['is'].description;
|
|
for (const state of ['visible', 'hidden', 'enabled', 'disabled', 'checked', 'editable', 'focused']) {
|
|
expect(desc).toContain(state);
|
|
}
|
|
});
|
|
|
|
// Regression: "press" lost common key examples
|
|
test('press command lists example keys', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['press'].description;
|
|
expect(desc).toContain('Enter');
|
|
expect(desc).toContain('Tab');
|
|
expect(desc).toContain('Escape');
|
|
});
|
|
|
|
// Regression: "console" lost --errors filter note
|
|
test('console command describes --errors behavior', () => {
|
|
const desc = COMMAND_DESCRIPTIONS['console'].description;
|
|
expect(desc).toContain('--errors');
|
|
});
|
|
|
|
// Regression: snapshot -i lost "@e refs" context
|
|
test('snapshot -i mentions @e refs', () => {
|
|
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-i')!;
|
|
expect(flag.description).toContain('@e');
|
|
});
|
|
|
|
// Regression: snapshot -C lost "@c refs" context
|
|
test('snapshot -C mentions @c refs', () => {
|
|
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-C')!;
|
|
expect(flag.description).toContain('@c');
|
|
});
|
|
|
|
// Guard: every description must be at least 8 chars (catches empty or stub descriptions)
|
|
test('all command descriptions have meaningful length', () => {
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
expect(meta.description.length).toBeGreaterThanOrEqual(8);
|
|
}
|
|
});
|
|
|
|
// Guard: snapshot flag descriptions must be at least 10 chars
|
|
test('all snapshot flag descriptions have meaningful length', () => {
|
|
for (const flag of SNAPSHOT_FLAGS) {
|
|
expect(flag.description.length).toBeGreaterThanOrEqual(10);
|
|
}
|
|
});
|
|
|
|
// Guard: descriptions must not contain pipe (breaks markdown table cells)
|
|
// Usage strings are backtick-wrapped in the table so pipes there are safe.
|
|
test('no command description contains pipe character', () => {
|
|
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
|
|
expect(meta.description).not.toContain('|');
|
|
}
|
|
});
|
|
|
|
// Guard: generated output uses → not ->
|
|
test('generated SKILL.md uses unicode arrows', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
// Check the Tips section specifically (where we regressed -> from →)
|
|
const tipsSection = content.slice(content.indexOf('## Tips'));
|
|
expect(tipsSection).toContain('→');
|
|
expect(tipsSection).not.toContain('->');
|
|
});
|
|
});
|
|
|
|
describe('REVIEW_DASHBOARD resolver', () => {
|
|
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review'];
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`review dashboard appears in ${skill} generated file`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('gstack-review');
|
|
expect(content).toContain('REVIEW READINESS DASHBOARD');
|
|
});
|
|
}
|
|
|
|
test('review dashboard appears in ship generated file', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('reviews.jsonl');
|
|
expect(content).toContain('REVIEW READINESS DASHBOARD');
|
|
});
|
|
|
|
test('dashboard treats review as a valid Eng Review source', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('plan-eng-review, review, plan-design-review');
|
|
expect(content).toContain('`review` (diff-scoped pre-landing review)');
|
|
expect(content).toContain('`plan-eng-review` (plan-stage architecture review)');
|
|
expect(content).toContain('from either \\`review\\` or \\`plan-eng-review\\`');
|
|
});
|
|
|
|
test('shared dashboard propagates review source to plan-eng-review', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('plan-eng-review, review, plan-design-review');
|
|
expect(content).toContain('`review` (diff-scoped pre-landing review)');
|
|
});
|
|
|
|
test('resolver output contains key dashboard elements', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('VERDICT');
|
|
expect(content).toContain('CLEARED');
|
|
expect(content).toContain('Eng Review');
|
|
expect(content).toContain('7 days');
|
|
expect(content).toContain('Design Review');
|
|
expect(content).toContain('skip_eng_review');
|
|
});
|
|
|
|
test('dashboard bash block includes git HEAD for staleness detection', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('git rev-parse --short HEAD');
|
|
expect(content).toContain('---HEAD---');
|
|
});
|
|
|
|
test('dashboard includes staleness detection prose', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Staleness detection');
|
|
expect(content).toContain('commit');
|
|
});
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`${skill} contains review chaining section`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Review Chaining');
|
|
});
|
|
|
|
test(`${skill} Review Log includes commit field`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('"commit"');
|
|
});
|
|
}
|
|
|
|
test('plan-ceo-review chaining mentions eng and design reviews', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-design-review');
|
|
});
|
|
|
|
test('plan-eng-review chaining mentions design and ceo reviews', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('/plan-design-review');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
});
|
|
|
|
test('plan-design-review chaining mentions eng, ceo, and design skills', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
expect(content).toContain('/design-shotgun');
|
|
expect(content).toContain('/design-html');
|
|
});
|
|
|
|
test('ship does NOT contain review chaining', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('Review Chaining');
|
|
});
|
|
});
|
|
|
|
// ─── Test Coverage Audit Resolver Tests ─────────────────────
|
|
|
|
describe('TEST_COVERAGE_AUDIT placeholders', () => {
|
|
const planSkill = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
|
|
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan and ship modes share codepath tracing methodology', () => {
|
|
// Review mode delegates test coverage to the Testing specialist subagent (Review Army)
|
|
const sharedPhrases = [
|
|
'Trace data flow',
|
|
'Diagram the execution',
|
|
'Quality scoring rubric',
|
|
'★★★',
|
|
'★★',
|
|
'GAP',
|
|
];
|
|
for (const phrase of sharedPhrases) {
|
|
expect(planSkill).toContain(phrase);
|
|
expect(shipSkill).toContain(phrase);
|
|
}
|
|
// Plan mode traces the plan, not a git diff
|
|
expect(planSkill).toContain('Trace every codepath in the plan');
|
|
expect(planSkill).not.toContain('git diff origin');
|
|
// Ship mode traces the diff
|
|
expect(shipSkill).toContain('Trace every codepath changed');
|
|
});
|
|
|
|
test('review mode uses Review Army for specialist dispatch', () => {
|
|
expect(reviewSkill).toContain('Review Army');
|
|
expect(reviewSkill).toContain('Specialist Dispatch');
|
|
expect(reviewSkill).toContain('testing.md');
|
|
});
|
|
|
|
test('plan and ship modes include E2E decision matrix', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('E2E Test Decision Matrix');
|
|
expect(skill).toContain('→E2E');
|
|
expect(skill).toContain('→EVAL');
|
|
}
|
|
});
|
|
|
|
test('plan and ship modes include regression rule', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('REGRESSION RULE');
|
|
expect(skill).toContain('IRON RULE');
|
|
}
|
|
});
|
|
|
|
test('plan and ship modes include test framework detection', () => {
|
|
// Review mode delegates to Testing specialist
|
|
for (const skill of [planSkill, shipSkill]) {
|
|
expect(skill).toContain('Test Framework Detection');
|
|
expect(skill).toContain('CLAUDE.md');
|
|
}
|
|
});
|
|
|
|
test('plan mode adds tests to plan + includes test plan artifact', () => {
|
|
expect(planSkill).toContain('Add missing tests to the plan');
|
|
expect(planSkill).toContain('eng-review-test-plan');
|
|
expect(planSkill).toContain('Test Plan Artifact');
|
|
});
|
|
|
|
test('ship mode auto-generates tests + includes before/after count', () => {
|
|
expect(shipSkill).toContain('Generate tests for uncovered paths');
|
|
expect(shipSkill).toContain('Before/after test count');
|
|
expect(shipSkill).toContain('30 code paths max');
|
|
expect(shipSkill).toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review mode uses Fix-First + Review Army for specialist coverage', () => {
|
|
expect(reviewSkill).toContain('Fix-First');
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
// Review Army handles test coverage via Testing specialist subagent
|
|
expect(reviewSkill).toContain('Review Army');
|
|
expect(reviewSkill).toContain('Testing');
|
|
});
|
|
|
|
test('plan mode does NOT include ship-specific content', () => {
|
|
expect(planSkill).not.toContain('Before/after test count');
|
|
expect(planSkill).not.toContain('30 code paths max');
|
|
expect(planSkill).not.toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review mode does NOT include test plan artifact', () => {
|
|
expect(reviewSkill).not.toContain('Test Plan Artifact');
|
|
expect(reviewSkill).not.toContain('eng-review-test-plan');
|
|
expect(reviewSkill).not.toContain('ship-test-plan');
|
|
});
|
|
|
|
test('review/specialists/ directory has all expected checklist files', () => {
|
|
const specDir = path.join(ROOT, 'review', 'specialists');
|
|
const expected = [
|
|
'testing.md',
|
|
'maintainability.md',
|
|
'security.md',
|
|
'performance.md',
|
|
'data-migration.md',
|
|
'api-contract.md',
|
|
'red-team.md',
|
|
];
|
|
for (const f of expected) {
|
|
expect(fs.existsSync(path.join(specDir, f))).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('each specialist file has standard header with scope and output format', () => {
|
|
const specDir = path.join(ROOT, 'review', 'specialists');
|
|
const files = fs.readdirSync(specDir).filter(f => f.endsWith('.md'));
|
|
for (const f of files) {
|
|
const content = fs.readFileSync(path.join(specDir, f), 'utf-8');
|
|
// All specialist files must have Scope and Output/JSON in header
|
|
expect(content).toContain('Scope:');
|
|
expect(content.toLowerCase()).toMatch(/output|json/);
|
|
// Must define NO FINDINGS behavior
|
|
expect(content).toContain('NO FINDINGS');
|
|
}
|
|
});
|
|
|
|
// Regression guard: ship output contains key phrases from before the refactor
|
|
test('ship SKILL.md regression guard — key phrases preserved', () => {
|
|
const regressionPhrases = [
|
|
'100% coverage is the goal',
|
|
'ASCII coverage diagram',
|
|
'processPayment',
|
|
'refundPayment',
|
|
'billing.test.ts',
|
|
'checkout.e2e.ts',
|
|
'COVERAGE:',
|
|
'QUALITY:',
|
|
'GAPS:',
|
|
'Code paths:',
|
|
'User flows:',
|
|
];
|
|
for (const phrase of regressionPhrases) {
|
|
expect(shipSkill).toContain(phrase);
|
|
}
|
|
});
|
|
|
|
test('ship SKILL.md contains review army specialist dispatch', () => {
|
|
expect(shipSkill).toContain('Specialist Dispatch');
|
|
expect(shipSkill).toContain('Step 9.1');
|
|
expect(shipSkill).toContain('Step 9.2');
|
|
});
|
|
|
|
test('ship SKILL.md contains cross-review finding dedup', () => {
|
|
expect(shipSkill).toContain('Cross-review finding dedup');
|
|
expect(shipSkill).toContain('Step 9.3');
|
|
});
|
|
|
|
test('ship SKILL.md contains re-run idempotency behavior', () => {
|
|
expect(shipSkill).toContain('Re-run behavior (idempotency)');
|
|
expect(shipSkill).toContain('Never skip a verification step');
|
|
});
|
|
});
|
|
|
|
// --- {{TEST_FAILURE_TRIAGE}} resolver tests ---
|
|
|
|
describe('TEST_FAILURE_TRIAGE resolver', () => {
|
|
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
|
|
test('contains all 4 triage steps', () => {
|
|
expect(shipSkill).toContain('Step T1: Classify each failure');
|
|
expect(shipSkill).toContain('Step T2: Handle in-branch failures');
|
|
expect(shipSkill).toContain('Step T3: Handle pre-existing failures');
|
|
expect(shipSkill).toContain('Step T4: Execute the chosen action');
|
|
});
|
|
|
|
test('T1 includes classification criteria (in-branch vs pre-existing)', () => {
|
|
expect(shipSkill).toContain('In-branch');
|
|
expect(shipSkill).toContain('Likely pre-existing');
|
|
expect(shipSkill).toContain('git diff origin/');
|
|
});
|
|
|
|
test('T3 branches on REPO_MODE (solo vs collaborative)', () => {
|
|
expect(shipSkill).toContain('REPO_MODE');
|
|
expect(shipSkill).toContain('solo');
|
|
expect(shipSkill).toContain('collaborative');
|
|
});
|
|
|
|
test('solo mode offers fix-now, TODO, and skip options', () => {
|
|
expect(shipSkill).toContain('Investigate and fix now');
|
|
expect(shipSkill).toContain('Add as P0 TODO');
|
|
expect(shipSkill).toContain('Skip');
|
|
});
|
|
|
|
test('collaborative mode offers blame + assign option', () => {
|
|
expect(shipSkill).toContain('Blame + assign GitHub issue');
|
|
expect(shipSkill).toContain('gh issue create');
|
|
});
|
|
|
|
test('defaults ambiguous failures to in-branch (safety)', () => {
|
|
expect(shipSkill).toContain('When ambiguous, default to in-branch');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_FILE_REVIEW_REPORT}} resolver tests ---
|
|
|
|
describe('PLAN_FILE_REVIEW_REPORT resolver', () => {
|
|
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review', 'codex'];
|
|
|
|
for (const skill of REVIEW_SKILLS) {
|
|
test(`plan file review report appears in ${skill} generated file`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK REVIEW REPORT');
|
|
});
|
|
}
|
|
|
|
test('resolver output contains key report elements', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Trigger');
|
|
expect(content).toContain('Findings');
|
|
expect(content).toContain('VERDICT');
|
|
expect(content).toContain('/plan-ceo-review');
|
|
expect(content).toContain('/plan-eng-review');
|
|
expect(content).toContain('/plan-design-review');
|
|
expect(content).toContain('/codex review');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_COMPLETION_AUDIT}} resolver tests ---
|
|
|
|
describe('PLAN_COMPLETION_AUDIT placeholders', () => {
|
|
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains plan completion audit step', () => {
|
|
expect(shipSkill).toContain('Plan Completion Audit');
|
|
expect(shipSkill).toContain('Step 8');
|
|
});
|
|
|
|
test('review SKILL.md contains plan completion in scope drift', () => {
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Actionable Item Extraction');
|
|
expect(reviewSkill).toContain('Integration with Scope Drift Detection');
|
|
});
|
|
|
|
test('both modes share plan file discovery methodology', () => {
|
|
expect(shipSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
// Both should have conversation context first
|
|
expect(shipSkill).toContain('Conversation context (primary)');
|
|
expect(reviewSkill).toContain('Conversation context (primary)');
|
|
// Both should have grep fallback
|
|
expect(shipSkill).toContain('Content-based search (fallback)');
|
|
expect(reviewSkill).toContain('Content-based search (fallback)');
|
|
});
|
|
|
|
test('ship mode has gate logic for NOT DONE items', () => {
|
|
expect(shipSkill).toContain('NOT DONE');
|
|
expect(shipSkill).toContain('Stop — implement the missing items');
|
|
expect(shipSkill).toContain('Ship anyway — defer');
|
|
expect(shipSkill).toContain('intentionally dropped');
|
|
});
|
|
|
|
test('review mode is INFORMATIONAL only', () => {
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
expect(reviewSkill).toContain('MISSING REQUIREMENTS');
|
|
expect(reviewSkill).toContain('SCOPE CREEP');
|
|
});
|
|
|
|
test('item extraction has 50-item cap', () => {
|
|
expect(shipSkill).toContain('at most 50 items');
|
|
});
|
|
|
|
test('uses file-level traceability (not commit-level)', () => {
|
|
expect(shipSkill).toContain('Cite the specific file');
|
|
expect(shipSkill).not.toContain('commit-level traceability');
|
|
});
|
|
});
|
|
|
|
// --- {{PLAN_VERIFICATION_EXEC}} resolver tests ---
|
|
|
|
describe('PLAN_VERIFICATION_EXEC placeholder', () => {
|
|
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains plan verification step', () => {
|
|
expect(shipSkill).toContain('Step 8.1');
|
|
expect(shipSkill).toContain('Plan Verification');
|
|
});
|
|
|
|
test('references /qa-only invocation', () => {
|
|
expect(shipSkill).toContain('qa-only/SKILL.md');
|
|
expect(shipSkill).toContain('qa-only');
|
|
});
|
|
|
|
test('contains localhost reachability check', () => {
|
|
expect(shipSkill).toContain('localhost:3000');
|
|
expect(shipSkill).toContain('NO_SERVER');
|
|
});
|
|
|
|
test('skips gracefully when no verification section', () => {
|
|
expect(shipSkill).toContain('No verification steps found in plan');
|
|
});
|
|
|
|
test('skips gracefully when no dev server', () => {
|
|
expect(shipSkill).toContain('No dev server detected');
|
|
});
|
|
});
|
|
|
|
// --- Coverage gate tests ---
|
|
|
|
describe('Coverage gate in ship', () => {
|
|
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains coverage gate with thresholds', () => {
|
|
expect(shipSkill).toContain('Coverage gate');
|
|
expect(shipSkill).toContain('>= target');
|
|
expect(shipSkill).toContain('< minimum');
|
|
});
|
|
|
|
test('ship SKILL.md supports configurable thresholds via CLAUDE.md', () => {
|
|
expect(shipSkill).toContain('## Test Coverage');
|
|
expect(shipSkill).toContain('Minimum:');
|
|
expect(shipSkill).toContain('Target:');
|
|
});
|
|
|
|
test('coverage gate skips on parse failure (not block)', () => {
|
|
expect(shipSkill).toContain('could not determine percentage — skipping');
|
|
});
|
|
|
|
test('review SKILL.md delegates coverage to Testing specialist', () => {
|
|
// Coverage audit moved to Testing specialist subagent in Review Army
|
|
expect(reviewSkill).toContain('testing.md');
|
|
expect(reviewSkill).toContain('INFORMATIONAL');
|
|
});
|
|
});
|
|
|
|
// --- Ship metrics logging ---
|
|
|
|
describe('Ship metrics logging', () => {
|
|
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains metrics persistence step', () => {
|
|
expect(shipSkill).toContain('Step 20');
|
|
expect(shipSkill).toContain('coverage_pct');
|
|
expect(shipSkill).toContain('plan_items_total');
|
|
expect(shipSkill).toContain('plan_items_done');
|
|
expect(shipSkill).toContain('verification_result');
|
|
});
|
|
});
|
|
|
|
// --- Plan file discovery shared helper ---
|
|
|
|
describe('Plan file discovery shared helper', () => {
|
|
// The shared helper should appear in ship (via PLAN_COMPLETION_AUDIT_SHIP)
|
|
// and in review (via PLAN_COMPLETION_AUDIT_REVIEW)
|
|
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan file discovery appears in both ship and review', () => {
|
|
expect(shipSkill).toContain('Plan File Discovery');
|
|
expect(reviewSkill).toContain('Plan File Discovery');
|
|
});
|
|
|
|
test('both include conversation context first', () => {
|
|
expect(shipSkill).toContain('Conversation context (primary)');
|
|
expect(reviewSkill).toContain('Conversation context (primary)');
|
|
});
|
|
|
|
test('both include content-based fallback', () => {
|
|
expect(shipSkill).toContain('Content-based search (fallback)');
|
|
expect(reviewSkill).toContain('Content-based search (fallback)');
|
|
});
|
|
});
|
|
|
|
// --- Retro plan completion ---
|
|
|
|
describe('Retro plan completion section', () => {
|
|
const retroSkill = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
|
|
|
|
test('retro SKILL.md contains plan completion section', () => {
|
|
expect(retroSkill).toContain('### Plan Completion');
|
|
expect(retroSkill).toContain('plan_items_total');
|
|
expect(retroSkill).toContain('Plan Completion This Period');
|
|
});
|
|
});
|
|
|
|
// --- Plan status footer in preamble ---
|
|
|
|
describe('Plan status footer in preamble', () => {
|
|
test('preamble contains plan status footer as neutral forward reference to EXIT PLAN MODE GATE', () => {
|
|
// Read any skill that uses PREAMBLE
|
|
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Plan Status Footer');
|
|
expect(content).toContain('GSTACK REVIEW REPORT');
|
|
expect(content).toContain('ExitPlanMode');
|
|
expect(content).toContain('EXIT PLAN MODE GATE');
|
|
// The preamble must NOT impose review-report rules on operational skills
|
|
// that have no review report. It's a forward reference, not enforcement.
|
|
expect(content).not.toContain('NO REVIEWS YET');
|
|
});
|
|
});
|
|
|
|
// --- make-pdf setup ordering ---
|
|
|
|
describe('make-pdf setup ordering', () => {
|
|
test('MAKE-PDF SETUP appears before generic preamble footer sections', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'make-pdf', 'SKILL.md'), 'utf-8');
|
|
const preambleIdx = content.indexOf('## Preamble (run first)');
|
|
const setupIdx = content.indexOf('## MAKE-PDF SETUP');
|
|
const planModeIdx = content.indexOf('## Plan Mode Safe Operations');
|
|
const telemetryIdx = content.indexOf('## Telemetry (run last)');
|
|
const workflowIdx = content.indexOf('# make-pdf: publication-quality PDFs from markdown');
|
|
|
|
expect(preambleIdx).toBeGreaterThanOrEqual(0);
|
|
expect(setupIdx).toBeGreaterThan(preambleIdx);
|
|
expect(setupIdx).toBeLessThan(planModeIdx);
|
|
expect(setupIdx).toBeLessThan(telemetryIdx);
|
|
expect(setupIdx).toBeLessThan(workflowIdx);
|
|
expect(content.match(/^## MAKE-PDF SETUP/gm)?.length ?? 0).toBe(1);
|
|
});
|
|
});
|
|
|
|
// --- Skill invocation during plan mode in preamble ---
|
|
|
|
describe('Skill invocation during plan mode in preamble', () => {
|
|
test('preamble contains skill invocation plan mode section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Skill Invocation During Plan Mode');
|
|
expect(content).toContain('precedence over generic plan mode behavior');
|
|
expect(content).toContain('Do not continue the workflow');
|
|
expect(content).toContain('cancel the skill or leave plan mode');
|
|
});
|
|
});
|
|
|
|
// --- {{SPEC_REVIEW_LOOP}} resolver tests ---
|
|
|
|
describe('SPEC_REVIEW_LOOP resolver', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
|
|
|
|
test('contains all 5 review dimensions', () => {
|
|
for (const dim of ['Completeness', 'Consistency', 'Clarity', 'Scope', 'Feasibility']) {
|
|
expect(content).toContain(dim);
|
|
}
|
|
});
|
|
|
|
test('references Agent tool for subagent dispatch', () => {
|
|
expect(content).toMatch(/Agent.*tool/i);
|
|
});
|
|
|
|
test('specifies max 3 iterations', () => {
|
|
expect(content).toMatch(/3.*iteration|maximum.*3/i);
|
|
});
|
|
|
|
test('includes quality score', () => {
|
|
expect(content).toContain('quality score');
|
|
});
|
|
|
|
test('includes metrics path', () => {
|
|
expect(content).toContain('spec-review.jsonl');
|
|
});
|
|
|
|
test('includes convergence guard', () => {
|
|
expect(content).toMatch(/[Cc]onvergence/);
|
|
});
|
|
|
|
test('includes graceful failure handling', () => {
|
|
expect(content).toMatch(/skip.*review|unavailable/i);
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_SKETCH}} resolver tests ---
|
|
|
|
describe('DESIGN_SKETCH resolver', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
|
|
|
|
test('references DESIGN.md for design system constraints', () => {
|
|
expect(content).toContain('DESIGN.md');
|
|
});
|
|
|
|
test('contains wireframe or sketch terminology', () => {
|
|
expect(content).toMatch(/wireframe|sketch/i);
|
|
});
|
|
|
|
test('references browse binary for rendering', () => {
|
|
expect(content).toContain('$B goto');
|
|
});
|
|
|
|
test('references screenshot capture', () => {
|
|
expect(content).toContain('$B screenshot');
|
|
});
|
|
|
|
test('specifies rough aesthetic', () => {
|
|
expect(content).toMatch(/[Rr]ough|hand-drawn/);
|
|
});
|
|
|
|
test('includes skip conditions', () => {
|
|
expect(content).toMatch(/no UI component|skip/i);
|
|
});
|
|
});
|
|
|
|
// --- {{CODEX_SECOND_OPINION}} resolver tests ---
|
|
|
|
describe('CODEX_SECOND_OPINION resolver', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
|
|
const codexContent = fs.readFileSync(path.join(ROOT, '.agents', 'skills', 'gstack-office-hours', 'SKILL.md'), 'utf-8');
|
|
|
|
test('Phase 3.5 section appears in office-hours SKILL.md', () => {
|
|
expect(content).toContain('Phase 3.5: Cross-Model Second Opinion');
|
|
});
|
|
|
|
test('contains codex exec invocation', () => {
|
|
expect(content).toContain('codex exec');
|
|
});
|
|
|
|
test('contains opt-in AskUserQuestion text', () => {
|
|
expect(content).toContain('second opinion from an independent AI perspective');
|
|
});
|
|
|
|
test('contains cross-model synthesis instructions', () => {
|
|
expect(content).toMatch(/[Ss]ynthesis/);
|
|
expect(content).toContain('Where Claude agrees with the second opinion');
|
|
});
|
|
|
|
test('contains Claude subagent fallback', () => {
|
|
expect(content).toContain('CODEX_NOT_AVAILABLE');
|
|
expect(content).toContain('Agent tool');
|
|
expect(content).toContain('SECOND OPINION (Claude subagent)');
|
|
});
|
|
|
|
test('contains premise revision check', () => {
|
|
expect(content).toContain('Codex challenged premise');
|
|
});
|
|
|
|
test('contains error handling for auth, timeout, and empty', () => {
|
|
expect(content).toMatch(/[Aa]uth.*fail/);
|
|
expect(content).toMatch(/[Tt]imeout/);
|
|
expect(content).toMatch(/[Ee]mpty response/);
|
|
});
|
|
|
|
test('Codex host variant does NOT contain the Phase 3.5 resolver output', () => {
|
|
// The resolver returns '' for codex host, so the interactive section is stripped.
|
|
// Static template references to "Phase 3.5" in prose/conditionals are fine.
|
|
// Other resolvers (design review lite) may contain CODEX_NOT_AVAILABLE, so we
|
|
// check for Phase 3.5-specific markers only.
|
|
expect(codexContent).not.toContain('Phase 3.5: Cross-Model Second Opinion');
|
|
expect(codexContent).not.toContain('TMPERR_OH');
|
|
expect(codexContent).not.toContain('gstack-codex-oh-');
|
|
});
|
|
});
|
|
|
|
// --- Codex filesystem boundary tests ---
|
|
|
|
describe('Codex filesystem boundary', () => {
|
|
// Skills that call codex exec/review and should contain boundary text
|
|
const CODEX_CALLING_SKILLS = [
|
|
'codex', // /codex skill — 3 modes
|
|
'autoplan', // /autoplan — CEO/design/eng voices
|
|
'review', // /review — adversarial step resolver
|
|
'ship', // /ship — adversarial step resolver
|
|
'plan-eng-review', // outside voice resolver
|
|
'plan-ceo-review', // outside voice resolver
|
|
'office-hours', // second opinion resolver
|
|
];
|
|
|
|
const BOUNDARY_MARKER = 'Do NOT read or execute any';
|
|
|
|
test('boundary instruction appears in all skills that call codex', () => {
|
|
for (const skill of CODEX_CALLING_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain(BOUNDARY_MARKER);
|
|
}
|
|
});
|
|
|
|
test('codex skill has Filesystem Boundary section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('## Filesystem Boundary');
|
|
expect(content).toContain('skill definitions meant for a different AI system');
|
|
});
|
|
|
|
test('codex skill has rabbit-hole detection rule', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Detect skill-file rabbit holes');
|
|
expect(content).toContain('gstack-update-check');
|
|
expect(content).toContain('Consider retrying');
|
|
});
|
|
|
|
test('review.ts CODEX_BOUNDARY constant is interpolated into resolver output', () => {
|
|
// The adversarial step resolver should include boundary text in codex exec prompts
|
|
const reviewContent = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
// Boundary should appear near codex exec invocations
|
|
const boundaryIdx = reviewContent.indexOf(BOUNDARY_MARKER);
|
|
const codexExecIdx = reviewContent.indexOf('codex exec');
|
|
// Both must exist and boundary must come before a codex exec call
|
|
expect(boundaryIdx).toBeGreaterThan(-1);
|
|
expect(codexExecIdx).toBeGreaterThan(-1);
|
|
});
|
|
|
|
test('autoplan boundary text avoids host-specific paths for cross-host compatibility', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'autoplan', 'SKILL.md.tmpl'), 'utf-8');
|
|
// autoplan template uses generic 'skills/gstack' pattern instead of host-specific
|
|
// paths like ~/.claude/ or .agents/skills (which break Codex/Claude output tests)
|
|
const boundaryStart = content.indexOf('Filesystem Boundary');
|
|
const boundaryEnd = content.indexOf('---', boundaryStart + 1);
|
|
const boundarySection = content.slice(boundaryStart, boundaryEnd);
|
|
expect(boundarySection).not.toContain('~/.claude/');
|
|
expect(boundarySection).not.toContain('.agents/skills');
|
|
expect(boundarySection).toContain('skills/gstack');
|
|
expect(boundarySection).toContain(BOUNDARY_MARKER);
|
|
});
|
|
});
|
|
|
|
// --- {{BENEFITS_FROM}} resolver tests ---
|
|
|
|
describe('BENEFITS_FROM resolver', () => {
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
const engContent = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan-ceo-review contains prerequisite skill offer', () => {
|
|
expect(ceoContent).toContain('Prerequisite Skill Offer');
|
|
expect(ceoContent).toContain('/office-hours');
|
|
});
|
|
|
|
test('plan-eng-review contains prerequisite skill offer', () => {
|
|
expect(engContent).toContain('Prerequisite Skill Offer');
|
|
expect(engContent).toContain('/office-hours');
|
|
});
|
|
|
|
test('offer includes graceful decline', () => {
|
|
expect(ceoContent).toContain('No worries');
|
|
});
|
|
|
|
test('skills without benefits-from do NOT have prerequisite offer', () => {
|
|
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
|
|
expect(qaContent).not.toContain('Prerequisite Skill Offer');
|
|
});
|
|
|
|
test('inline invocation — no "another window" language', () => {
|
|
expect(ceoContent).not.toContain('another window');
|
|
expect(engContent).not.toContain('another window');
|
|
});
|
|
|
|
test('inline invocation — read-and-follow path present', () => {
|
|
expect(ceoContent).toContain('office-hours/SKILL.md');
|
|
expect(engContent).toContain('office-hours/SKILL.md');
|
|
});
|
|
|
|
test('BENEFITS_FROM delegates to INVOKE_SKILL pattern', () => {
|
|
// Should contain the INVOKE_SKILL-style loading prose (not the old manual skip list)
|
|
expect(engContent).toContain('Follow its instructions from top to bottom');
|
|
expect(engContent).toContain('skipping these sections');
|
|
expect(ceoContent).toContain('Follow its instructions from top to bottom');
|
|
});
|
|
});
|
|
|
|
// --- {{INVOKE_SKILL}} resolver tests ---
|
|
|
|
describe('INVOKE_SKILL resolver', () => {
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
|
|
test('plan-ceo-review uses INVOKE_SKILL for mid-session office-hours fallback', () => {
|
|
// The mid-session detection path should use INVOKE_SKILL-generated prose
|
|
expect(ceoContent).toContain('office-hours/SKILL.md');
|
|
expect(ceoContent).toContain('Follow its instructions from top to bottom');
|
|
});
|
|
|
|
test('INVOKE_SKILL output includes default skip list', () => {
|
|
expect(ceoContent).toContain('Preamble (run first)');
|
|
expect(ceoContent).toContain('Telemetry (run last)');
|
|
expect(ceoContent).toContain('AskUserQuestion Format');
|
|
});
|
|
|
|
test('INVOKE_SKILL output includes error handling', () => {
|
|
expect(ceoContent).toContain('If unreadable');
|
|
expect(ceoContent).toContain('Could not load');
|
|
});
|
|
|
|
test('template uses {{INVOKE_SKILL:office-hours}} placeholder', () => {
|
|
const tmpl = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(tmpl).toContain('{{INVOKE_SKILL:office-hours}}');
|
|
});
|
|
});
|
|
|
|
// --- {{CHANGELOG_WORKFLOW}} resolver tests ---
|
|
|
|
describe('CHANGELOG_WORKFLOW resolver', () => {
|
|
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
|
|
test('ship SKILL.md contains changelog workflow', () => {
|
|
expect(shipContent).toContain('CHANGELOG (auto-generate)');
|
|
expect(shipContent).toContain('git log <base>..HEAD --oneline');
|
|
});
|
|
|
|
test('changelog workflow includes cross-check step', () => {
|
|
expect(shipContent).toContain('Cross-check');
|
|
expect(shipContent).toContain('Every commit must map to at least one bullet point');
|
|
});
|
|
|
|
test('changelog workflow includes voice guidance', () => {
|
|
expect(shipContent).toContain('Lead with what the user can now **do**');
|
|
});
|
|
|
|
test('template uses {{CHANGELOG_WORKFLOW}} placeholder', () => {
|
|
const tmpl = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md.tmpl'), 'utf-8');
|
|
expect(tmpl).toContain('{{CHANGELOG_WORKFLOW}}');
|
|
// Should NOT contain the old inline changelog content
|
|
expect(tmpl).not.toContain('Group commits by theme');
|
|
});
|
|
|
|
test('changelog workflow includes keep-changelog format', () => {
|
|
expect(shipContent).toContain('### Added');
|
|
expect(shipContent).toContain('### Fixed');
|
|
});
|
|
});
|
|
|
|
// --- Parameterized resolver infrastructure tests ---
|
|
|
|
describe('parameterized resolver support', () => {
|
|
test('gen-skill-docs regex handles colon-separated args', () => {
|
|
// Verify the template containing {{INVOKE_SKILL:office-hours}} was processed
|
|
// without leaving unresolved placeholders
|
|
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
expect(ceoContent).not.toMatch(/\{\{INVOKE_SKILL:[^}]+\}\}/);
|
|
});
|
|
|
|
test('templates with parameterized resolvers pass unresolved check', () => {
|
|
// All generated SKILL.md files should have no unresolved {{...}} placeholders
|
|
const skillDirs = fs.readdirSync(ROOT).filter(d =>
|
|
fs.existsSync(path.join(ROOT, d, 'SKILL.md'))
|
|
);
|
|
for (const dir of skillDirs) {
|
|
const content = fs.readFileSync(path.join(ROOT, dir, 'SKILL.md'), 'utf-8');
|
|
const unresolved = content.match(/\{\{[A-Z_]+(?::[^}]*)?\}\}/g);
|
|
if (unresolved) {
|
|
throw new Error(`${dir}/SKILL.md has unresolved placeholders: ${unresolved.join(', ')}`);
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
// --- Preamble routing injection tests ---
|
|
|
|
describe('preamble routing injection', () => {
|
|
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
|
|
test('preamble bash checks for routing section in CLAUDE.md', () => {
|
|
expect(shipContent).toContain('grep -q "## Skill routing" CLAUDE.md');
|
|
expect(shipContent).toContain('HAS_ROUTING');
|
|
});
|
|
|
|
test('preamble bash reads routing_declined config', () => {
|
|
expect(shipContent).toContain('routing_declined');
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
});
|
|
|
|
test('preamble includes routing injection AskUserQuestion', () => {
|
|
expect(shipContent).toContain('Add routing rules to CLAUDE.md');
|
|
expect(shipContent).toContain("I'll invoke skills manually");
|
|
});
|
|
|
|
test('routing injection respects prior decline', () => {
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
expect(shipContent).toMatch(/routing_declined.*true/);
|
|
});
|
|
|
|
test('routing injection only fires when all conditions met', () => {
|
|
// Must be: HAS_ROUTING=no AND ROUTING_DECLINED=false AND PROACTIVE_PROMPTED=yes
|
|
expect(shipContent).toContain('HAS_ROUTING');
|
|
expect(shipContent).toContain('ROUTING_DECLINED');
|
|
expect(shipContent).toContain('PROACTIVE_PROMPTED');
|
|
});
|
|
|
|
test('routing section content includes key routing rules', () => {
|
|
expect(shipContent).toContain('invoke /office-hours');
|
|
expect(shipContent).toContain('invoke /investigate');
|
|
expect(shipContent).toContain('invoke /ship');
|
|
expect(shipContent).toContain('invoke /qa');
|
|
});
|
|
|
|
test('routing section uses renamed checkpoint skills (not stale /checkpoint)', () => {
|
|
expect(shipContent).toContain('invoke /context-save');
|
|
expect(shipContent).toContain('invoke /context-restore');
|
|
expect(shipContent).not.toContain('invoke checkpoint');
|
|
});
|
|
|
|
test('routing section uses soft "when in doubt" policy, not hard "ALWAYS invoke"', () => {
|
|
expect(shipContent).toContain('When in doubt, invoke the skill');
|
|
expect(shipContent).not.toContain('Do NOT answer directly');
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_OUTSIDE_VOICES}} resolver tests ---
|
|
|
|
describe('DESIGN_OUTSIDE_VOICES resolver', () => {
|
|
test('plan-design-review contains outside voices section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('CODEX_AVAILABLE');
|
|
expect(content).toContain('LITMUS SCORECARD');
|
|
});
|
|
|
|
test('design-review contains outside voices section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('source audit');
|
|
});
|
|
|
|
test('design-consultation contains outside voices section', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-consultation', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Outside Voices');
|
|
expect(content).toContain('design direction');
|
|
});
|
|
|
|
test('branches correctly per skillName — different prompts', () => {
|
|
const planContent = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
|
|
const consultContent = fs.readFileSync(path.join(ROOT, 'design-consultation', 'SKILL.md'), 'utf-8');
|
|
// plan-design-review uses analytical prompt (high reasoning)
|
|
expect(planContent).toContain('model_reasoning_effort="high"');
|
|
// design-consultation uses creative prompt (medium reasoning)
|
|
expect(consultContent).toContain('model_reasoning_effort="medium"');
|
|
});
|
|
});
|
|
|
|
// --- {{DESIGN_HARD_RULES}} resolver tests ---
|
|
|
|
describe('DESIGN_HARD_RULES resolver', () => {
|
|
test('plan-design-review Pass 4 contains hard rules', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Hard Rules');
|
|
expect(content).toContain('Classifier');
|
|
expect(content).toContain('MARKETING/LANDING PAGE');
|
|
expect(content).toContain('APP UI');
|
|
});
|
|
|
|
test('design-review contains hard rules', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Design Hard Rules');
|
|
});
|
|
|
|
test('includes all 3 rule sets', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Landing page rules');
|
|
expect(content).toContain('App UI rules');
|
|
expect(content).toContain('Universal rules');
|
|
});
|
|
|
|
test('references shared AI slop blacklist items', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('3-column feature grid');
|
|
expect(content).toContain('Purple/violet/indigo');
|
|
});
|
|
|
|
test('includes OpenAI hard rejection criteria', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Generic SaaS card grid');
|
|
expect(content).toContain('Carousel with no narrative purpose');
|
|
});
|
|
|
|
test('includes OpenAI litmus checks', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Brand/product unmistakable');
|
|
expect(content).toContain('premium with all decorative shadows removed');
|
|
});
|
|
});
|
|
|
|
// --- Extended DESIGN_SKETCH resolver tests ---
|
|
|
|
describe('DESIGN_SKETCH extended with outside voices', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
|
|
|
|
test('contains outside design voices step', () => {
|
|
expect(content).toContain('Outside design voices');
|
|
});
|
|
|
|
test('offers opt-in via AskUserQuestion', () => {
|
|
expect(content).toContain('outside design perspectives');
|
|
});
|
|
|
|
test('still contains original wireframe steps', () => {
|
|
expect(content).toContain('wireframe');
|
|
expect(content).toContain('$B goto');
|
|
});
|
|
});
|
|
|
|
// --- Extended DESIGN_REVIEW_LITE resolver tests ---
|
|
|
|
describe('DESIGN_REVIEW_LITE extended with Codex', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
|
|
test('contains Codex design voice block', () => {
|
|
expect(content).toContain('Codex design voice');
|
|
expect(content).toContain('CODEX (design)');
|
|
});
|
|
|
|
test('still contains original checklist steps', () => {
|
|
expect(content).toContain('design-checklist.md');
|
|
expect(content).toContain('SCOPE_FRONTEND');
|
|
});
|
|
|
|
});
|
|
|
|
// ─── Codex Generation Tests ─────────────────────────────────
|
|
|
|
describe('Codex generation (--host codex)', () => {
|
|
const AGENTS_DIR = path.join(ROOT, '.agents', 'skills');
|
|
|
|
// .agents/ is gitignored (v0.11.2.0) — generate on demand for tests
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
|
|
// Dynamic discovery of expected Codex skills: all templates except /codex
|
|
// Also excludes skills where .agents/skills/{name} is a symlink back to the repo root
|
|
// (vendored dev mode — gen-skill-docs skips these to avoid overwriting Claude SKILL.md)
|
|
const CODEX_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; codexName: string }> = [];
|
|
const isSymlinkLoop = (codexName: string): boolean => {
|
|
const agentSkillDir = path.join(ROOT, '.agents', 'skills', codexName);
|
|
try {
|
|
return fs.realpathSync(agentSkillDir) === fs.realpathSync(ROOT);
|
|
} catch { return false; }
|
|
};
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
if (!isSymlinkLoop('gstack')) {
|
|
skills.push({ dir: '.', codexName: 'gstack' });
|
|
}
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (entry.name === 'codex') continue; // /codex is excluded from Codex output
|
|
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
|
|
const codexName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
|
|
if (isSymlinkLoop(codexName)) continue;
|
|
skills.push({ dir: entry.name, codexName });
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
test('--host codex generates correct output paths', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const skillMd = path.join(AGENTS_DIR, skill.codexName, 'SKILL.md');
|
|
expect(fs.existsSync(skillMd)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('root gstack bundle has OpenAI metadata for Codex skill browsing', () => {
|
|
const rootMetadata = path.join(ROOT, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(rootMetadata)).toBe(true);
|
|
const content = fs.readFileSync(rootMetadata, 'utf-8');
|
|
expect(content).toContain('display_name: "gstack"');
|
|
expect(content).toContain('Use $gstack to locate the bundled gstack skills.');
|
|
expect(content).toContain('allow_implicit_invocation: true');
|
|
});
|
|
|
|
test('externalSkillName mapping: root is gstack, others are gstack-{dir}', () => {
|
|
// Root → gstack
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack', 'SKILL.md'))).toBe(true);
|
|
// Subdirectories → gstack-{dir}
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'))).toBe(true);
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'))).toBe(true);
|
|
// gstack-upgrade doesn't double-prefix
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-upgrade', 'SKILL.md'))).toBe(true);
|
|
// No double-prefix: gstack-gstack-upgrade must NOT exist
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-gstack-upgrade', 'SKILL.md'))).toBe(false);
|
|
});
|
|
|
|
test('Codex frontmatter has ONLY name + description', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content.startsWith('---\n')).toBe(true);
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
expect(fmEnd).toBeGreaterThan(0);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
// Must have name and description
|
|
expect(frontmatter).toContain('name:');
|
|
expect(frontmatter).toContain('description:');
|
|
// Must NOT have allowed-tools, version, or hooks
|
|
expect(frontmatter).not.toContain('allowed-tools:');
|
|
expect(frontmatter).not.toContain('version:');
|
|
expect(frontmatter).not.toContain('hooks:');
|
|
}
|
|
});
|
|
|
|
test('all Codex skills have agents/openai.yaml metadata', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const metadata = path.join(AGENTS_DIR, skill.codexName, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(metadata)).toBe(true);
|
|
const content = fs.readFileSync(metadata, 'utf-8');
|
|
expect(content).toContain(`display_name: "${skill.codexName}"`);
|
|
expect(content).toContain('short_description:');
|
|
expect(content).toContain('allow_implicit_invocation: true');
|
|
}
|
|
});
|
|
|
|
test('no .claude/skills/ in Codex output', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('no ~/.claude/ paths in Codex output', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('~/.claude/');
|
|
}
|
|
});
|
|
|
|
test('/codex skill excluded from Codex output', () => {
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex'))).toBe(false);
|
|
});
|
|
|
|
test('Codex output includes Claude outside-voice skill with read-only boundary', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-claude', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('claude -p');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-prompt-');
|
|
expect(content).toContain('mktemp /tmp/gstack-claude-diff-');
|
|
expect(content).not.toContain('/tmp/gstack-claude-diff-$$');
|
|
expect(content).toContain('cat "$PROMPT_FILE" | claude -p');
|
|
expect(content).toContain('--disable-slash-commands');
|
|
expect(content).toContain('--tools ""');
|
|
expect(content).toContain('--allowedTools Read,Grep,Glob');
|
|
expect(content).toContain('--disallowedTools Bash,Edit,Write');
|
|
expect(content).toContain('is_error');
|
|
});
|
|
|
|
test('Codex review step stripped from Codex-host ship and review', () => {
|
|
const shipContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(shipContent).not.toContain('codex review --base');
|
|
expect(shipContent).not.toContain('CODEX_REVIEWS');
|
|
|
|
const reviewContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(reviewContent).not.toContain('codex review --base');
|
|
expect(reviewContent).not.toContain('CODEX_REVIEWS');
|
|
});
|
|
|
|
test('--host codex --dry-run freshness', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// Every Codex skill should be FRESH
|
|
for (const skill of CODEX_SKILLS) {
|
|
expect(output).toContain(`FRESH: .agents/skills/${skill.codexName}/SKILL.md`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('--host agents alias produces same output as --host codex', () => {
|
|
const codexResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
const agentsResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'agents', '--dry-run'], {
|
|
cwd: ROOT,
|
|
stdout: 'pipe',
|
|
stderr: 'pipe',
|
|
});
|
|
expect(codexResult.exitCode).toBe(0);
|
|
expect(agentsResult.exitCode).toBe(0);
|
|
// Both should produce the same output (same FRESH lines)
|
|
expect(codexResult.stdout.toString()).toBe(agentsResult.stdout.toString());
|
|
});
|
|
|
|
test('multiline descriptions preserved in Codex output', () => {
|
|
// office-hours has a multiline description — verify it survives the frontmatter transform
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-office-hours', 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
// Description should span multiple lines (block scalar)
|
|
const descLines = frontmatter.split('\n').filter(l => l.startsWith(' '));
|
|
expect(descLines.length).toBeGreaterThan(1);
|
|
// Verify key phrases survived
|
|
expect(frontmatter).toContain('YC Office Hours');
|
|
});
|
|
|
|
test('hook skills have safety prose and no hooks: in frontmatter', () => {
|
|
const HOOK_SKILLS = ['gstack-careful', 'gstack-freeze', 'gstack-guard'];
|
|
for (const skillName of HOOK_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skillName, 'SKILL.md'), 'utf-8');
|
|
// Must have safety advisory prose
|
|
expect(content).toContain('Safety Advisory');
|
|
// Must NOT have hooks: in frontmatter
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).not.toContain('hooks:');
|
|
}
|
|
});
|
|
|
|
test('all Codex SKILL.md files have auto-generated header', () => {
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
|
|
expect(content).toContain('Regenerate: bun run gen:skill-docs');
|
|
}
|
|
});
|
|
|
|
test('Codex preamble resolves runtime assets from repo-local or global gstack roots', () => {
|
|
// Check a skill that has a preamble (review is a good candidate)
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT');
|
|
expect(content).toContain('$_ROOT/.agents/skills/gstack');
|
|
expect(content).toContain('$GSTACK_BIN/gstack-config');
|
|
expect(content).toContain('$GSTACK_ROOT/gstack-upgrade/SKILL.md');
|
|
expect(content).not.toContain('~/.codex/skills/gstack/bin/gstack-config get telemetry');
|
|
});
|
|
|
|
// ─── Path rewriting regression tests ─────────────────────────
|
|
|
|
test('sidecar paths point to .agents/skills/gstack/review/ (not gstack-review/)', () => {
|
|
// Regression: gen-skill-docs rewrote .claude/skills/review → .agents/skills/gstack-review
|
|
// but setup puts sidecars under .agents/skills/gstack/review/. Must match setup layout.
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
// Correct: references to sidecar files use gstack/review/ path
|
|
expect(content).toContain('.agents/skills/gstack/review/checklist.md');
|
|
// design-checklist.md is now referenced via Review Army specialist (Claude only, stripped for Codex)
|
|
// Wrong: must NOT reference gstack-review/checklist.md (file doesn't exist there)
|
|
expect(content).not.toContain('.agents/skills/gstack-review/checklist.md');
|
|
});
|
|
|
|
test('sidecar paths in ship skill point to gstack/review/ for pre-landing review', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
// Ship references the review checklist in its pre-landing review step
|
|
if (content.includes('checklist.md')) {
|
|
expect(content).toContain('.agents/skills/gstack/review/');
|
|
expect(content).not.toContain('.agents/skills/gstack-review/checklist');
|
|
}
|
|
});
|
|
|
|
test('greptile-triage sidecar path is correct', () => {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
if (content.includes('greptile-triage')) {
|
|
expect(content).toContain('.agents/skills/gstack/review/greptile-triage.md');
|
|
expect(content).not.toContain('.agents/skills/gstack-review/greptile-triage');
|
|
}
|
|
});
|
|
|
|
test('all four path rewrite rules produce correct output', () => {
|
|
// Test each of the 4 path rewrite rules individually
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
|
|
// Rule 1: ~/.claude/skills/gstack → $GSTACK_ROOT
|
|
expect(content).not.toContain('~/.claude/skills/gstack');
|
|
expect(content).toContain('$GSTACK_ROOT');
|
|
|
|
// Rule 2: .claude/skills/gstack → .agents/skills/gstack
|
|
expect(content).not.toContain('.claude/skills/gstack');
|
|
|
|
// Rule 3: .claude/skills/review → .agents/skills/gstack/review
|
|
expect(content).not.toContain('.claude/skills/review');
|
|
|
|
// Rule 4: .claude/skills → .agents/skills (catch-all)
|
|
expect(content).not.toContain('.claude/skills');
|
|
});
|
|
|
|
test('path rewrite rules apply to all Codex skills with sidecar references', () => {
|
|
// Verify across ALL generated skills, not just review
|
|
for (const skill of CODEX_SKILLS) {
|
|
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
|
|
// No skill should reference Claude paths
|
|
expect(content).not.toContain('~/.claude/skills');
|
|
expect(content).not.toContain('.claude/skills');
|
|
if (content.includes('gstack-config') || content.includes('gstack-update-check') || content.includes('gstack-telemetry-log')) {
|
|
expect(content).toContain('$GSTACK_ROOT');
|
|
}
|
|
// If a skill references checklist.md, it must use the correct sidecar path
|
|
if (content.includes('checklist.md') && !content.includes('design-checklist.md')) {
|
|
expect(content).not.toContain('gstack-review/checklist.md');
|
|
}
|
|
}
|
|
});
|
|
|
|
// ─── Claude output regression guard ─────────────────────────
|
|
|
|
test('Claude output unchanged: review skill still uses .claude/skills/ paths', () => {
|
|
// Codex changes must NOT affect Claude output
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('.claude/skills/review/checklist.md');
|
|
expect(content).toContain('~/.claude/skills/gstack');
|
|
// Must NOT contain Codex paths
|
|
expect(content).not.toContain('.agents/skills');
|
|
expect(content).not.toContain('~/.codex/');
|
|
});
|
|
|
|
test('Claude output unchanged: ship skill still uses .claude/skills/ paths', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('~/.claude/skills/gstack');
|
|
expect(content).not.toContain('.agents/skills');
|
|
expect(content).not.toContain('~/.codex/');
|
|
});
|
|
|
|
test('Claude output unchanged: all Claude skills have zero Codex paths', () => {
|
|
for (const skill of CLAUDE_GENERATED_SKILLS) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
|
|
// pair-agent legitimately documents how Codex agents store credentials.
|
|
// codex + autoplan document the Codex CLI auth file (~/.codex/auth.json)
|
|
// and log path (~/.codex/logs/) — those are user-facing Codex CLI paths,
|
|
// not the gstack Codex host install path.
|
|
if (skill.dir !== 'pair-agent' && skill.dir !== 'codex' && skill.dir !== 'autoplan') {
|
|
expect(content).not.toContain('~/.codex/');
|
|
}
|
|
// gstack-upgrade legitimately references .agents/skills for cross-platform detection
|
|
if (skill.dir !== 'gstack-upgrade') {
|
|
expect(content).not.toContain('.agents/skills');
|
|
}
|
|
}
|
|
});
|
|
|
|
// ─── Design outside voices: Codex host guard ─────────────────
|
|
|
|
test('codex host produces empty outside voices in design-review', () => {
|
|
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-design-review', 'SKILL.md'), 'utf-8');
|
|
expect(codexContent).not.toContain('Design Outside Voices');
|
|
});
|
|
|
|
test('codex host does not include Codex design block in ship', () => {
|
|
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(codexContent).not.toContain('Codex design voice');
|
|
});
|
|
});
|
|
|
|
// ─── Factory generation tests ────────────────────────────────
|
|
|
|
describe('Factory generation (--host factory)', () => {
|
|
const FACTORY_DIR = path.join(ROOT, '.factory', 'skills');
|
|
|
|
// Generate Factory output for tests
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
|
|
const FACTORY_SKILLS = (() => {
|
|
const skills: Array<{ dir: string; factoryName: string }> = [];
|
|
const isSymlinkLoop = (name: string): boolean => {
|
|
const factorySkillDir = path.join(ROOT, '.factory', 'skills', name);
|
|
try { return fs.realpathSync(factorySkillDir) === fs.realpathSync(ROOT); }
|
|
catch { return false; }
|
|
};
|
|
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
|
|
if (!isSymlinkLoop('gstack')) skills.push({ dir: '.', factoryName: 'gstack' });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
if (entry.name === 'codex') continue;
|
|
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
|
|
const factoryName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
|
|
if (isSymlinkLoop(factoryName)) continue;
|
|
skills.push({ dir: entry.name, factoryName });
|
|
}
|
|
return skills;
|
|
})();
|
|
|
|
test('--host factory generates correct output paths', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const skillMd = path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md');
|
|
expect(fs.existsSync(skillMd)).toBe(true);
|
|
}
|
|
});
|
|
|
|
test('Factory frontmatter has name + description + user-invocable', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).toContain('name:');
|
|
expect(frontmatter).toContain('description:');
|
|
expect(frontmatter).toContain('user-invocable: true');
|
|
expect(frontmatter).not.toContain('allowed-tools:');
|
|
expect(frontmatter).not.toContain('preamble-tier:');
|
|
expect(frontmatter).not.toContain('sensitive:');
|
|
}
|
|
});
|
|
|
|
test('sensitive skills have disable-model-invocation', () => {
|
|
const SENSITIVE = ['gstack-ship', 'gstack-land-and-deploy', 'gstack-guard', 'gstack-careful', 'gstack-freeze', 'gstack-unfreeze'];
|
|
for (const name of SENSITIVE) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).toContain('disable-model-invocation: true');
|
|
}
|
|
});
|
|
|
|
test('non-sensitive skills lack disable-model-invocation', () => {
|
|
const NON_SENSITIVE = ['gstack-qa', 'gstack-review', 'gstack-investigate', 'gstack-browse'];
|
|
for (const name of NON_SENSITIVE) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(4, fmEnd);
|
|
expect(frontmatter).not.toContain('disable-model-invocation');
|
|
}
|
|
});
|
|
|
|
test('no .claude/skills/ in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('no ~/.claude/skills/ paths in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
|
|
// ~/.claude/skills should be rewritten, but ~/.claude/plans is legitimate
|
|
// (plan directory lookup) and ~/.claude/ in codex prompts is intentional
|
|
expect(content).not.toContain('~/.claude/skills');
|
|
}
|
|
});
|
|
|
|
test('/codex skill excluded from Factory output', () => {
|
|
expect(fs.existsSync(path.join(FACTORY_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
expect(fs.existsSync(path.join(FACTORY_DIR, 'gstack-codex'))).toBe(false);
|
|
});
|
|
|
|
test('Factory keeps Codex integration blocks', () => {
|
|
// Factory users CAN use Codex second opinions (codex exec is a standalone binary)
|
|
const shipContent = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
|
|
expect(shipContent).toContain('codex');
|
|
});
|
|
|
|
test('no agents/openai.yaml in Factory output', () => {
|
|
for (const skill of FACTORY_SKILLS) {
|
|
const yamlPath = path.join(FACTORY_DIR, skill.factoryName, 'agents', 'openai.yaml');
|
|
expect(fs.existsSync(yamlPath)).toBe(false);
|
|
}
|
|
});
|
|
|
|
test('--host droid alias works', () => {
|
|
const factoryResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
const droidResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'droid', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(factoryResult.exitCode).toBe(0);
|
|
expect(droidResult.exitCode).toBe(0);
|
|
expect(factoryResult.stdout.toString()).toBe(droidResult.stdout.toString());
|
|
});
|
|
|
|
test('--host factory --dry-run freshness', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
for (const skill of FACTORY_SKILLS) {
|
|
expect(output).toContain(`FRESH: .factory/skills/${skill.factoryName}/SKILL.md`);
|
|
}
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
test('Factory preamble uses .factory paths', () => {
|
|
const content = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT');
|
|
expect(content).toContain('$_ROOT/.factory/skills/gstack');
|
|
expect(content).toContain('$GSTACK_BIN/gstack-config');
|
|
});
|
|
});
|
|
|
|
// ─── Parameterized host smoke tests (config-driven) ─────────
|
|
|
|
import { ALL_HOST_CONFIGS, getExternalHosts } from '../hosts/index';
|
|
|
|
describe('Parameterized host smoke tests', () => {
|
|
for (const hostConfig of getExternalHosts()) {
|
|
describe(`${hostConfig.displayName} (--host ${hostConfig.name})`, () => {
|
|
const hostDir = path.join(ROOT, hostConfig.hostSubdir, 'skills');
|
|
|
|
test('generates output that exists on disk', () => {
|
|
// Generated dir should exist (created by earlier bun run gen:skill-docs --host all)
|
|
if (!fs.existsSync(hostDir)) {
|
|
// Generate if not already done
|
|
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
}
|
|
expect(fs.existsSync(hostDir)).toBe(true);
|
|
const skills = fs.readdirSync(hostDir).filter(d =>
|
|
fs.existsSync(path.join(hostDir, d, 'SKILL.md'))
|
|
);
|
|
expect(skills.length).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('no .claude/skills path leakage outside repo-root sidecar symlinks', () => {
|
|
if (!fs.existsSync(hostDir)) return; // skip if not generated
|
|
const skills = fs.readdirSync(hostDir);
|
|
for (const skill of skills) {
|
|
// Dev installs may mount the repo root at host/skills/gstack as a runtime
|
|
// sidecar. The generator skips that symlink loop, so leakage checks should too.
|
|
if (isRepoRootSymlink(path.join(hostDir, skill))) continue;
|
|
const skillMd = path.join(hostDir, skill, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
// Strip bash blocks (which have legitimate fallback paths)
|
|
const noBash = content.replace(/```bash\n[\s\S]*?```/g, '');
|
|
const leaks = noBash.split('\n').filter(l => l.includes('.claude/skills'));
|
|
if (leaks.length > 0) {
|
|
throw new Error(`${skill}: .claude/skills leakage:\n${leaks.slice(0, 3).join('\n')}`);
|
|
}
|
|
}
|
|
});
|
|
|
|
test('frontmatter has name and description', () => {
|
|
if (!fs.existsSync(hostDir)) return;
|
|
const skills = fs.readdirSync(hostDir);
|
|
for (const skill of skills) {
|
|
const skillMd = path.join(hostDir, skill, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content).toMatch(/^---\n/);
|
|
expect(content).toMatch(/^name:\s/m);
|
|
expect(content).toMatch(/^description:\s/m);
|
|
}
|
|
});
|
|
|
|
test('generates Claude outside-voice skill for external hosts', () => {
|
|
const skillMd = path.join(hostDir, 'gstack-claude', 'SKILL.md');
|
|
expect(fs.existsSync(skillMd)).toBe(true);
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content).toContain('claude -p');
|
|
expect(content).toContain('--disable-slash-commands');
|
|
expect(content).toContain('--allowedTools Read,Grep,Glob');
|
|
expect(content).toContain('--disallowedTools Bash,Edit,Write');
|
|
});
|
|
|
|
test('--dry-run freshness check passes', () => {
|
|
const result = Bun.spawnSync(
|
|
['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name, '--dry-run'],
|
|
{ cwd: ROOT, stdout: 'pipe', stderr: 'pipe' }
|
|
);
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
expect(output).not.toContain('STALE');
|
|
});
|
|
|
|
if (hostConfig.generation.skipSkills?.includes('codex')) {
|
|
test('/codex skill excluded', () => {
|
|
expect(fs.existsSync(path.join(hostDir, 'gstack-codex', 'SKILL.md'))).toBe(false);
|
|
});
|
|
}
|
|
});
|
|
}
|
|
});
|
|
|
|
// ─── --host all tests ────────────────────────────────────────
|
|
|
|
describe('--host all', () => {
|
|
test('--host all generates for all registered hosts', () => {
|
|
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'all', '--dry-run'], {
|
|
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
|
|
});
|
|
expect(result.exitCode).toBe(0);
|
|
const output = result.stdout.toString();
|
|
// All hosts should appear in output
|
|
expect(output).toContain('FRESH: SKILL.md'); // claude
|
|
for (const hostConfig of getExternalHosts()) {
|
|
expect(output).toContain(`FRESH: ${hostConfig.hostSubdir}/skills/`);
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── Setup script validation ─────────────────────────────────
|
|
// These tests verify the setup script's install layout matches
|
|
// what the generator produces — catching the bug where setup
|
|
// installed Claude-format source dirs for Codex users.
|
|
|
|
describe('setup script validation', () => {
|
|
const setupContent = fs.readFileSync(path.join(ROOT, 'setup'), 'utf-8');
|
|
|
|
test('setup has separate link functions for Claude and Codex', () => {
|
|
expect(setupContent).toContain('link_claude_skill_dirs');
|
|
expect(setupContent).toContain('link_codex_skill_dirs');
|
|
// Old unified function must not exist
|
|
expect(setupContent).not.toMatch(/^link_skill_dirs\(\)/m);
|
|
});
|
|
|
|
test('Claude install uses link_claude_skill_dirs', () => {
|
|
// The Claude install section (section 4) should use the Claude function
|
|
const claudeSection = setupContent.slice(
|
|
setupContent.indexOf('# 4. Install for Claude'),
|
|
setupContent.indexOf('# 5. Install for Codex')
|
|
);
|
|
expect(claudeSection).toContain('link_claude_skill_dirs');
|
|
expect(claudeSection).not.toContain('link_codex_skill_dirs');
|
|
});
|
|
|
|
test('Codex install uses link_codex_skill_dirs', () => {
|
|
// The Codex install section (section 5) should use the Codex function
|
|
const codexSection = setupContent.slice(
|
|
setupContent.indexOf('# 5. Install for Codex'),
|
|
setupContent.indexOf('# 6. Create')
|
|
);
|
|
expect(codexSection).toContain('create_codex_runtime_root');
|
|
expect(codexSection).toContain('link_codex_skill_dirs');
|
|
expect(codexSection).not.toContain('link_claude_skill_dirs');
|
|
expect(codexSection).not.toContain('_link_or_copy "$GSTACK_DIR" "$CODEX_GSTACK"');
|
|
});
|
|
|
|
test('Codex install prefers repo-local .agents/skills when setup runs from there', () => {
|
|
expect(setupContent).toContain('SKILLS_PARENT_BASENAME');
|
|
expect(setupContent).toContain('CODEX_REPO_LOCAL=0');
|
|
expect(setupContent).toContain('[ "$SKILLS_PARENT_BASENAME" = ".agents" ]');
|
|
expect(setupContent).toContain('CODEX_REPO_LOCAL=1');
|
|
expect(setupContent).toContain('CODEX_SKILLS="$INSTALL_SKILLS_DIR"');
|
|
});
|
|
|
|
test('setup separates install path from source path for symlinked repo-local installs', () => {
|
|
expect(setupContent).toContain('INSTALL_GSTACK_DIR=');
|
|
expect(setupContent).toContain('SOURCE_GSTACK_DIR=');
|
|
expect(setupContent).toContain('INSTALL_SKILLS_DIR=');
|
|
expect(setupContent).toContain('CODEX_GSTACK="$INSTALL_GSTACK_DIR"');
|
|
expect(setupContent).toContain('link_codex_skill_dirs "$SOURCE_GSTACK_DIR" "$CODEX_SKILLS"');
|
|
});
|
|
|
|
test('Codex installs always create sidecar runtime assets for the real skill target', () => {
|
|
expect(setupContent).toContain('if [ "$INSTALL_CODEX" -eq 1 ]; then');
|
|
expect(setupContent).toContain('create_agents_sidecar "$SOURCE_GSTACK_DIR"');
|
|
});
|
|
|
|
test('link_codex_skill_dirs reads from .agents/skills/', () => {
|
|
// The Codex link function must reference .agents/skills for generated Codex skills
|
|
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('.agents/skills');
|
|
expect(fnBody).toContain('gstack*');
|
|
});
|
|
|
|
test('link_claude_skill_dirs creates real directories with absolute SKILL.md symlinks', () => {
|
|
// Claude links should be real directories with absolute SKILL.md symlinks
|
|
// to ensure Claude Code discovers them as top-level skills (not nested under gstack/)
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('mkdir -p "$target"');
|
|
// v1.36.0.0: routes through _link_or_copy helper for Windows fallback (cp on MSYS2/Git Bash).
|
|
expect(fnBody).toContain('_link_or_copy "$gstack_dir/$dir_name/SKILL.md" "$target/SKILL.md"');
|
|
});
|
|
|
|
// REGRESSION: cleanup functions must handle both old symlinks AND new real-directory pattern
|
|
test('cleanup functions handle real directories with symlinked SKILL.md', () => {
|
|
// cleanup_old_claude_symlinks must detect and remove real dirs with SKILL.md symlinks
|
|
const cleanupOldStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
|
|
const cleanupOldEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up old', cleanupOldStart));
|
|
const cleanupOldBody = setupContent.slice(cleanupOldStart, cleanupOldEnd);
|
|
expect(cleanupOldBody).toContain('-d "$old_target"');
|
|
expect(cleanupOldBody).toContain('-L "$old_target/SKILL.md"');
|
|
expect(cleanupOldBody).toContain('rm -rf "$old_target"');
|
|
|
|
// cleanup_prefixed_claude_symlinks must also handle the new pattern
|
|
const cleanupPrefixedStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
|
|
const cleanupPrefixedEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up prefixed', cleanupPrefixedStart));
|
|
const cleanupPrefixedBody = setupContent.slice(cleanupPrefixedStart, cleanupPrefixedEnd);
|
|
expect(cleanupPrefixedBody).toContain('-d "$prefixed_target"');
|
|
expect(cleanupPrefixedBody).toContain('-L "$prefixed_target/SKILL.md"');
|
|
expect(cleanupPrefixedBody).toContain('rm -rf "$prefixed_target"');
|
|
});
|
|
|
|
// REGRESSION: link function must upgrade old directory symlinks
|
|
test('link_claude_skill_dirs removes old directory symlinks before creating real dirs', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// Must check for and remove old symlinks before mkdir
|
|
expect(fnBody).toContain('if [ -L "$target" ]');
|
|
expect(fnBody).toContain('rm -f "$target"');
|
|
});
|
|
|
|
test('setup supports --host auto|claude|codex|kiro|opencode', () => {
|
|
expect(setupContent).toContain('--host');
|
|
expect(setupContent).toContain('claude|codex|kiro|factory|opencode|auto');
|
|
});
|
|
|
|
test('auto mode detects claude, codex, kiro, and opencode binaries', () => {
|
|
expect(setupContent).toContain('command -v claude');
|
|
expect(setupContent).toContain('command -v codex');
|
|
expect(setupContent).toContain('command -v kiro-cli');
|
|
expect(setupContent).toContain('command -v opencode');
|
|
});
|
|
|
|
// T1: Sidecar skip guard — prevents .agents/skills/gstack from being linked as a skill
|
|
test('link_codex_skill_dirs skips the gstack sidecar directory', () => {
|
|
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('[ "$skill_name" = "gstack" ] && continue');
|
|
});
|
|
|
|
// T2: Dynamic $GSTACK_ROOT paths in generated Codex preambles
|
|
test('generated Codex preambles use dynamic GSTACK_ROOT paths', () => {
|
|
const codexSkillDir = path.join(ROOT, '.agents', 'skills', 'gstack-ship');
|
|
if (!fs.existsSync(codexSkillDir)) return; // skip if .agents/ not generated
|
|
const content = fs.readFileSync(path.join(codexSkillDir, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('GSTACK_ROOT=');
|
|
expect(content).toContain('$GSTACK_BIN/');
|
|
});
|
|
|
|
test('setup supports --host kiro with install section and sed rewrites', () => {
|
|
expect(setupContent).toContain('INSTALL_KIRO=');
|
|
expect(setupContent).toContain('kiro-cli');
|
|
expect(setupContent).toContain('KIRO_SKILLS=');
|
|
expect(setupContent).toContain('~/.kiro/skills/gstack');
|
|
});
|
|
|
|
test('setup supports --host opencode with install section and OpenCode skill path vars', () => {
|
|
expect(setupContent).toContain('INSTALL_OPENCODE=');
|
|
expect(setupContent).toContain('OPENCODE_SKILLS="$HOME/.config/opencode/skills"');
|
|
expect(setupContent).toContain('OPENCODE_GSTACK="$OPENCODE_SKILLS/gstack"');
|
|
});
|
|
|
|
test('setup installs OpenCode skills into a nested gstack runtime root', () => {
|
|
expect(setupContent).toContain('create_opencode_runtime_root');
|
|
expect(setupContent).toContain('.opencode/skills');
|
|
expect(setupContent).toContain('review/specialists');
|
|
expect(setupContent).toContain('qa/templates');
|
|
expect(setupContent).toContain('qa/references');
|
|
expect(setupContent).toContain('dx-hall-of-fame.md');
|
|
});
|
|
|
|
test('create_agents_sidecar links runtime assets', () => {
|
|
// Sidecar must link bin, browse, review, qa
|
|
const fnStart = setupContent.indexOf('create_agents_sidecar()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('bin');
|
|
expect(fnBody).toContain('browse');
|
|
expect(fnBody).toContain('review');
|
|
expect(fnBody).toContain('qa');
|
|
});
|
|
|
|
test('create_codex_runtime_root exposes only runtime assets', () => {
|
|
const fnStart = setupContent.indexOf('create_codex_runtime_root()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', setupContent.indexOf('review/', fnStart)));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('gstack/SKILL.md');
|
|
expect(fnBody).toContain('browse/dist');
|
|
expect(fnBody).toContain('browse/bin');
|
|
expect(fnBody).toContain('gstack-upgrade/SKILL.md');
|
|
// Review runtime assets (individual files, not the whole dir)
|
|
expect(fnBody).toContain('checklist.md');
|
|
expect(fnBody).toContain('design-checklist.md');
|
|
expect(fnBody).toContain('greptile-triage.md');
|
|
expect(fnBody).toContain('TODOS-format.md');
|
|
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir" "$codex_gstack"');
|
|
});
|
|
|
|
test('direct Codex installs are migrated out of ~/.codex/skills/gstack', () => {
|
|
expect(setupContent).toContain('migrate_direct_codex_install');
|
|
expect(setupContent).toContain('$HOME/.gstack/repos/gstack');
|
|
expect(setupContent).toContain('avoid duplicate skill discovery');
|
|
});
|
|
|
|
// --- Symlink prefix tests (PR #503) ---
|
|
|
|
test('link_claude_skill_dirs applies gstack- prefix by default', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('SKILL_PREFIX');
|
|
expect(fnBody).toContain('link_name="gstack-$skill_name"');
|
|
});
|
|
|
|
test('link_claude_skill_dirs preserves already-prefixed dirs', () => {
|
|
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// gstack-* dirs should keep their name (e.g., gstack-upgrade stays gstack-upgrade)
|
|
expect(fnBody).toContain('gstack-*) link_name="$skill_name"');
|
|
});
|
|
|
|
test('setup supports --no-prefix flag', () => {
|
|
expect(setupContent).toContain('--no-prefix');
|
|
expect(setupContent).toContain('SKILL_PREFIX=0');
|
|
});
|
|
|
|
test('cleanup_old_claude_symlinks removes only gstack-pointing symlinks', () => {
|
|
expect(setupContent).toContain('cleanup_old_claude_symlinks');
|
|
const fnStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
// Should check readlink before removing
|
|
expect(fnBody).toContain('readlink');
|
|
expect(fnBody).toContain('gstack/*');
|
|
// Should skip already-prefixed dirs
|
|
expect(fnBody).toContain('gstack-*) continue');
|
|
});
|
|
|
|
test('cleanup runs before link when prefix is enabled', () => {
|
|
// In the Claude install section, cleanup should happen before linking
|
|
const claudeInstallSection = setupContent.slice(
|
|
setupContent.indexOf('INSTALL_CLAUDE'),
|
|
setupContent.lastIndexOf('link_claude_skill_dirs')
|
|
);
|
|
expect(claudeInstallSection).toContain('cleanup_old_claude_symlinks');
|
|
});
|
|
|
|
// --- Persistent config + interactive prompt tests ---
|
|
|
|
test('setup reads skill_prefix from config', () => {
|
|
expect(setupContent).toContain('get skill_prefix');
|
|
expect(setupContent).toContain('GSTACK_CONFIG');
|
|
});
|
|
|
|
test('setup supports --prefix flag', () => {
|
|
expect(setupContent).toContain('--prefix)');
|
|
expect(setupContent).toContain('SKILL_PREFIX=1; SKILL_PREFIX_FLAG=1');
|
|
});
|
|
|
|
test('--prefix and --no-prefix persist to config', () => {
|
|
expect(setupContent).toContain('set skill_prefix');
|
|
});
|
|
|
|
test('interactive prompt shows when no config', () => {
|
|
expect(setupContent).toContain('Short names');
|
|
expect(setupContent).toContain('Namespaced');
|
|
expect(setupContent).toContain('Choice [1/2]');
|
|
});
|
|
|
|
test('non-TTY defaults to flat names', () => {
|
|
// Should check if stdin is a TTY before prompting
|
|
expect(setupContent).toContain('-t 0');
|
|
});
|
|
|
|
test('cleanup_prefixed_claude_symlinks exists and uses readlink', () => {
|
|
expect(setupContent).toContain('cleanup_prefixed_claude_symlinks');
|
|
const fnStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
|
|
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
|
|
const fnBody = setupContent.slice(fnStart, fnEnd);
|
|
expect(fnBody).toContain('readlink');
|
|
expect(fnBody).toContain('gstack-$skill_name');
|
|
});
|
|
|
|
test('reverse cleanup runs before link when prefix is disabled', () => {
|
|
const claudeInstallSection = setupContent.slice(
|
|
setupContent.indexOf('INSTALL_CLAUDE'),
|
|
setupContent.lastIndexOf('link_claude_skill_dirs')
|
|
);
|
|
expect(claudeInstallSection).toContain('cleanup_prefixed_claude_symlinks');
|
|
});
|
|
|
|
test('welcome message references SKILL_PREFIX', () => {
|
|
// gstack-upgrade is always called gstack-upgrade (it's the actual dir name)
|
|
// but the welcome section should exist near the prefix logic
|
|
expect(setupContent).toContain('Run /gstack-upgrade anytime');
|
|
});
|
|
});
|
|
|
|
describe('discover-skills hidden directory filtering', () => {
|
|
test('discoverTemplates skips dot-prefixed directories', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-discover-'));
|
|
try {
|
|
// Create a hidden dir with a template (should be excluded)
|
|
fs.mkdirSync(path.join(tmpDir, '.hidden'), { recursive: true });
|
|
fs.writeFileSync(path.join(tmpDir, '.hidden', 'SKILL.md.tmpl'), '---\nname: evil\n---\ntest');
|
|
// Create a visible dir with a template (should be included)
|
|
fs.mkdirSync(path.join(tmpDir, 'visible'), { recursive: true });
|
|
fs.writeFileSync(path.join(tmpDir, 'visible', 'SKILL.md.tmpl'), '---\nname: good\n---\ntest');
|
|
|
|
const { discoverTemplates } = require('../scripts/discover-skills');
|
|
const results = discoverTemplates(tmpDir);
|
|
const dirs = results.map((r: { tmpl: string }) => r.tmpl);
|
|
|
|
expect(dirs).toContain('visible/SKILL.md.tmpl');
|
|
expect(dirs).not.toContain('.hidden/SKILL.md.tmpl');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('telemetry', () => {
|
|
test('generated SKILL.md contains telemetry start block', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('_TEL_START');
|
|
expect(content).toContain('_SESSION_ID');
|
|
expect(content).toContain('TELEMETRY:');
|
|
expect(content).toContain('TEL_PROMPTED:');
|
|
expect(content).toContain('gstack-config get telemetry');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry opt-in prompt', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('.telemetry-prompted');
|
|
expect(content).toContain('Help gstack get better');
|
|
expect(content).toContain('gstack-config set telemetry community');
|
|
expect(content).toContain('gstack-config set telemetry anonymous');
|
|
expect(content).toContain('gstack-config set telemetry off');
|
|
});
|
|
|
|
test('generated SKILL.md contains telemetry epilogue', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Telemetry (run last)');
|
|
expect(content).toContain('gstack-telemetry-log');
|
|
expect(content).toContain('_TEL_END');
|
|
expect(content).toContain('_TEL_DUR');
|
|
expect(content).toContain('SKILL_NAME');
|
|
expect(content).toContain('OUTCOME');
|
|
expect(content).toContain('PLAN MODE EXCEPTION');
|
|
});
|
|
|
|
test('generated SKILL.md contains pending marker handling', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('.pending');
|
|
expect(content).toContain('_pending_finalize');
|
|
});
|
|
|
|
test('telemetry blocks appear in all skill files that use PREAMBLE', () => {
|
|
const skills = ['qa', 'ship', 'review', 'plan-ceo-review', 'plan-eng-review', 'retro'];
|
|
for (const skill of skills) {
|
|
const skillPath = path.join(ROOT, skill, 'SKILL.md');
|
|
if (fs.existsSync(skillPath)) {
|
|
const content = fs.readFileSync(skillPath, 'utf-8');
|
|
expect(content).toContain('_TEL_START');
|
|
expect(content).toContain('Telemetry (run last)');
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('community fixes wave', () => {
|
|
// Helper to get all generated SKILL.md files
|
|
function getAllSkillMds(): Array<{ name: string; content: string }> {
|
|
const results: Array<{ name: string; content: string }> = [];
|
|
const rootPath = path.join(ROOT, 'SKILL.md');
|
|
if (fs.existsSync(rootPath)) {
|
|
results.push({ name: 'root', content: fs.readFileSync(rootPath, 'utf-8') });
|
|
}
|
|
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
|
|
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
|
|
const skillPath = path.join(ROOT, entry.name, 'SKILL.md');
|
|
if (fs.existsSync(skillPath)) {
|
|
results.push({ name: entry.name, content: fs.readFileSync(skillPath, 'utf-8') });
|
|
}
|
|
}
|
|
return results;
|
|
}
|
|
|
|
// #594 — Discoverability: every SKILL.md.tmpl description contains "gstack"
|
|
test('every SKILL.md.tmpl description contains "gstack"', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
const content = fs.readFileSync(tmplPath, 'utf-8');
|
|
const desc = extractDescription(content);
|
|
expect(desc.toLowerCase()).toContain('gstack');
|
|
}
|
|
});
|
|
|
|
// #594 — Discoverability: first line of each description is under 120 chars
|
|
test('every SKILL.md.tmpl description first line is under 120 chars', () => {
|
|
for (const skill of ALL_SKILLS) {
|
|
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
|
|
const content = fs.readFileSync(tmplPath, 'utf-8');
|
|
const desc = extractDescription(content);
|
|
const firstLine = desc.split('\n')[0];
|
|
expect(firstLine.length).toBeLessThanOrEqual(120);
|
|
}
|
|
});
|
|
|
|
// #573 — Feature signals: ship/SKILL.md contains feature signal detection
|
|
test('ship/SKILL.md contains feature signal detection in Step 4', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
|
|
expect(content.toLowerCase()).toContain('feature signal');
|
|
});
|
|
|
|
// #510 — Context warnings: no SKILL.md contains "running low on context"
|
|
test('no generated SKILL.md contains "running low on context"', () => {
|
|
const skills = getAllSkillMds();
|
|
for (const { name, content } of skills) {
|
|
expect(content).not.toContain('running low on context');
|
|
}
|
|
});
|
|
|
|
// #510 — Context warnings: plan-eng-review has explicit anti-warning
|
|
test('plan-eng-review/SKILL.md contains "Do not preemptively warn"', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Do not preemptively warn');
|
|
});
|
|
|
|
// #474 — Safety Net: no SKILL.md uses find with -delete
|
|
test('no generated SKILL.md contains find with -delete flag', () => {
|
|
const skills = getAllSkillMds();
|
|
for (const { name, content } of skills) {
|
|
// Match find commands that use -delete (but not prose mentioning the word "delete")
|
|
const lines = content.split('\n');
|
|
for (const line of lines) {
|
|
if (line.includes('find ') && line.includes('-delete')) {
|
|
throw new Error(`${name}/SKILL.md contains find with -delete: ${line.trim()}`);
|
|
}
|
|
}
|
|
}
|
|
});
|
|
|
|
// #467 — Telemetry: preamble JSONL writes are gated by telemetry setting
|
|
test('preamble JSONL writes are inside telemetry conditional', () => {
|
|
const preamble = fs.readFileSync(path.join(ROOT, 'scripts/resolvers/preamble.ts'), 'utf-8');
|
|
// Find all skill-usage.jsonl write lines
|
|
const lines = preamble.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
if (lines[i].includes('skill-usage.jsonl') && lines[i].includes('>>')) {
|
|
// Look backwards for a telemetry conditional within 5 lines
|
|
let foundConditional = false;
|
|
for (let j = i - 1; j >= Math.max(0, i - 5); j--) {
|
|
if (lines[j].includes('_TEL') && lines[j].includes('off')) {
|
|
foundConditional = true;
|
|
break;
|
|
}
|
|
}
|
|
expect(foundConditional).toBe(true);
|
|
}
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('codex commands must not use inline $(git rev-parse --show-toplevel) for cwd', () => {
|
|
// Regression test: inline $(git rev-parse --show-toplevel) in codex exec -C
|
|
// or codex review without cd evaluates in whatever cwd the background shell
|
|
// inherits, which may be a different project in Conductor workspaces.
|
|
// The fix is to resolve _REPO_ROOT eagerly at the top of each bash block.
|
|
|
|
// Scan all source files that could contain codex commands
|
|
// Use Bun.Glob to avoid ELOOP from .claude/skills/gstack symlink back to ROOT
|
|
const tmplGlob = new Bun.Glob('**/*.tmpl');
|
|
const sourceFiles = [
|
|
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
|
|
.filter(f => f.endsWith('.ts'))
|
|
.map(f => `scripts/resolvers/${f}`),
|
|
'scripts/gen-skill-docs.ts',
|
|
];
|
|
|
|
test('no codex exec command uses inline $(git rev-parse --show-toplevel) in -C flag', () => {
|
|
const violations: string[] = [];
|
|
for (const rel of sourceFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1}`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('no generated SKILL.md has codex exec with inline $(git rev-parse --show-toplevel) in -C flag', () => {
|
|
const violations: string[] = [];
|
|
const skillMdGlob = new Bun.Glob('**/SKILL.md');
|
|
const skillMdFiles = Array.from(skillMdGlob.scanSync({ cwd: ROOT, followSymlinks: false }));
|
|
for (const rel of skillMdFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1}`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('codex review commands must be preceded by cd "$_REPO_ROOT" (no -C support)', () => {
|
|
// codex review does not support -C, so the pattern must be:
|
|
// _REPO_ROOT=$(git rev-parse --show-toplevel) || { ... }
|
|
// cd "$_REPO_ROOT"
|
|
// codex review ...
|
|
// NOT: codex review ... with inline $(git rev-parse --show-toplevel)
|
|
const allFiles = [
|
|
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...Array.from(new Bun.Glob('**/SKILL.md').scanSync({ cwd: ROOT, followSymlinks: false })),
|
|
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
|
|
.filter(f => f.endsWith('.ts'))
|
|
.map(f => `scripts/resolvers/${f}`),
|
|
'scripts/gen-skill-docs.ts',
|
|
];
|
|
const violations: string[] = [];
|
|
for (const rel of allFiles) {
|
|
const abs = path.join(ROOT, rel);
|
|
if (!fs.existsSync(abs)) continue;
|
|
const content = fs.readFileSync(abs, 'utf-8');
|
|
const lines = content.split('\n');
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
// Skip non-executable lines (markdown table cells, prose references)
|
|
if (line.includes('|') && line.includes('`/codex review`')) continue;
|
|
if (line.includes('`codex review`')) continue;
|
|
// Check for codex review with inline $(git rev-parse)
|
|
if (line.includes('codex review') && line.includes('$(git rev-parse --show-toplevel)')) {
|
|
violations.push(`${rel}:${i + 1} — inline git rev-parse in codex review`);
|
|
}
|
|
}
|
|
}
|
|
expect(violations).toEqual([]);
|
|
});
|
|
|
|
test('codex review commands pass diff scope through prompt, not --base', () => {
|
|
const checkedFiles = [
|
|
'codex/SKILL.md.tmpl',
|
|
'codex/SKILL.md',
|
|
'scripts/resolvers/review.ts',
|
|
'review/SKILL.md',
|
|
'ship/SKILL.md',
|
|
];
|
|
|
|
for (const rel of checkedFiles) {
|
|
const content = fs.readFileSync(path.join(ROOT, rel), 'utf-8');
|
|
expect(content).not.toContain('--base <base> -c \'model_reasoning_effort="high"\'');
|
|
expect(content).toContain('Run git diff origin/<base>...HEAD 2>/dev/null || git diff <base>...HEAD');
|
|
}
|
|
});
|
|
});
|
|
|
|
// ─── Learnings + Confidence Resolver Tests ─────────────────────
|
|
|
|
describe('LEARNINGS_SEARCH resolver', () => {
|
|
const SEARCH_SKILLS = ['review', 'ship', 'plan-eng-review', 'investigate', 'office-hours', 'plan-ceo-review'];
|
|
|
|
for (const skill of SEARCH_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains learnings search`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Prior Learnings');
|
|
expect(content).toContain('gstack-learnings-search');
|
|
});
|
|
}
|
|
|
|
test('learnings search includes cross-project config check', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('cross_project_learnings');
|
|
expect(content).toContain('--cross-project');
|
|
});
|
|
|
|
test('learnings search includes AskUserQuestion for first-time cross-project opt-in', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Enable cross-project learnings');
|
|
expect(content).toContain('project-scoped only');
|
|
});
|
|
|
|
test('learnings search mentions prior learning applied display format', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Prior learning applied');
|
|
});
|
|
});
|
|
|
|
describe('LEARNINGS_LOG resolver', () => {
|
|
const LOG_SKILLS = ['review', 'retro', 'investigate'];
|
|
|
|
for (const skill of LOG_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains learnings log`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Capture Learnings');
|
|
expect(content).toContain('gstack-learnings-log');
|
|
});
|
|
}
|
|
|
|
test('learnings log documents all type values', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
for (const type of ['pattern', 'pitfall', 'preference', 'architecture', 'tool']) {
|
|
expect(content).toContain(type);
|
|
}
|
|
});
|
|
|
|
test('learnings log documents all source values', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
for (const source of ['observed', 'user-stated', 'inferred', 'cross-model']) {
|
|
expect(content).toContain(source);
|
|
}
|
|
});
|
|
|
|
test('learnings log includes files field for staleness detection', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('"files"');
|
|
expect(content).toContain('staleness detection');
|
|
});
|
|
});
|
|
|
|
describe('CONFIDENCE_CALIBRATION resolver', () => {
|
|
const CONFIDENCE_SKILLS = ['review', 'ship', 'plan-eng-review', 'cso'];
|
|
|
|
for (const skill of CONFIDENCE_SKILLS) {
|
|
test(`${skill} generated SKILL.md contains confidence calibration`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Confidence Calibration');
|
|
expect(content).toContain('confidence score');
|
|
});
|
|
}
|
|
|
|
test('confidence calibration includes scoring rubric with all tiers', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('9-10');
|
|
expect(content).toContain('7-8');
|
|
expect(content).toContain('5-6');
|
|
expect(content).toContain('3-4');
|
|
expect(content).toContain('1-2');
|
|
});
|
|
|
|
test('confidence calibration includes display rules', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('Show normally');
|
|
expect(content).toContain('Suppress from main report');
|
|
});
|
|
|
|
test('confidence calibration includes finding format example', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('[P1] (confidence:');
|
|
expect(content).toContain('SQL injection');
|
|
});
|
|
|
|
test('confidence calibration includes calibration learning feedback loop', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('calibration event');
|
|
expect(content).toContain('Log the corrected pattern');
|
|
});
|
|
|
|
test('skills without confidence calibration do NOT contain it', () => {
|
|
// office-hours and retro do NOT use confidence calibration
|
|
for (const skill of ['office-hours', 'retro']) {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
expect(content).not.toContain('## Confidence Calibration');
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('gen-skill-docs prefix warning (#620/#578)', () => {
|
|
const { execSync } = require('child_process');
|
|
|
|
test('warns about skill_prefix when config has prefix=true', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
|
|
try {
|
|
// Create a fake ~/.gstack/config.yaml with skill_prefix: true
|
|
const fakeHome = tmpDir;
|
|
const fakeGstack = path.join(fakeHome, '.gstack');
|
|
fs.mkdirSync(fakeGstack, { recursive: true });
|
|
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: true\n');
|
|
|
|
const output = execSync('bun run scripts/gen-skill-docs.ts', {
|
|
cwd: ROOT,
|
|
env: { ...process.env, HOME: fakeHome },
|
|
encoding: 'utf-8',
|
|
timeout: 30000,
|
|
});
|
|
expect(output).toContain('skill_prefix is true');
|
|
expect(output).toContain('gstack-relink');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
|
|
test('no warning when skill_prefix is false or absent', () => {
|
|
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
|
|
try {
|
|
const fakeHome = tmpDir;
|
|
const fakeGstack = path.join(fakeHome, '.gstack');
|
|
fs.mkdirSync(fakeGstack, { recursive: true });
|
|
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: false\n');
|
|
|
|
const output = execSync('bun run scripts/gen-skill-docs.ts', {
|
|
cwd: ROOT,
|
|
env: { ...process.env, HOME: fakeHome },
|
|
encoding: 'utf-8',
|
|
timeout: 30000,
|
|
});
|
|
expect(output).not.toContain('skill_prefix is true');
|
|
} finally {
|
|
fs.rmSync(tmpDir, { recursive: true, force: true });
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('voice-triggers processing', () => {
|
|
const { extractVoiceTriggers, processVoiceTriggers } = require('../scripts/gen-skill-docs') as {
|
|
extractVoiceTriggers: (content: string) => string[];
|
|
processVoiceTriggers: (content: string) => string;
|
|
};
|
|
|
|
test('extractVoiceTriggers parses valid YAML list', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit.\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
|
|
const triggers = extractVoiceTriggers(content);
|
|
expect(triggers).toEqual(['see-so', 'security review']);
|
|
});
|
|
|
|
test('extractVoiceTriggers returns [] when no field present', () => {
|
|
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
|
|
expect(extractVoiceTriggers(content)).toEqual([]);
|
|
});
|
|
|
|
test('processVoiceTriggers appends voice triggers to description', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
|
|
const result = processVoiceTriggers(content);
|
|
expect(result).toContain('Voice triggers (speech-to-text aliases): "see-so", "security review".');
|
|
});
|
|
|
|
test('processVoiceTriggers strips voice-triggers field from output', () => {
|
|
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n---\nBody`;
|
|
const result = processVoiceTriggers(content);
|
|
expect(result).not.toContain('voice-triggers:');
|
|
});
|
|
|
|
test('processVoiceTriggers returns content unchanged when no voice-triggers', () => {
|
|
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
|
|
expect(processVoiceTriggers(content)).toBe(content);
|
|
});
|
|
|
|
test('generated CSO SKILL.md contains voice triggers in description', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
expect(content).toContain('"see-so"');
|
|
expect(content).toContain('Voice triggers (speech-to-text aliases):');
|
|
});
|
|
|
|
test('generated CSO SKILL.md does NOT contain raw voice-triggers field', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
|
|
const fmEnd = content.indexOf('\n---', 4);
|
|
const frontmatter = content.slice(0, fmEnd);
|
|
expect(frontmatter).not.toContain('voice-triggers:');
|
|
});
|
|
});
|
|
|
|
describe('plan-mode-info resolver (handshake-replacement)', () => {
|
|
const REVIEW_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-eng-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
];
|
|
|
|
// Header for the vestigial handshake that was removed. If it ever reappears,
|
|
// someone accidentally re-introduced the resolver.
|
|
const HANDSHAKE_MARKER = '## Plan Mode Handshake';
|
|
// Header for the new plan-mode-info section (previously lived at the tail
|
|
// of completion-status.ts; now hoisted to position 1 of the preamble).
|
|
const PLAN_MODE_INFO_MARKER = '## Skill Invocation During Plan Mode';
|
|
|
|
test('vestigial handshake is absent from all generated Claude SKILL.md files', () => {
|
|
// Scan every generated SKILL.md under ROOT (top-level directory per skill).
|
|
// Using fs.readdirSync + filter instead of a glob so we catch any skill
|
|
// that gets added later without updating this list.
|
|
const entries = fs.readdirSync(ROOT, { withFileTypes: true });
|
|
let checked = 0;
|
|
for (const entry of entries) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(ROOT, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content, `handshake marker in ${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
|
|
checked++;
|
|
}
|
|
expect(checked).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('vestigial handshake is absent from non-Claude host outputs when present on disk', () => {
|
|
// Non-Claude hosts render to hostSubdirs (.agents/, .openclaw/, etc). The
|
|
// plan-mode-info resolver has no host-scoping — all hosts get the new
|
|
// section, none get the old handshake. Scan all candidate host dirs.
|
|
const hostDirs = ['.agents', '.openclaw', '.opencode', '.factory', '.hermes', '.kiro', '.cursor', '.slate'];
|
|
let checked = 0;
|
|
for (const host of hostDirs) {
|
|
const skillsRoot = path.join(ROOT, host, 'skills');
|
|
if (!fs.existsSync(skillsRoot)) continue;
|
|
const entries = fs.readdirSync(skillsRoot, { withFileTypes: true });
|
|
for (const entry of entries) {
|
|
if (!entry.isDirectory()) continue;
|
|
const skillMd = path.join(skillsRoot, entry.name, 'SKILL.md');
|
|
if (!fs.existsSync(skillMd)) continue;
|
|
const content = fs.readFileSync(skillMd, 'utf-8');
|
|
expect(content, `handshake marker in ${host}/skills/${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
|
|
checked++;
|
|
}
|
|
}
|
|
if (checked === 0) {
|
|
// eslint-disable-next-line no-console
|
|
console.warn(
|
|
'plan-mode-info: no non-Claude host outputs found for cross-host absence check — ' +
|
|
'run `bun run gen:skill-docs --host all` to populate',
|
|
);
|
|
}
|
|
});
|
|
|
|
test.each(REVIEW_SKILLS)(
|
|
'%s/SKILL.md contains the new plan-mode-info section near the top',
|
|
(skill) => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const idx = content.indexOf(PLAN_MODE_INFO_MARKER);
|
|
expect(idx).toBeGreaterThan(0);
|
|
// Position 1 in preamble composition = within the first ~300 lines.
|
|
// Roughly translates to first ~15KB of text.
|
|
expect(idx).toBeLessThan(15_000);
|
|
},
|
|
);
|
|
|
|
test('plan-mode-info is wired BEFORE generateUpgradeCheck in preamble', () => {
|
|
const content = fs.readFileSync(
|
|
path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
|
|
'utf-8',
|
|
);
|
|
const planModeIdx = content.indexOf(PLAN_MODE_INFO_MARKER);
|
|
const upgradeIdx = content.indexOf('UPGRADE_AVAILABLE');
|
|
expect(planModeIdx).toBeGreaterThan(0);
|
|
expect(upgradeIdx).toBeGreaterThan(0);
|
|
expect(planModeIdx).toBeLessThan(upgradeIdx);
|
|
});
|
|
|
|
test('0C-bis STOP block present in plan-ceo-review/SKILL.md', () => {
|
|
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
|
|
const presentIdx = content.indexOf('Present these approach options via AskUserQuestion');
|
|
const preludeIdx = content.indexOf('### 0D-prelude');
|
|
expect(presentIdx).toBeGreaterThan(0);
|
|
expect(preludeIdx).toBeGreaterThan(presentIdx);
|
|
const between = content.slice(presentIdx, preludeIdx);
|
|
expect(between).toContain('**STOP.**');
|
|
expect(between).toContain('Do NOT proceed to Step 0D or 0F until the user responds to 0C-bis');
|
|
});
|
|
});
|
|
|
|
// GSTACK REVIEW REPORT report-at-bottom contract — verifies the prompt-text
|
|
// fix in scripts/resolvers/review.ts (the load-bearing change for the
|
|
// "report not at bottom of plan in plan mode" bug). The bug is in the
|
|
// prompt's contradictory write-flow instructions, not in observable
|
|
// runtime behavior we can cheaply gate in CI. Verifying the prompt text
|
|
// directly is the deterministic equivalent of the regression test the
|
|
// PTY harness can't reliably drive (autoplan needs auto-progression of
|
|
// AskUserQuestions to reach the report-write step, which the harness
|
|
// doesn't support today).
|
|
describe('GSTACK REVIEW REPORT delete-then-append flow', () => {
|
|
const PLAN_REVIEW_SKILLS = [
|
|
'plan-ceo-review',
|
|
'plan-design-review',
|
|
'plan-devex-review',
|
|
'plan-eng-review',
|
|
];
|
|
|
|
for (const skill of PLAN_REVIEW_SKILLS) {
|
|
test(`${skill}/SKILL.md prescribes delete-then-append, not in-place replace`, () => {
|
|
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
|
|
// The new (correct) instruction must be present.
|
|
expect(content).toContain('delete-then-append flow');
|
|
expect(content).toContain('never mid-file');
|
|
expect(content).toContain('Do NOT replace the section in place');
|
|
|
|
// The old contradictory bullets must be gone. The signature phrase
|
|
// from the buggy prompt was 'replace it entirely using the Edit tool'
|
|
// which is what allowed mid-file reports to stay mid-file.
|
|
expect(content).not.toContain('replace it** entirely using the Edit tool');
|
|
expect(content).not.toContain('If it was found mid-file, move it');
|
|
});
|
|
}
|
|
|
|
test('scripts/resolvers/review.ts source has the rewritten flow', () => {
|
|
const src = fs.readFileSync(path.join(ROOT, 'scripts', 'resolvers', 'review.ts'), 'utf-8');
|
|
expect(src).toContain('delete-then-append flow');
|
|
expect(src).toContain('never mid-file');
|
|
expect(src).toContain('Do NOT replace the section in place');
|
|
// Old contradictory bullets are gone from the source resolver.
|
|
expect(src).not.toContain('replace it** entirely using the Edit tool');
|
|
expect(src).not.toContain('If it was found mid-file, move it');
|
|
});
|
|
});
|
|
|
|
describe('LEARNINGS_SEARCH resolver: query parameter', () => {
|
|
// Lazy-load resolver and types after describe block to keep test file self-contained.
|
|
const { generateLearningsSearch } = require('../scripts/resolvers/learnings');
|
|
const { HOST_PATHS } = require('../scripts/resolvers/types');
|
|
|
|
const claudeCtx = {
|
|
skillName: 'test',
|
|
tmplPath: 'test/SKILL.md.tmpl',
|
|
host: 'claude',
|
|
paths: HOST_PATHS.claude,
|
|
};
|
|
const codexCtx = { ...claudeCtx, host: 'codex', paths: HOST_PATHS.codex };
|
|
|
|
test('no args → bash does not contain --query (backwards-compat)', () => {
|
|
const out = generateLearningsSearch(claudeCtx);
|
|
expect(out).not.toContain('--query');
|
|
});
|
|
|
|
test('claude host + query=foo bar → both cross-project and project-scoped branches contain --query', () => {
|
|
const out = generateLearningsSearch(claudeCtx, ['query=foo bar']);
|
|
// Both branches of the if/else must carry the flag.
|
|
const lines = out.split('\n').filter(l => l.includes('gstack-learnings-search'));
|
|
expect(lines.length).toBeGreaterThanOrEqual(2);
|
|
for (const line of lines) {
|
|
expect(line).toContain('--query "foo bar"');
|
|
}
|
|
});
|
|
|
|
test('codex host + query=foo bar → codex bash variant contains --query', () => {
|
|
const out = generateLearningsSearch(codexCtx, ['query=foo bar']);
|
|
expect(out).toContain('--query "foo bar"');
|
|
expect(out).toContain('$GSTACK_BIN/gstack-learnings-search');
|
|
});
|
|
|
|
test('empty value query= → bash does not contain --query (locked semantics: falls through)', () => {
|
|
const claudeOut = generateLearningsSearch(claudeCtx, ['query=']);
|
|
expect(claudeOut).not.toContain('--query');
|
|
const codexOut = generateLearningsSearch(codexCtx, ['query=']);
|
|
expect(codexOut).not.toContain('--query');
|
|
});
|
|
|
|
test('shell-injection chars in query= → throws at gen-time (defense in depth)', () => {
|
|
for (const bad of ['$(whoami)', '`cmd`', 'a;b', 'a&b', 'a"b', 'a\\b', 'foo$x']) {
|
|
expect(() => generateLearningsSearch(claudeCtx, [`query=${bad}`])).toThrow(/alphanumeric/);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('EXIT PLAN MODE GATE placement', () => {
|
|
// Fresh skill list — do NOT reuse REVIEW_SKILLS upstream (3 entries, missing plan-devex).
|
|
const planSkills = ['plan-eng-review', 'plan-ceo-review', 'plan-design-review', 'plan-devex-review'];
|
|
|
|
// Strip fenced code blocks before matching headings — PLAN_FILE_REVIEW_REPORT
|
|
// already contains `## GSTACK REVIEW REPORT` inside a markdown example fence,
|
|
// and the gate text itself shows `## GSTACK REVIEW REPORT` inside a fence too.
|
|
const stripFences = (md: string) => md.replace(/```[\s\S]*?```/g, '');
|
|
|
|
test('gate is the terminal ## heading in every plan-* review SKILL.md', () => {
|
|
for (const skill of planSkills) {
|
|
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
|
|
const stripped = stripFences(md);
|
|
const headings = [...stripped.matchAll(/^## .+$/gm)].map(m => m[0]);
|
|
const lastH2 = headings.at(-1);
|
|
expect(lastH2, `${skill}/SKILL.md last ## heading (fences stripped)`).toBe('## EXIT PLAN MODE GATE (BLOCKING)');
|
|
expect(md, `${skill}/SKILL.md gate body`).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation');
|
|
}
|
|
});
|
|
|
|
test('codex/SKILL.md contains gate (mid-file per D5; Step 2B/2C follow)', () => {
|
|
const codex = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
|
|
expect(codex).toContain('## EXIT PLAN MODE GATE (BLOCKING)');
|
|
expect(codex).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation');
|
|
});
|
|
});
|