Files
gstack/test/gen-skill-docs.test.ts
Garry Tan 7ca04d8ef0 v1.42.0.0 Daegu wave: 23 community-filed bugs + PTY classifier enforcement (24 bisect commits) (#1594)
* fix(gstack-paths): guard CLAUDE_PLUGIN_DATA against cross-plugin contamination (#1569)

gstack-paths previously trusted CLAUDE_PLUGIN_DATA as a fallback for
GSTACK_STATE_ROOT whenever GSTACK_HOME was unset. When another plugin
(e.g. Codex) persists its own CLAUDE_PLUGIN_DATA into the session env
via CLAUDE_ENV_FILE, gstack picked it up and wrote checkpoints,
analytics, and learnings into that plugin's directory. Anyone with the
Codex plugin installed alongside gstack hit this silently.

Fix: guard the CLAUDE_PLUGIN_DATA branch so it only fires when
CLAUDE_PLUGIN_ROOT confirms we're running as the gstack plugin (path
contains "gstack"). Skill installs fall through to \$HOME/.gstack.

Contributed by @ElliotDrel via #1570. Closes #1569.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(gbrain-sync): sourceLocalPath handles wrapped {sources:[...]} shape from gbrain v0.20+

gbrain v0.20+ changed `gbrain sources list --json` to return
{sources: [...]} instead of a flat array. sourceLocalPath crashed
upstream with `list.find is not a function` on every /sync-gbrain
invocation against modern gbrain. Accept both shapes for
forward/backward compat, matching probeSource/sourcePageCount in
lib/gbrain-sources.ts.

Contributed by @jakehann11 via #1571. Closes #1567. Supersedes #1564
(@tonyjzhou, same fix, different shape — credit retained).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(brain-context-load): probe gbrain via execFile, not shell builtin (#1559)

gbrainAvailable() used `execFileSync("command", ["-v", "gbrain"])`,
which fails in any environment where the `command` builtin isn't on
the spawned process's PATH (most non-interactive shells). The probe
then reported gbrain as missing even when it was installed, and
context-load silently skipped vector/list queries.

Fix: probe `gbrain --version` directly with a 500ms timeout (matching
the rest of the file's MCP_TIMEOUT_MS). Same semantics, works
everywhere execFile works.

Contributed by @jbetala7 via #1560. Closes #1559.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(gbrain-doctor): pin schema_version:2 doctor parse path (#1418)

Adds an exec-path regression test that runs a fake gbrain shim emitting
the v0.25+ doctor JSON shape (schema_version: 2, status: "warnings",
exit 1 for health_score < 100, no top-level `engine` field). Confirms
freshDetectEngineTier recovers stdout from the non-zero exit and falls
back to GBRAIN_HOME/config.json for the engine label.

The pre-existing test for #1415 only stripped gbrain from PATH; this
test exercises the actual doctor parse path, closing the gap that
codex's plan review flagged.

Also documents the schema_version separation in
lib/gbrain-local-status.ts: the local CacheEntry stays at version 1,
distinct from the doctor-output schema_version which we accept across
versions in gstack-memory-helpers.

Closes #1418 (credit @mvanhorn for surfacing the doctor + schema_v2
collapse). The fix landed pre-emptively in v1.29.x; this commit pins
it with a stronger test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(memory-ingest): pin put_page regression + scrub stale name from --help and comments (#1346)

#1346 reported that gstack-memory-ingest still called the renamed
gbrain put_page subcommand on gbrain v0.18+. The actual code migrated
to `gbrain put` and later to batch `gbrain import <dir>` before this
report landed — only documentation lag remained.

This commit:
- Updates the --help string ("Skip gbrain put calls (still updates
  state file)") so user-facing docs match the shipped subcommand
- Updates two inline comments that still referenced the old name
- Adds test/memory-ingest-no-put_page.test.ts: a regression pin that
  strips comments from bin/gstack-memory-ingest.ts and fails the build
  if "put_page" appears in any active code or string literal, plus a
  sanity check that the file still calls a supported gbrain page-write
  verb (put or import)

Closes #1346. Reporter @kylma-code surfaced the doc lag; the original
code migration credit is on the v1.27.x wave.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(resolvers): rewrite all gbrain put_page instructions to canonical put <slug>

scripts/resolvers/gbrain.ts emitted user-facing copy-paste instructions
using the renamed `gbrain put_page` subcommand across 10 skills
(office-hours, investigate, plan-ceo-review, retro, plan-eng-review,
ship, cso, design-consultation, fallback, entity-stub). Every gstack
user copying those snippets hit "unknown command: put_page" on gbrain
v0.18+.

This commit:
- Rewrites all 10 instruction templates to use `gbrain put <slug>
  --content "$(cat <<EOF...EOF)"` with title/tags moved into YAML
  frontmatter inside --content, matching the v0.18+ subcommand shape
- Updates README.md and USING_GBRAIN_WITH_GSTACK.md "common commands"
  table to reference `gbrain put` and `gbrain get`
- Adds test/resolvers-gbrain-put-rewrite.test.ts pinning two
  invariants: (a) resolver source ships only canonical instructions,
  (b) every tracked SKILL.md file is free of `gbrain put_page`

CHANGELOG entries are deliberately left untouched (historical record).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(build): extract package.json build to scripts/build.sh for Windows Bun compat (#1538, #1537, #1530, #1457, #1561)

Bun's Windows shell parser rejects multiple constructs the inline
package.json build chain used: brace groups `{ cmd; }`, subshells with
redirection `( git ... ) > path/.version`, and (in Bun 1.3.x) subshells
near redirections in general. Every Windows install + every
auto-upgrade since v1.34.2.0 has failed on `bun run build`.

Extracts the build chain to scripts/build.sh and the .version writes to
scripts/write-version-files.sh. POSIX-portable, no Bun shell parsing
involved. Also adds Windows-specific bun.exe handling for non-ASCII
PATHs (a separate Windows footgun where Bun's --compile fails when the
binary lives under a path with non-ASCII chars).

Updates test/build-script-shell-compat.test.ts to assert the new shape:
no subshells with redirections anywhere in the build chain, and build
delegates to scripts/build.sh which delegates .version writes.

Contributed by @Charlie-El via #1544. Supersedes #1531 (@scarson, fixed
in build helper), #1480 (@mikepsinn, partial overlap), #1460
(@realcarsonterry, brace-group fix subsumed) — credit retained.
Closes #1538, #1537, #1530, #1457, #1561.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(windows): .exe glob in .gitignore + .exe extension resolution in find-browse (#1554)

bun build --compile on Windows appends .exe to the output filename,
producing browse.exe instead of browse. find-browse's existsSync probe
only checked the bare path and returned null on Windows even when the
binary was correctly built. .gitignore similarly only excluded the
bare bin/gstack-global-discover path, leaving the .exe variant
tracked.

This commit:
- .gitignore: changes `bin/gstack-global-discover` →
  `bin/gstack-global-discover*` so the Windows .exe variant is ignored
- browse/src/find-browse.ts: adds isExecutable + findExecutable helpers
  that fall back to .exe/.cmd/.bat probing on Windows, mirroring the
  same helper already in make-pdf/src/browseClient.ts and pdftotext.ts

Contributed by @Mike-E-Log via #1554.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(windows): add fresh-install E2E gate that runs bun run build on windows-latest

Adds .github/workflows/windows-setup-e2e.yml as the gate that catches
Bun shell-parser regressions in the build chain before they reach
users. Triggers on PRs touching package.json, scripts/build.sh,
scripts/write-version-files.sh, setup, browse cli/find-browse, or
gstack-paths.

What it verifies:
1. bun run build completes on Windows (the previously-broken path that
   #1538/#1537/#1530/#1457/#1561 reported)
2. All compiled binaries land on disk (browse.exe, find-browse.exe,
   design.exe, gstack-global-discover.exe)
3. find-browse resolves to the .exe variant on Windows (regression
   gate for #1554)
4. gstack-paths returns non-empty GSTACK_STATE_ROOT/PLAN_ROOT/TMP_ROOT
   on Windows (regression gate for #1570)

Complements the existing windows-free-tests.yml (curated unit subset);
this new workflow exercises the install path itself.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(codex): move diff scope into prompt instead of --base (Codex CLI 0.130+ argv conflict) (#1209)

Codex CLI ≥ 0.130.0 rejects passing a custom prompt and --base together
(mutually exclusive at argv level). Every /codex review, /review, and
/ship structured Codex review call ended with an argv error before the
model ran.

Fix: scope the diff in prompt text using
"Run git diff origin/<base>...HEAD 2>/dev/null || git diff <base>...HEAD"
instead of `--base <base>`. Preserves the filesystem boundary
instruction across all invocations and keeps Codex's review prompt
tuning.

Touches:
- codex/SKILL.md.tmpl + regenerated codex/SKILL.md
- scripts/resolvers/review.ts + regenerated review/SKILL.md, ship/SKILL.md
- test/gen-skill-docs.test.ts: new regression that fails if any of the
  five known files still contain the prompt+--base shape
- test/skill-validation.test.ts: corresponding negative + positive pin
  on the rendered SKILL.md files

Contributed by @jbetala7 via #1209. Closes #1479. Supersedes #1527
(@mvanhorn — same intent, different patch shape, CONFLICTING) and
#1449 (@Gujiassh — broader refactor, CONFLICTING). Credit retained
in CHANGELOG.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): diff from git merge-base, not git diff origin/<base> (#1492)

git diff origin/<base> shows everything since the common ancestor in
both directions — it includes commits that landed on origin/<base>
after this branch was created as deletions. That made /review and
/ship's pre-landing structured review report inflated diff totals and
flagged "removed" code that was actually still present in the working
tree.

Fix: compute DIFF_BASE via git merge-base origin/<base> HEAD and diff
the working tree against that point. Same coverage of uncommitted
edits, no phantom deletions from out-of-order base advancement.

Applies to /review's Step 1 (diff existence check), Step 3 (get the
diff), the build-on-intent scope-creep check, the structured review
DIFF_INS/DIFF_DEL stats, and the Claude adversarial subagent prompt.
Same change flows into ship/SKILL.md via the shared resolver.

Touches:
- review/SKILL.md.tmpl + regenerated review/SKILL.md, ship/SKILL.md
- scripts/resolvers/review.ts
- scripts/resolvers/review-army.ts

Contributed by @mvanhorn via #1492.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(codex): pin filesystem-boundary preservation across all codex review surfaces (#1503, #1522)

#1503 reported that the bare codex review --base path stripped the
filesystem boundary instruction, letting Codex spend tokens reading
.claude/skills/ and agents/. #1522 proposed adding a skill-path
detector that switched to the custom-instructions route when the diff
touched skill files.

After C10 (#1209) restructured codex review to always carry the
boundary in the prompt (the prompt+--base argv conflict forced the
restructure), the skill-path detector becomes redundant — every
default call already preserves the boundary.

This commit pins the post-#1209 invariant with a test that fails the
build if any future refactor strips the boundary from codex/SKILL.md,
review/SKILL.md, or ship/SKILL.md. Closes #1503 by regression test.

#1522 (@genisis0x) is superseded by #1209 (the prompt rewrite covers
its safety concern); credit retained in CHANGELOG.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(skills): use command -v instead of which for codex detection (#1197)

`which` is not on PATH in every shell — some Windows shells, BusyBox-
only containers, and minimal CI images all fail when skills probe
codex availability via `which codex`. `command -v` is a POSIX builtin
and always available where the skill is running.

Touched:
- codex/SKILL.md.tmpl: CODEX_BIN=$(command -v codex || echo "")
- scripts/resolvers/review.ts and scripts/resolvers/design.ts:
  3 + 3 sites each rewritten to `command -v codex >/dev/null 2>&1`
- Regenerated all 10 affected SKILL.md files (codex, review, ship,
  design-consultation, design-review, office-hours, plan-ceo-review,
  plan-design-review, plan-devex-review, plan-eng-review)
- test/skill-validation.test.ts: updated pin + defensive regression
  test that fails if `which codex` returns to codex/SKILL.md
- test/skill-e2e-plan.test.ts: updated summary regex

Contributed by @mvanhorn via #1197.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(codex): surface non-zero exits so wrappers stop reading as silent stalls (#1467, #1327)

When codex exits non-zero (parse errors, arg-shape breaks, model API
errors that propagate as non-zero status), the calling agent
previously saw an empty output and burned 30-60 minutes misdiagnosing
as a silent model/API stall. The hang-detection block only caught
exit 124 (the timeout-wrapper signal).

Adds elif blocks in all four codex invocation sites (Review default,
Challenge, Consult new-session, Consult resume) that:
- Echo "[codex exit N] <stderr first line>" to stdout
- Indent the first 20 stderr lines for inline context
- Log codex_nonzero_exit telemetry tagged with the call site

Contributed by @genisis0x via #1467. Closes #1327.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(design): disclose OpenAI key source + warn on cwd .env match (#1278, closes #1248)

The design binary previously called process.env.OPENAI_API_KEY without
checking where the key came from. If a user ran $D inside someone
else's project that had OPENAI_API_KEY in its .env, the resulting
generation billed that project's account. Silent and irreversible.

Fix: resolveApiKeyInfo() returns both the key and its source. When the
env-var path matches an OPENAI_API_KEY entry in the current
directory's .env, .env.<NODE_ENV>, or .env.local file, we set a
warning. requireApiKey() prints "Using OpenAI key from <source>" plus
the warning before the run — never the key itself.

Adds 6 unit tests covering: config-vs-env precedence, env-only (no
match), env+cwd .env match, quoted/exported values, value-mismatch
(no false positive), and the no-leak invariant for requireApiKey
stderr output.

Contributed by @jbetala7 via #1278. Closes #1248.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(browse): guard full-page screenshots against Anthropic vision API >2000px brick (#1214)

Full-page screenshots of tall pages routinely exceeded 2000px on the
longest dimension, silently bricking the agent's session: the
resulting base64 reached the Anthropic vision API which rejected the
oversized image, leaving the agent burning turns on a useless blob
with no stderr trace from the browse side.

Adds browse/src/screenshot-size-guard.ts as a shared helper:
- guardScreenshotBuffer(buf) → downscales in-memory if max(w,h) > 2000
- guardScreenshotPath(path) → file-mode variant that rewrites in place
- Aspect ratio preserved via sharp's resize fit:inside
- Stderr diagnostic on any downscale so callers can see when it fired
- Lazy sharp import so non-screenshot paths pay no startup cost

Wires the guard into all three full-page callsites codex review
flagged:
- browse/src/snapshot.ts: annotated + heatmap fullPage captures
- browse/src/meta-commands.ts: screenshot command (path + base64
  fullPage modes) plus the responsive 3-viewport sweep
- browse/src/write-commands.ts: prettyscreenshot fullPage path

Covers seven unit cases (pass-through, downscale, aspect ratio,
exactly-2000px edge, file-mode rewrite) plus a static invariant test
that fails the build if any of the three callsites stops importing the
guard.

Closes #1214.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(security): add Node sidecar entry for L4 prompt-injection classifier (#1370)

The L4 TestSavant classifier in browse/src/security-classifier.ts
can't be imported into the compiled browse server (onnxruntime-node
dlopen fails from Bun's compile extract dir per CLAUDE.md). The agent
that used to host it (sidebar-agent.ts) was removed when the PTY
proved out — leaving the classifier file shipped but with zero
callers. Exactly the gap codex flagged in #1370.

Adds browse/src/security-sidecar-entry.ts: a Node script that runs the
classifier as a subprocess of the browse server. It reads NDJSON
requests from stdin and writes id-correlated NDJSON responses to
stdout, supporting:
  - op: "scan-page-content" — full L4 classifier scan
  - op: "ping" — liveness probe for the client's health check
  - op: "status" — classifier readiness (used by /pty-inject-scan to
    surface l4 { available: bool } in its response)

Plus browse/src/find-security-sidecar.ts: a resolver that locates
node + the bundled JS entry (browse/dist/security-sidecar.js, built in
a follow-up package.json change) or falls back to the dev TS entry.
Returns null cleanly when node isn't on PATH so the calling endpoint
can degrade per D7 (extension WARN + user confirm).

C17 of the security-stack wave. C18 adds the IPC client + lifecycle
management; C19 wires the endpoint; C20 routes the extension through it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(security): sidecar IPC client with lifecycle + circuit breaker (#1370)

Adds browse/src/security-sidecar-client.ts to manage the Node L4
classifier subprocess from the compiled browse server:

- Lazy spawn on first scan; reuses the same process across requests
- Id-correlated request/response via NDJSON over stdio
- 5s default per-scan timeout; 64KB payload cap (short-circuits before
  spawn so oversized requests don't waste a process)
- 3-in-10-minutes respawn cap → trips circuit breaker; subsequent
  scans throw immediately so the /pty-inject-scan endpoint can surface
  l4 { available: false } to the extension and degrade to WARN+confirm
- process.on('exit') sends SIGTERM to the child for clean teardown
- isSidecarAvailable() lets the endpoint probe before scan calls so
  the response shape reflects degraded mode honestly

Unit tests cover the payload cap, the availability probe, and the
breaker-doesn't-crash invariant under repeated rejected calls.

C18 of the security-stack wave. C19 adds POST /pty-inject-scan; C20
routes the extension through it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(security): add POST /pty-inject-scan endpoint for pre-PTY-inject scans (#1370)

The sidebar's gstackInjectToTerminal callers (toolbar Cleanup,
Inspector "Send to Code") were piping page-derived text directly into
the live claude PTY with ZERO classifier processing — the gap codex
flagged in #1370. The documented sidebar security stack had a hole
the size of every Cleanup-button click.

Adds POST /pty-inject-scan to browse/src/server.ts:
- Local-only binding (NOT in TUNNEL_PATHS — tunnel attempts get the
  general 404 path; never reaches the scan logic)
- Root-token auth via existing validateAuth() — 401 on unauth
- 64KB request cap → 413 + payload-too-large body
- 5s scan timeout via sidecar client
- URL-blocklist forced to BLOCK in PTY context (page-derived REPL
  input is higher-risk than ordinary tool output)
- L4 ML classifier via the sidecar when available; degrades to WARN
  per D7 when sidecar is unavailable
- Response goes through JSON.stringify(..., sanitizeReplacer) per
  v1.38.0.0 Unicode-egress hardening
- Imports only from security-sidecar-client.ts, never directly from
  security-classifier.ts (which would brick the compiled Bun binary)

Seven static-invariant tests pin the POST verb, auth gate, 64KB cap,
tunnel-listener exclusion, sanitizeReplacer wrapping, l4 availability
shape, and the no-direct-classifier-import rule.

C19 of the security-stack wave. C20 routes the extension through it;
C21 adds the invariant AST check.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(extension): route gstackInjectToTerminal through /pty-inject-scan (#1370)

Closes the documented-vs-shipped gap codex flagged in #1370. The
sidebar's two PTY-injection call sites (Inspector "Send to Code" and
toolbar Cleanup) now pre-scan via the new /pty-inject-scan endpoint
before writing to the live claude REPL.

Adds window.gstackScanForPTYInject(text, origin) to
extension/sidepanel-terminal.js:
- Async, returns { allow, verdict, reasons, l4 }
- POST to /pty-inject-scan with the existing root-token auth
- WARN+confirm on scan failure (network down, sidecar absent, etc.)
  rather than silent PASS — D7 honest-degradation

gstackInjectToTerminal stays synchronous, returns boolean. Per D6:
keeping the inject sync means existing `const ok = ...?.()` callers
don't break, and the invariant test in
test/extension-pty-inject-invariant.test.ts can statically pin that
every call goes through the scan first.

extension/sidepanel.js call sites updated:
- inspectorSendBtn click → await scan, BLOCK drops + WARN prompts via
  window.confirm, PASS injects silently
- runCleanup() → same flow. Static cleanup prompt always PASSes but
  still routes through scan to honor the invariant.

C20 of the security-stack wave. C21 adds the static invariant test.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(security): invariant — extension PTY inject must be scan-gated (#1370)

Static-analysis invariant test that fails the build if any
extension/*.js path calls window.gstackInjectToTerminal without a
preceding window.gstackScanForPTYInject in the same enclosing
function. Closes the documented-vs-shipped gap codex demanded a
machine check on.

Rules:
- Rule 1: any file that calls inject must also reference scan
- Rule 2: in the enclosing function (function declaration, arrow,
  async (), event handler), a scan call must appear before the inject
  call by source position
- Exemption: sidepanel-terminal.js (the file that DEFINES the inject
  function) is exempt from Rule 2 since the definition is not a call

Plus two structural checks:
- sidepanel-terminal.js defines both the inject and scan functions
- inject stays SYNCHRONOUS (no `async` modifier) per D6 — async would
  silently break the `const ok = ...?.()` pattern at every caller

C21 of the security-stack wave. The sidecar architecture (#1370) is
complete: server-side L1-L3 + L4-via-sidecar (C17+C18+C19), extension
pre-scan wiring (C20), and now the regression gate (C21).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(browse): opt-in extended stealth mode with 6 detection-vector patches (#1112)

Rebases @garrytan's PR #1112 (Apr 2026, abandoned) onto the current
browse/src/stealth.ts contract. The existing minimal "codex narrowed"
stealth (webdriver-mask + AutomationControlled launch arg) stays the
default. PR #1112's six additional patches are added behind an opt-in
GSTACK_STEALTH=extended env flag.

Extended-mode patches (applied AFTER the default mask, in order):
  1. delete navigator.webdriver from prototype (not just the getter —
     detectors check `"webdriver" in navigator`)
  2. WebGL renderer spoof to Apple M1 Pro (SwiftShader was the #1
     software-GPU tell in containers)
  3. navigator.plugins returns a PluginArray-prototype-passing array
     with MimeType objects and namedItem()
  4. window.chrome populated with chrome.app, chrome.runtime,
     chrome.loadTimes(), chrome.csi() with realistic shapes
  5. navigator.mediaDevices backfilled when headless drops it
  6. CDP cdc_*-prefixed window globals cleared

Why opt-in: the default mode's contract is fingerprint CONSISTENCY,
which protects against detectors that flag spoofing mismatch. Extended
mode actively lies about the environment; sites that reflect on these
properties can break. Users who hit detection in default mode can flip
GSTACK_STEALTH=extended for SannySoft 100% pass-rate.

Twenty unit tests pin the env-flag semantics, all six patches' code
presence, and the applyStealth wiring order. Live SannySoft pass-rate
verification stays in the periodic-tier E2E suite.

Contributed by @garrytan via #1112 (rebased — original PR opened
before the codex-narrowed minimum landed; rebase preserves the
narrowed default while adding the SannySoft-passing path as opt-in).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(fixtures): regenerate ship-SKILL.md golden baselines after C10-C13 + C16 templates

Updates the three ship-SKILL.md golden baselines (claude, codex,
factory hosts) to match the new shape produced by:
- C10 #1209 codex argv (prompt + diff scope, no --base)
- C11 #1492 merge-base diff (DIFF_BASE= preamble)
- C13 #1197 command -v for codex detection
- C12 + boundary preservation per regen-enforcing test

Per CLAUDE.md SKILL.md workflow: edit the .tmpl, run gen:skill-docs,
commit the regenerated outputs together. Goldens are part of the
regen contract — without this commit, test/host-config.test.ts'
golden-baseline checks fail with the diff codex review surfaced.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(release): v1.41.0.0 — Daegu wave (24 bisect commits, 14 user-facing fixes)

Bumps VERSION 1.40.0.0 → 1.41.0.0. CHANGELOG entry follows the
release-summary format in CLAUDE.md: two-line headline, lead
paragraph, "The numbers that matter" table, "What this means for
builders" closer, then itemized Added/Changed/Fixed/For contributors
with inline credit to every PR author and original issue reporter.

Scale-aware bump per CLAUDE.md: 24 commits, ~6000 LOC net,
substantial new capability across security (PTY sidecar wiring),
install (Windows build chain), compat (gbrain 0.18-0.35, Codex CLI
0.130+), and quality (screenshot guard, design key disclosure,
extended stealth opt-in). MINOR is the right call.

Closes for users: #1567, #1559, #1569, #1346, #1418, #1538, #1537,
#1530, #1457, #1561, #1554, #1479, #1503, #1248, #1214, #1370, #1327,
#1193 pattern, #1152 pattern. Credit retained inline.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(find-browse): resolve source-checkout layout <git-root>/browse/dist/browse[.exe]

windows-setup-e2e.yml runs `bun browse/src/find-browse.ts` against a
freshly-built repo where binaries land at browse/dist/browse.exe (no
.claude/skills/gstack/ install layout). The previous markers chain
only matched .codex/.agents/.claude prefixed paths, so find-browse
exited "not found" even when the binary was present.

Adds a source-checkout fallback after the marker scan: if no
installed layout resolves but <git-root>/browse/dist/browse[.exe]
exists, return that. Three real callers hit this path:
- gstack repo dev workflow before `./setup` runs
- windows-setup-e2e.yml CI (the breakage that surfaced this)
- make-pdf consumers running from a sibling source checkout

Smoke-verified: a fresh git repo with browse/dist/browse on disk now
resolves through the source-checkout branch (was returning null
before this commit).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(release): bump v1.41.0.0 → v1.42.0.0 to clear queue collision with #1574

The version-gate workflow flagged a collision: PR #1574
(garrytan/colombo-v3) already claims v1.41.0.0, and #1592
(fix/audit-critical-high-bugs) claims v1.41.1.0. Per CLAUDE.md's
workspace-aware ship rule, queue-advancing past a claimed version
within the same bump level is permitted — MINOR work landing on top
of a queued MINOR still reads as MINOR relative to main.

Util's suggested next slot is v1.42.0.0; taking it. CHANGELOG entry
header bumped + dated 2026-05-19; entry body unchanged (same wave
content, same credit list).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 07:35:01 -07:00

3144 lines
134 KiB
TypeScript

import { describe, test, expect } from 'bun:test';
import { COMMAND_DESCRIPTIONS } from '../browse/src/commands';
import { SNAPSHOT_FLAGS } from '../browse/src/snapshot';
import * as fs from 'fs';
import * as path from 'path';
import * as os from 'os';
const ROOT = path.resolve(import.meta.dir, '..');
const MAX_SKILL_DESCRIPTION_LENGTH = 1024;
function extractDescription(content: string): string {
const fmEnd = content.indexOf('\n---', 4);
expect(fmEnd).toBeGreaterThan(0);
const frontmatter = content.slice(4, fmEnd);
const lines = frontmatter.split('\n');
let description = '';
let inDescription = false;
const descLines: string[] = [];
for (const line of lines) {
if (line.match(/^description:\s*\|?\s*$/)) {
inDescription = true;
continue;
}
if (line.match(/^description:\s*\S/)) {
return line.replace(/^description:\s*/, '').trim();
}
if (inDescription) {
if (line === '' || line.match(/^\s/)) {
descLines.push(line.replace(/^ /, ''));
} else {
break;
}
}
}
if (descLines.length > 0) {
description = descLines.join('\n').trim();
}
return description;
}
function extractMarkdownSection(content: string, heading: string): string {
const escaped = heading.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const startMatch = content.match(new RegExp(`^${escaped}.*$`, 'm'));
expect(startMatch?.index).toBeDefined();
const start = startMatch!.index!;
const afterHeading = start + startMatch![0].length;
const nextSection = content.slice(afterHeading).match(/\n## /);
const end = nextSection?.index === undefined
? content.length
: afterHeading + nextSection.index;
return content.slice(start, end).trim();
}
function extractPreambleBeforeWorkflow(content: string, workflowMarkers: string[]): string {
const markerIndexes = workflowMarkers
.map(marker => content.indexOf(marker))
.filter(index => index >= 0);
expect(markerIndexes.length).toBeGreaterThan(0);
return content.slice(0, Math.min(...markerIndexes));
}
function isRepoRootSymlink(candidateDir: string): boolean {
try {
return fs.realpathSync(candidateDir) === fs.realpathSync(ROOT);
} catch {
return false;
}
}
// Dynamic template discovery — matches the generator's findTemplates() behavior.
// New skills automatically get test coverage without updating a static list.
const ALL_SKILLS = (() => {
const skills: Array<{ dir: string; name: string }> = [];
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
skills.push({ dir: '.', name: 'root gstack' });
}
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
if (fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) {
skills.push({ dir: entry.name, name: entry.name });
}
}
return skills;
})();
const CLAUDE_SKIPPED_SKILL_DIRS = new Set(['claude']);
const CLAUDE_GENERATED_SKILLS = ALL_SKILLS.filter(skill => !CLAUDE_SKIPPED_SKILL_DIRS.has(skill.dir));
describe('gen-skill-docs', () => {
test('generated SKILL.md contains all command categories', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
const categories = new Set(Object.values(COMMAND_DESCRIPTIONS).map(d => d.category));
for (const cat of categories) {
expect(content).toContain(`### ${cat}`);
}
});
test('generated SKILL.md contains all commands', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
const display = meta.usage || cmd;
expect(content).toContain(display);
}
});
test('command table is sorted alphabetically within categories', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
// Extract command names from the Navigation section as a test
const navSection = content.match(/### Navigation\n\|.*\n\|.*\n([\s\S]*?)(?=\n###|\n## )/);
expect(navSection).not.toBeNull();
const rows = navSection![1].trim().split('\n');
const commands = rows.map(r => {
const match = r.match(/\| `(\w+)/);
return match ? match[1] : '';
}).filter(Boolean);
const sorted = [...commands].sort();
expect(commands).toEqual(sorted);
});
test('generated header is present in SKILL.md', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
expect(content).toContain('Regenerate: bun run gen:skill-docs');
});
test('generated header is present in browse/SKILL.md', () => {
const content = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md'), 'utf-8');
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
});
test('snapshot flags section contains all flags', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
for (const flag of SNAPSHOT_FLAGS) {
expect(content).toContain(flag.short);
expect(content).toContain(flag.description);
}
});
test('every skill has a SKILL.md.tmpl template', () => {
for (const skill of ALL_SKILLS) {
const tmplPath = path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
expect(fs.existsSync(tmplPath)).toBe(true);
}
});
test('every skill has a generated SKILL.md with auto-generated header', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const mdPath = path.join(ROOT, skill.dir, 'SKILL.md');
expect(fs.existsSync(mdPath)).toBe(true);
const content = fs.readFileSync(mdPath, 'utf-8');
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
expect(content).toContain('Regenerate: bun run gen:skill-docs');
}
});
test('every generated SKILL.md has valid YAML frontmatter', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
expect(content.startsWith('---\n')).toBe(true);
expect(content).toContain('name:');
expect(content).toContain('description:');
}
});
test(`every generated SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
const description = extractDescription(content);
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
}
});
test('Claude outside-voice skill is not generated for Claude host', () => {
expect(fs.existsSync(path.join(ROOT, 'claude', 'SKILL.md.tmpl'))).toBe(true);
expect(fs.existsSync(path.join(ROOT, 'claude', 'SKILL.md'))).toBe(false);
});
test(`every Codex SKILL.md description stays within ${MAX_SKILL_DESCRIPTION_LENGTH} chars`, () => {
const agentsDir = path.join(ROOT, '.agents', 'skills');
if (!fs.existsSync(agentsDir)) return; // skip if not generated
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
if (!entry.isDirectory()) continue;
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
const description = extractDescription(content);
expect(description.length).toBeLessThanOrEqual(MAX_SKILL_DESCRIPTION_LENGTH);
}
});
test('every Codex SKILL.md description stays under 900-char warning threshold', () => {
const WARN_THRESHOLD = 900;
const agentsDir = path.join(ROOT, '.agents', 'skills');
if (!fs.existsSync(agentsDir)) return;
const violations: string[] = [];
for (const entry of fs.readdirSync(agentsDir, { withFileTypes: true })) {
if (!entry.isDirectory()) continue;
const skillMd = path.join(agentsDir, entry.name, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
const description = extractDescription(content);
if (description.length > WARN_THRESHOLD) {
violations.push(`${entry.name}: ${description.length} chars (limit ${MAX_SKILL_DESCRIPTION_LENGTH}, ${MAX_SKILL_DESCRIPTION_LENGTH - description.length} remaining)`);
}
}
expect(violations).toEqual([]);
});
test('package.json version matches VERSION file', () => {
const pkg = JSON.parse(fs.readFileSync(path.join(ROOT, 'package.json'), 'utf-8'));
const version = fs.readFileSync(path.join(ROOT, 'VERSION'), 'utf-8').trim();
expect(pkg.version).toBe(version);
});
test('generated files are fresh (match --dry-run)', () => {
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--dry-run'], {
cwd: ROOT,
stdout: 'pipe',
stderr: 'pipe',
});
expect(result.exitCode).toBe(0);
const output = result.stdout.toString();
// Every skill should be FRESH
for (const skill of CLAUDE_GENERATED_SKILLS) {
const file = skill.dir === '.' ? 'SKILL.md' : `${skill.dir}/SKILL.md`;
expect(output).toContain(`FRESH: ${file}`);
}
expect(output).not.toContain('STALE');
});
test('no generated SKILL.md contains unresolved placeholders', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
const unresolved = content.match(/\{\{[A-Z_]+\}\}/g);
expect(unresolved).toBeNull();
}
});
test('templates contain placeholders', () => {
const rootTmpl = fs.readFileSync(path.join(ROOT, 'SKILL.md.tmpl'), 'utf-8');
expect(rootTmpl).toContain('{{COMMAND_REFERENCE}}');
expect(rootTmpl).toContain('{{SNAPSHOT_FLAGS}}');
expect(rootTmpl).toContain('{{PREAMBLE}}');
const browseTmpl = fs.readFileSync(path.join(ROOT, 'browse', 'SKILL.md.tmpl'), 'utf-8');
expect(browseTmpl).toContain('{{COMMAND_REFERENCE}}');
expect(browseTmpl).toContain('{{SNAPSHOT_FLAGS}}');
expect(browseTmpl).toContain('{{PREAMBLE}}');
});
test('generated SKILL.md contains operational self-improvement (replaced contributor mode)', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).not.toContain('Contributor Mode');
expect(content).not.toContain('gstack_contributor');
expect(content).not.toContain('contributor-logs');
expect(content).toContain('Operational Self-Improvement');
expect(content).toContain('gstack-learnings-log');
expect(content).toContain('gstack-learnings-search --limit 3');
});
test('generated SKILL.md with LEARNINGS_LOG contains operational type', () => {
// Check a skill that has LEARNINGS_LOG (e.g., review)
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('operational');
});
test('generated SKILL.md contains session awareness', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('_SESSIONS');
expect(content).toContain('RECOMMENDATION');
});
test('generated SKILL.md contains branch detection', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('_BRANCH');
expect(content).toContain('git branch --show-current');
});
test('tier 2+ skills contain ELI10 simplification rules (AskUserQuestion format)', () => {
// Root SKILL.md is tier 1 (no AskUserQuestion format). Check a tier 2+ skill instead.
// v1.7.0.0 Pros/Cons format uses "ELI10 (ALWAYS)" rather than "Simplify (ELI10".
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
expect(content).toContain('ELI10');
expect(content).toContain('plain English');
expect(content).toContain('not function names');
});
test('tier 1 skills do NOT contain AskUserQuestion format', () => {
// Use benchmark (tier 1) instead of root — root SKILL.md gets overwritten by Codex test setup
const content = fs.readFileSync(path.join(ROOT, 'benchmark', 'SKILL.md'), 'utf-8');
expect(content).not.toContain('## AskUserQuestion Format');
expect(content).not.toContain('## Completeness Principle');
});
test('generated SKILL.md contains telemetry line', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('skill-usage.jsonl');
expect(content).toContain('~/.gstack/analytics');
});
test('plan-review generated preambles stay under the Option A budget', () => {
const reviewSkills = [
{
path: path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
markers: ['# Mega Plan Review Mode', '## Step 0: Detect platform and base branch'],
},
{
path: path.join(ROOT, 'plan-eng-review', 'SKILL.md'),
markers: ['# Plan Review Mode'],
},
];
// Plan skills carry the same preamble surface as other tier-≥2 skills
// (Artifacts Sync, Context Recovery, Routing Injection are load-bearing
// functionality, not optional). Budget is set to current size + small
// headroom; ratchet down if a future slim trims real bytes.
// Ratcheted from 33000 → 35000 when the gbrain context-load block was
// added (per /sync-gbrain plan §4). Ratcheted 35000 → 36500 in v1.27.0.0
// when generate-brain-sync-block.ts gained the gbrain_mcp_mode probe +
// remote-mode ARTIFACTS_SYNC status line (Path 4 of /setup-gbrain).
// Ratcheted 36500 → 39000 in the contributor wave when #1205 added the
// \\u-escape CJK rule (rule 12 + self-check item) to the AskUserQuestion
// preamble.
for (const skill of reviewSkills) {
const content = fs.readFileSync(skill.path, 'utf-8');
const preamble = extractPreambleBeforeWorkflow(content, skill.markers);
expect(Buffer.byteLength(preamble, 'utf-8')).toBeLessThan(39_000);
}
});
test('voice and writing-style preamble sections stay compact', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
const voice = extractMarkdownSection(content, '## Voice');
const writingStyle = extractMarkdownSection(content, '## Writing Style');
expect(Buffer.byteLength(voice, 'utf-8')).toBeLessThan(3_000);
expect(Buffer.byteLength(writingStyle, 'utf-8')).toBeLessThan(2_000);
});
test('slim voice section preserves the gstack voice contract', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
const voice = extractMarkdownSection(content, '## Voice');
expect(voice).toMatch(/lead with the point|direct/i);
expect(voice).toMatch(/file|function|line|command|real numbers/i);
expect(voice).toMatch(/user.*outcome|user.*experience|real user/i);
expect(voice).toMatch(/corporate|academic|PR|hype/i);
expect(voice).toMatch(/AI vocabulary|delve|crucial|robust/i);
expect(voice).toMatch(/user decides|user.*context|sovereignty|recommendation, not a decision/i);
});
test('preamble .pending-* glob is zsh-safe (uses find, not shell glob)', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
if (!content.includes('.pending-')) continue;
// Must NOT have a bare shell glob ".pending-*" outside of find's -name argument
expect(content).not.toMatch(/for _PF in [^\n]*\/\.pending-\*/);
// Must use find to avoid zsh NOMATCH error on glob expansion
expect(content).toContain("find ~/.gstack/analytics -maxdepth 1 -name '.pending-*'");
}
});
test('bash blocks with shell globs are zsh-safe (setopt guard or find)', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
const bashBlocks = [...content.matchAll(/```bash\n([\s\S]*?)```/g)].map(m => m[1]);
for (const block of bashBlocks) {
const lines = block.split('\n');
for (const line of lines) {
const trimmed = line.trimStart();
if (trimmed.startsWith('#')) continue;
if (!trimmed.includes('*')) continue;
// Skip lines where * is inside find -name, git pathspecs, or $(find)
if (/\bfind\b/.test(trimmed)) continue;
if (/\bgit\b/.test(trimmed)) continue;
if (/\$\(find\b/.test(trimmed)) continue;
// Check 1: "for VAR in <glob>" must use $(find ...) — caught above by the
// $(find check, so any surviving for-in with a glob pattern is a violation
if (/\bfor\s+\w+\s+in\b/.test(trimmed) && /\*\./.test(trimmed)) {
throw new Error(
`Unsafe for-in glob in ${skill.dir}/SKILL.md: "${trimmed}". ` +
`Use \`for f in $(find ... -name '*.ext')\` for zsh compatibility.`
);
}
// Check 2: ls/cat/rm/grep with glob file args must have setopt guard
const isGlobCmd = /\b(?:ls|cat|rm|grep)\b/.test(trimmed) &&
/(?:\/\*[a-z.*]|\*\.[a-z])/.test(trimmed);
if (isGlobCmd) {
expect(block).toContain('setopt +o nomatch');
}
}
}
}
});
test('preamble-using skills have correct skill name in telemetry', () => {
const PREAMBLE_SKILLS = [
{ dir: '.', name: 'gstack' },
{ dir: 'ship', name: 'ship' },
{ dir: 'review', name: 'review' },
{ dir: 'qa', name: 'qa' },
{ dir: 'retro', name: 'retro' },
];
for (const skill of PREAMBLE_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
expect(content).toContain(`"skill":"${skill.name}"`);
}
});
test('qa and qa-only templates use QA_METHODOLOGY placeholder', () => {
const qaTmpl = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md.tmpl'), 'utf-8');
expect(qaTmpl).toContain('{{QA_METHODOLOGY}}');
const qaOnlyTmpl = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md.tmpl'), 'utf-8');
expect(qaOnlyTmpl).toContain('{{QA_METHODOLOGY}}');
});
test('QA_METHODOLOGY appears expanded in both qa and qa-only generated files', () => {
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
// Both should contain the health score rubric
expect(qaContent).toContain('Health Score Rubric');
expect(qaOnlyContent).toContain('Health Score Rubric');
// Both should contain framework guidance
expect(qaContent).toContain('Framework-Specific Guidance');
expect(qaOnlyContent).toContain('Framework-Specific Guidance');
// Both should contain the important rules
expect(qaContent).toContain('Important Rules');
expect(qaOnlyContent).toContain('Important Rules');
// Both should contain the 6 phases
expect(qaContent).toContain('Phase 1');
expect(qaOnlyContent).toContain('Phase 1');
expect(qaContent).toContain('Phase 6');
expect(qaOnlyContent).toContain('Phase 6');
});
test('qa-only has no-fix guardrails', () => {
const qaOnlyContent = fs.readFileSync(path.join(ROOT, 'qa-only', 'SKILL.md'), 'utf-8');
expect(qaOnlyContent).toContain('Never fix bugs');
expect(qaOnlyContent).toContain('NEVER fix anything');
// Should not have Edit, Glob, or Grep in allowed-tools.
// Scope to frontmatter (between the first two --- lines) — the body can
// legitimately mention these tool names in prose (e.g., Claude model
// overlay says "prefer Read, Edit, Write, Glob, Grep over Bash").
const fmMatch = qaOnlyContent.match(/^---\n([\s\S]*?)\n---/);
expect(fmMatch).not.toBeNull();
const frontmatter = fmMatch![1];
expect(frontmatter).toMatch(/allowed-tools:/);
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Edit/);
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Glob/);
expect(frontmatter).not.toMatch(/allowed-tools:[\s\S]*?- Grep/);
});
test('qa has fix-loop tools and phases', () => {
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
// Should have Edit, Glob, Grep in allowed-tools
expect(qaContent).toContain('Edit');
expect(qaContent).toContain('Glob');
expect(qaContent).toContain('Grep');
// Should have fix-loop phases
expect(qaContent).toContain('Phase 7');
expect(qaContent).toContain('Phase 8');
expect(qaContent).toContain('Fix Loop');
expect(qaContent).toContain('Triage');
expect(qaContent).toContain('WTF');
});
});
describe('BASE_BRANCH_DETECT resolver', () => {
// Find a generated SKILL.md that uses the placeholder (ship is guaranteed to)
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
test('resolver output contains PR base detection command', () => {
expect(shipContent).toContain('gh pr view --json baseRefName');
});
test('resolver output contains repo default branch detection command', () => {
expect(shipContent).toContain('gh repo view --json defaultBranchRef');
});
test('resolver output contains fallback to main', () => {
expect(shipContent).toMatch(/fall\s*back\s+to\s+`main`/i);
});
test('resolver output uses "the base branch" phrasing', () => {
expect(shipContent).toContain('the base branch');
});
test('resolver output contains GitLab CLI commands', () => {
expect(shipContent).toContain('glab');
});
test('resolver output contains git-native fallback', () => {
expect(shipContent).toContain('git symbolic-ref');
});
test('resolver output mentions GitLab platform', () => {
expect(shipContent).toMatch(/gitlab/i);
});
});
describe('GitLab support in generated skills', () => {
const retroContent = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
const shipSkillContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
test('retro contains GitLab MR number extraction', () => {
expect(retroContent).toContain('[#!]');
});
test('retro uses BASE_BRANCH_DETECT (contains glab)', () => {
expect(retroContent).toContain('glab');
});
test('ship contains glab mr create', () => {
expect(shipSkillContent).toContain('glab mr create');
});
test('ship checks .gitlab-ci.yml', () => {
expect(shipSkillContent).toContain('.gitlab-ci.yml');
});
});
/**
* Quality evals — catch description regressions.
*
* These test that generated output is *useful for an AI agent*,
* not just structurally valid. Each test targets a specific
* regression we actually shipped and caught in review.
*/
describe('description quality evals', () => {
// Regression: snapshot flags lost value hints (-d <N>, -s <sel>, -o <path>)
test('snapshot flags with values include value hints in output', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
for (const flag of SNAPSHOT_FLAGS) {
if (flag.takesValue) {
expect(flag.valueHint).toBeDefined();
expect(content).toContain(`${flag.short} ${flag.valueHint}`);
}
}
});
// Regression: "is" lost the valid states enum
test('is command lists valid state values', () => {
const desc = COMMAND_DESCRIPTIONS['is'].description;
for (const state of ['visible', 'hidden', 'enabled', 'disabled', 'checked', 'editable', 'focused']) {
expect(desc).toContain(state);
}
});
// Regression: "press" lost common key examples
test('press command lists example keys', () => {
const desc = COMMAND_DESCRIPTIONS['press'].description;
expect(desc).toContain('Enter');
expect(desc).toContain('Tab');
expect(desc).toContain('Escape');
});
// Regression: "console" lost --errors filter note
test('console command describes --errors behavior', () => {
const desc = COMMAND_DESCRIPTIONS['console'].description;
expect(desc).toContain('--errors');
});
// Regression: snapshot -i lost "@e refs" context
test('snapshot -i mentions @e refs', () => {
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-i')!;
expect(flag.description).toContain('@e');
});
// Regression: snapshot -C lost "@c refs" context
test('snapshot -C mentions @c refs', () => {
const flag = SNAPSHOT_FLAGS.find(f => f.short === '-C')!;
expect(flag.description).toContain('@c');
});
// Guard: every description must be at least 8 chars (catches empty or stub descriptions)
test('all command descriptions have meaningful length', () => {
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
expect(meta.description.length).toBeGreaterThanOrEqual(8);
}
});
// Guard: snapshot flag descriptions must be at least 10 chars
test('all snapshot flag descriptions have meaningful length', () => {
for (const flag of SNAPSHOT_FLAGS) {
expect(flag.description.length).toBeGreaterThanOrEqual(10);
}
});
// Guard: descriptions must not contain pipe (breaks markdown table cells)
// Usage strings are backtick-wrapped in the table so pipes there are safe.
test('no command description contains pipe character', () => {
for (const [cmd, meta] of Object.entries(COMMAND_DESCRIPTIONS)) {
expect(meta.description).not.toContain('|');
}
});
// Guard: generated output uses → not ->
test('generated SKILL.md uses unicode arrows', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
// Check the Tips section specifically (where we regressed -> from →)
const tipsSection = content.slice(content.indexOf('## Tips'));
expect(tipsSection).toContain('→');
expect(tipsSection).not.toContain('->');
});
});
describe('REVIEW_DASHBOARD resolver', () => {
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review'];
for (const skill of REVIEW_SKILLS) {
test(`review dashboard appears in ${skill} generated file`, () => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).toContain('gstack-review');
expect(content).toContain('REVIEW READINESS DASHBOARD');
});
}
test('review dashboard appears in ship generated file', () => {
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
expect(content).toContain('reviews.jsonl');
expect(content).toContain('REVIEW READINESS DASHBOARD');
});
test('dashboard treats review as a valid Eng Review source', () => {
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
expect(content).toContain('plan-eng-review, review, plan-design-review');
expect(content).toContain('`review` (diff-scoped pre-landing review)');
expect(content).toContain('`plan-eng-review` (plan-stage architecture review)');
expect(content).toContain('from either \\`review\\` or \\`plan-eng-review\\`');
});
test('shared dashboard propagates review source to plan-eng-review', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('plan-eng-review, review, plan-design-review');
expect(content).toContain('`review` (diff-scoped pre-landing review)');
});
test('resolver output contains key dashboard elements', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('VERDICT');
expect(content).toContain('CLEARED');
expect(content).toContain('Eng Review');
expect(content).toContain('7 days');
expect(content).toContain('Design Review');
expect(content).toContain('skip_eng_review');
});
test('dashboard bash block includes git HEAD for staleness detection', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('git rev-parse --short HEAD');
expect(content).toContain('---HEAD---');
});
test('dashboard includes staleness detection prose', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Staleness detection');
expect(content).toContain('commit');
});
for (const skill of REVIEW_SKILLS) {
test(`${skill} contains review chaining section`, () => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).toContain('Review Chaining');
});
test(`${skill} Review Log includes commit field`, () => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).toContain('"commit"');
});
}
test('plan-ceo-review chaining mentions eng and design reviews', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('/plan-eng-review');
expect(content).toContain('/plan-design-review');
});
test('plan-eng-review chaining mentions design and ceo reviews', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('/plan-design-review');
expect(content).toContain('/plan-ceo-review');
});
test('plan-design-review chaining mentions eng, ceo, and design skills', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('/plan-eng-review');
expect(content).toContain('/plan-ceo-review');
expect(content).toContain('/design-shotgun');
expect(content).toContain('/design-html');
});
test('ship does NOT contain review chaining', () => {
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
expect(content).not.toContain('Review Chaining');
});
});
// ─── Test Coverage Audit Resolver Tests ─────────────────────
describe('TEST_COVERAGE_AUDIT placeholders', () => {
const planSkill = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
test('plan and ship modes share codepath tracing methodology', () => {
// Review mode delegates test coverage to the Testing specialist subagent (Review Army)
const sharedPhrases = [
'Trace data flow',
'Diagram the execution',
'Quality scoring rubric',
'★★★',
'★★',
'GAP',
];
for (const phrase of sharedPhrases) {
expect(planSkill).toContain(phrase);
expect(shipSkill).toContain(phrase);
}
// Plan mode traces the plan, not a git diff
expect(planSkill).toContain('Trace every codepath in the plan');
expect(planSkill).not.toContain('git diff origin');
// Ship mode traces the diff
expect(shipSkill).toContain('Trace every codepath changed');
});
test('review mode uses Review Army for specialist dispatch', () => {
expect(reviewSkill).toContain('Review Army');
expect(reviewSkill).toContain('Specialist Dispatch');
expect(reviewSkill).toContain('testing.md');
});
test('plan and ship modes include E2E decision matrix', () => {
// Review mode delegates to Testing specialist
for (const skill of [planSkill, shipSkill]) {
expect(skill).toContain('E2E Test Decision Matrix');
expect(skill).toContain('→E2E');
expect(skill).toContain('→EVAL');
}
});
test('plan and ship modes include regression rule', () => {
// Review mode delegates to Testing specialist
for (const skill of [planSkill, shipSkill]) {
expect(skill).toContain('REGRESSION RULE');
expect(skill).toContain('IRON RULE');
}
});
test('plan and ship modes include test framework detection', () => {
// Review mode delegates to Testing specialist
for (const skill of [planSkill, shipSkill]) {
expect(skill).toContain('Test Framework Detection');
expect(skill).toContain('CLAUDE.md');
}
});
test('plan mode adds tests to plan + includes test plan artifact', () => {
expect(planSkill).toContain('Add missing tests to the plan');
expect(planSkill).toContain('eng-review-test-plan');
expect(planSkill).toContain('Test Plan Artifact');
});
test('ship mode auto-generates tests + includes before/after count', () => {
expect(shipSkill).toContain('Generate tests for uncovered paths');
expect(shipSkill).toContain('Before/after test count');
expect(shipSkill).toContain('30 code paths max');
expect(shipSkill).toContain('ship-test-plan');
});
test('review mode uses Fix-First + Review Army for specialist coverage', () => {
expect(reviewSkill).toContain('Fix-First');
expect(reviewSkill).toContain('INFORMATIONAL');
// Review Army handles test coverage via Testing specialist subagent
expect(reviewSkill).toContain('Review Army');
expect(reviewSkill).toContain('Testing');
});
test('plan mode does NOT include ship-specific content', () => {
expect(planSkill).not.toContain('Before/after test count');
expect(planSkill).not.toContain('30 code paths max');
expect(planSkill).not.toContain('ship-test-plan');
});
test('review mode does NOT include test plan artifact', () => {
expect(reviewSkill).not.toContain('Test Plan Artifact');
expect(reviewSkill).not.toContain('eng-review-test-plan');
expect(reviewSkill).not.toContain('ship-test-plan');
});
test('review/specialists/ directory has all expected checklist files', () => {
const specDir = path.join(ROOT, 'review', 'specialists');
const expected = [
'testing.md',
'maintainability.md',
'security.md',
'performance.md',
'data-migration.md',
'api-contract.md',
'red-team.md',
];
for (const f of expected) {
expect(fs.existsSync(path.join(specDir, f))).toBe(true);
}
});
test('each specialist file has standard header with scope and output format', () => {
const specDir = path.join(ROOT, 'review', 'specialists');
const files = fs.readdirSync(specDir).filter(f => f.endsWith('.md'));
for (const f of files) {
const content = fs.readFileSync(path.join(specDir, f), 'utf-8');
// All specialist files must have Scope and Output/JSON in header
expect(content).toContain('Scope:');
expect(content.toLowerCase()).toMatch(/output|json/);
// Must define NO FINDINGS behavior
expect(content).toContain('NO FINDINGS');
}
});
// Regression guard: ship output contains key phrases from before the refactor
test('ship SKILL.md regression guard — key phrases preserved', () => {
const regressionPhrases = [
'100% coverage is the goal',
'ASCII coverage diagram',
'processPayment',
'refundPayment',
'billing.test.ts',
'checkout.e2e.ts',
'COVERAGE:',
'QUALITY:',
'GAPS:',
'Code paths:',
'User flows:',
];
for (const phrase of regressionPhrases) {
expect(shipSkill).toContain(phrase);
}
});
test('ship SKILL.md contains review army specialist dispatch', () => {
expect(shipSkill).toContain('Specialist Dispatch');
expect(shipSkill).toContain('Step 9.1');
expect(shipSkill).toContain('Step 9.2');
});
test('ship SKILL.md contains cross-review finding dedup', () => {
expect(shipSkill).toContain('Cross-review finding dedup');
expect(shipSkill).toContain('Step 9.3');
});
test('ship SKILL.md contains re-run idempotency behavior', () => {
expect(shipSkill).toContain('Re-run behavior (idempotency)');
expect(shipSkill).toContain('Never skip a verification step');
});
});
// --- {{TEST_FAILURE_TRIAGE}} resolver tests ---
describe('TEST_FAILURE_TRIAGE resolver', () => {
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
test('contains all 4 triage steps', () => {
expect(shipSkill).toContain('Step T1: Classify each failure');
expect(shipSkill).toContain('Step T2: Handle in-branch failures');
expect(shipSkill).toContain('Step T3: Handle pre-existing failures');
expect(shipSkill).toContain('Step T4: Execute the chosen action');
});
test('T1 includes classification criteria (in-branch vs pre-existing)', () => {
expect(shipSkill).toContain('In-branch');
expect(shipSkill).toContain('Likely pre-existing');
expect(shipSkill).toContain('git diff origin/');
});
test('T3 branches on REPO_MODE (solo vs collaborative)', () => {
expect(shipSkill).toContain('REPO_MODE');
expect(shipSkill).toContain('solo');
expect(shipSkill).toContain('collaborative');
});
test('solo mode offers fix-now, TODO, and skip options', () => {
expect(shipSkill).toContain('Investigate and fix now');
expect(shipSkill).toContain('Add as P0 TODO');
expect(shipSkill).toContain('Skip');
});
test('collaborative mode offers blame + assign option', () => {
expect(shipSkill).toContain('Blame + assign GitHub issue');
expect(shipSkill).toContain('gh issue create');
});
test('defaults ambiguous failures to in-branch (safety)', () => {
expect(shipSkill).toContain('When ambiguous, default to in-branch');
});
});
// --- {{PLAN_FILE_REVIEW_REPORT}} resolver tests ---
describe('PLAN_FILE_REVIEW_REPORT resolver', () => {
const REVIEW_SKILLS = ['plan-ceo-review', 'plan-eng-review', 'plan-design-review', 'codex'];
for (const skill of REVIEW_SKILLS) {
test(`plan file review report appears in ${skill} generated file`, () => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).toContain('GSTACK REVIEW REPORT');
});
}
test('resolver output contains key report elements', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Trigger');
expect(content).toContain('Findings');
expect(content).toContain('VERDICT');
expect(content).toContain('/plan-ceo-review');
expect(content).toContain('/plan-eng-review');
expect(content).toContain('/plan-design-review');
expect(content).toContain('/codex review');
});
});
// --- {{PLAN_COMPLETION_AUDIT}} resolver tests ---
describe('PLAN_COMPLETION_AUDIT placeholders', () => {
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
test('ship SKILL.md contains plan completion audit step', () => {
expect(shipSkill).toContain('Plan Completion Audit');
expect(shipSkill).toContain('Step 8');
});
test('review SKILL.md contains plan completion in scope drift', () => {
expect(reviewSkill).toContain('Plan File Discovery');
expect(reviewSkill).toContain('Actionable Item Extraction');
expect(reviewSkill).toContain('Integration with Scope Drift Detection');
});
test('both modes share plan file discovery methodology', () => {
expect(shipSkill).toContain('Plan File Discovery');
expect(reviewSkill).toContain('Plan File Discovery');
// Both should have conversation context first
expect(shipSkill).toContain('Conversation context (primary)');
expect(reviewSkill).toContain('Conversation context (primary)');
// Both should have grep fallback
expect(shipSkill).toContain('Content-based search (fallback)');
expect(reviewSkill).toContain('Content-based search (fallback)');
});
test('ship mode has gate logic for NOT DONE items', () => {
expect(shipSkill).toContain('NOT DONE');
expect(shipSkill).toContain('Stop — implement the missing items');
expect(shipSkill).toContain('Ship anyway — defer');
expect(shipSkill).toContain('intentionally dropped');
});
test('review mode is INFORMATIONAL only', () => {
expect(reviewSkill).toContain('INFORMATIONAL');
expect(reviewSkill).toContain('MISSING REQUIREMENTS');
expect(reviewSkill).toContain('SCOPE CREEP');
});
test('item extraction has 50-item cap', () => {
expect(shipSkill).toContain('at most 50 items');
});
test('uses file-level traceability (not commit-level)', () => {
expect(shipSkill).toContain('Cite the specific file');
expect(shipSkill).not.toContain('commit-level traceability');
});
});
// --- {{PLAN_VERIFICATION_EXEC}} resolver tests ---
describe('PLAN_VERIFICATION_EXEC placeholder', () => {
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
test('ship SKILL.md contains plan verification step', () => {
expect(shipSkill).toContain('Step 8.1');
expect(shipSkill).toContain('Plan Verification');
});
test('references /qa-only invocation', () => {
expect(shipSkill).toContain('qa-only/SKILL.md');
expect(shipSkill).toContain('qa-only');
});
test('contains localhost reachability check', () => {
expect(shipSkill).toContain('localhost:3000');
expect(shipSkill).toContain('NO_SERVER');
});
test('skips gracefully when no verification section', () => {
expect(shipSkill).toContain('No verification steps found in plan');
});
test('skips gracefully when no dev server', () => {
expect(shipSkill).toContain('No dev server detected');
});
});
// --- Coverage gate tests ---
describe('Coverage gate in ship', () => {
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
test('ship SKILL.md contains coverage gate with thresholds', () => {
expect(shipSkill).toContain('Coverage gate');
expect(shipSkill).toContain('>= target');
expect(shipSkill).toContain('< minimum');
});
test('ship SKILL.md supports configurable thresholds via CLAUDE.md', () => {
expect(shipSkill).toContain('## Test Coverage');
expect(shipSkill).toContain('Minimum:');
expect(shipSkill).toContain('Target:');
});
test('coverage gate skips on parse failure (not block)', () => {
expect(shipSkill).toContain('could not determine percentage — skipping');
});
test('review SKILL.md delegates coverage to Testing specialist', () => {
// Coverage audit moved to Testing specialist subagent in Review Army
expect(reviewSkill).toContain('testing.md');
expect(reviewSkill).toContain('INFORMATIONAL');
});
});
// --- Ship metrics logging ---
describe('Ship metrics logging', () => {
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
test('ship SKILL.md contains metrics persistence step', () => {
expect(shipSkill).toContain('Step 20');
expect(shipSkill).toContain('coverage_pct');
expect(shipSkill).toContain('plan_items_total');
expect(shipSkill).toContain('plan_items_done');
expect(shipSkill).toContain('verification_result');
});
});
// --- Plan file discovery shared helper ---
describe('Plan file discovery shared helper', () => {
// The shared helper should appear in ship (via PLAN_COMPLETION_AUDIT_SHIP)
// and in review (via PLAN_COMPLETION_AUDIT_REVIEW)
const shipSkill = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
const reviewSkill = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
test('plan file discovery appears in both ship and review', () => {
expect(shipSkill).toContain('Plan File Discovery');
expect(reviewSkill).toContain('Plan File Discovery');
});
test('both include conversation context first', () => {
expect(shipSkill).toContain('Conversation context (primary)');
expect(reviewSkill).toContain('Conversation context (primary)');
});
test('both include content-based fallback', () => {
expect(shipSkill).toContain('Content-based search (fallback)');
expect(reviewSkill).toContain('Content-based search (fallback)');
});
});
// --- Retro plan completion ---
describe('Retro plan completion section', () => {
const retroSkill = fs.readFileSync(path.join(ROOT, 'retro', 'SKILL.md'), 'utf-8');
test('retro SKILL.md contains plan completion section', () => {
expect(retroSkill).toContain('### Plan Completion');
expect(retroSkill).toContain('plan_items_total');
expect(retroSkill).toContain('Plan Completion This Period');
});
});
// --- Plan status footer in preamble ---
describe('Plan status footer in preamble', () => {
test('preamble contains plan status footer as neutral forward reference to EXIT PLAN MODE GATE', () => {
// Read any skill that uses PREAMBLE
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
expect(content).toContain('Plan Status Footer');
expect(content).toContain('GSTACK REVIEW REPORT');
expect(content).toContain('ExitPlanMode');
expect(content).toContain('EXIT PLAN MODE GATE');
// The preamble must NOT impose review-report rules on operational skills
// that have no review report. It's a forward reference, not enforcement.
expect(content).not.toContain('NO REVIEWS YET');
});
});
// --- make-pdf setup ordering ---
describe('make-pdf setup ordering', () => {
test('MAKE-PDF SETUP appears before generic preamble footer sections', () => {
const content = fs.readFileSync(path.join(ROOT, 'make-pdf', 'SKILL.md'), 'utf-8');
const preambleIdx = content.indexOf('## Preamble (run first)');
const setupIdx = content.indexOf('## MAKE-PDF SETUP');
const planModeIdx = content.indexOf('## Plan Mode Safe Operations');
const telemetryIdx = content.indexOf('## Telemetry (run last)');
const workflowIdx = content.indexOf('# make-pdf: publication-quality PDFs from markdown');
expect(preambleIdx).toBeGreaterThanOrEqual(0);
expect(setupIdx).toBeGreaterThan(preambleIdx);
expect(setupIdx).toBeLessThan(planModeIdx);
expect(setupIdx).toBeLessThan(telemetryIdx);
expect(setupIdx).toBeLessThan(workflowIdx);
expect(content.match(/^## MAKE-PDF SETUP/gm)?.length ?? 0).toBe(1);
});
});
// --- Skill invocation during plan mode in preamble ---
describe('Skill invocation during plan mode in preamble', () => {
test('preamble contains skill invocation plan mode section', () => {
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
expect(content).toContain('Skill Invocation During Plan Mode');
expect(content).toContain('precedence over generic plan mode behavior');
expect(content).toContain('Do not continue the workflow');
expect(content).toContain('cancel the skill or leave plan mode');
});
});
// --- {{SPEC_REVIEW_LOOP}} resolver tests ---
describe('SPEC_REVIEW_LOOP resolver', () => {
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
test('contains all 5 review dimensions', () => {
for (const dim of ['Completeness', 'Consistency', 'Clarity', 'Scope', 'Feasibility']) {
expect(content).toContain(dim);
}
});
test('references Agent tool for subagent dispatch', () => {
expect(content).toMatch(/Agent.*tool/i);
});
test('specifies max 3 iterations', () => {
expect(content).toMatch(/3.*iteration|maximum.*3/i);
});
test('includes quality score', () => {
expect(content).toContain('quality score');
});
test('includes metrics path', () => {
expect(content).toContain('spec-review.jsonl');
});
test('includes convergence guard', () => {
expect(content).toMatch(/[Cc]onvergence/);
});
test('includes graceful failure handling', () => {
expect(content).toMatch(/skip.*review|unavailable/i);
});
});
// --- {{DESIGN_SKETCH}} resolver tests ---
describe('DESIGN_SKETCH resolver', () => {
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
test('references DESIGN.md for design system constraints', () => {
expect(content).toContain('DESIGN.md');
});
test('contains wireframe or sketch terminology', () => {
expect(content).toMatch(/wireframe|sketch/i);
});
test('references browse binary for rendering', () => {
expect(content).toContain('$B goto');
});
test('references screenshot capture', () => {
expect(content).toContain('$B screenshot');
});
test('specifies rough aesthetic', () => {
expect(content).toMatch(/[Rr]ough|hand-drawn/);
});
test('includes skip conditions', () => {
expect(content).toMatch(/no UI component|skip/i);
});
});
// --- {{CODEX_SECOND_OPINION}} resolver tests ---
describe('CODEX_SECOND_OPINION resolver', () => {
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
const codexContent = fs.readFileSync(path.join(ROOT, '.agents', 'skills', 'gstack-office-hours', 'SKILL.md'), 'utf-8');
test('Phase 3.5 section appears in office-hours SKILL.md', () => {
expect(content).toContain('Phase 3.5: Cross-Model Second Opinion');
});
test('contains codex exec invocation', () => {
expect(content).toContain('codex exec');
});
test('contains opt-in AskUserQuestion text', () => {
expect(content).toContain('second opinion from an independent AI perspective');
});
test('contains cross-model synthesis instructions', () => {
expect(content).toMatch(/[Ss]ynthesis/);
expect(content).toContain('Where Claude agrees with the second opinion');
});
test('contains Claude subagent fallback', () => {
expect(content).toContain('CODEX_NOT_AVAILABLE');
expect(content).toContain('Agent tool');
expect(content).toContain('SECOND OPINION (Claude subagent)');
});
test('contains premise revision check', () => {
expect(content).toContain('Codex challenged premise');
});
test('contains error handling for auth, timeout, and empty', () => {
expect(content).toMatch(/[Aa]uth.*fail/);
expect(content).toMatch(/[Tt]imeout/);
expect(content).toMatch(/[Ee]mpty response/);
});
test('Codex host variant does NOT contain the Phase 3.5 resolver output', () => {
// The resolver returns '' for codex host, so the interactive section is stripped.
// Static template references to "Phase 3.5" in prose/conditionals are fine.
// Other resolvers (design review lite) may contain CODEX_NOT_AVAILABLE, so we
// check for Phase 3.5-specific markers only.
expect(codexContent).not.toContain('Phase 3.5: Cross-Model Second Opinion');
expect(codexContent).not.toContain('TMPERR_OH');
expect(codexContent).not.toContain('gstack-codex-oh-');
});
});
// --- Codex filesystem boundary tests ---
describe('Codex filesystem boundary', () => {
// Skills that call codex exec/review and should contain boundary text
const CODEX_CALLING_SKILLS = [
'codex', // /codex skill — 3 modes
'autoplan', // /autoplan — CEO/design/eng voices
'review', // /review — adversarial step resolver
'ship', // /ship — adversarial step resolver
'plan-eng-review', // outside voice resolver
'plan-ceo-review', // outside voice resolver
'office-hours', // second opinion resolver
];
const BOUNDARY_MARKER = 'Do NOT read or execute any';
test('boundary instruction appears in all skills that call codex', () => {
for (const skill of CODEX_CALLING_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).toContain(BOUNDARY_MARKER);
}
});
test('codex skill has Filesystem Boundary section', () => {
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
expect(content).toContain('## Filesystem Boundary');
expect(content).toContain('skill definitions meant for a different AI system');
});
test('codex skill has rabbit-hole detection rule', () => {
const content = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
expect(content).toContain('Detect skill-file rabbit holes');
expect(content).toContain('gstack-update-check');
expect(content).toContain('Consider retrying');
});
test('review.ts CODEX_BOUNDARY constant is interpolated into resolver output', () => {
// The adversarial step resolver should include boundary text in codex exec prompts
const reviewContent = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
// Boundary should appear near codex exec invocations
const boundaryIdx = reviewContent.indexOf(BOUNDARY_MARKER);
const codexExecIdx = reviewContent.indexOf('codex exec');
// Both must exist and boundary must come before a codex exec call
expect(boundaryIdx).toBeGreaterThan(-1);
expect(codexExecIdx).toBeGreaterThan(-1);
});
test('autoplan boundary text avoids host-specific paths for cross-host compatibility', () => {
const content = fs.readFileSync(path.join(ROOT, 'autoplan', 'SKILL.md.tmpl'), 'utf-8');
// autoplan template uses generic 'skills/gstack' pattern instead of host-specific
// paths like ~/.claude/ or .agents/skills (which break Codex/Claude output tests)
const boundaryStart = content.indexOf('Filesystem Boundary');
const boundaryEnd = content.indexOf('---', boundaryStart + 1);
const boundarySection = content.slice(boundaryStart, boundaryEnd);
expect(boundarySection).not.toContain('~/.claude/');
expect(boundarySection).not.toContain('.agents/skills');
expect(boundarySection).toContain('skills/gstack');
expect(boundarySection).toContain(BOUNDARY_MARKER);
});
});
// --- {{BENEFITS_FROM}} resolver tests ---
describe('BENEFITS_FROM resolver', () => {
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
const engContent = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
test('plan-ceo-review contains prerequisite skill offer', () => {
expect(ceoContent).toContain('Prerequisite Skill Offer');
expect(ceoContent).toContain('/office-hours');
});
test('plan-eng-review contains prerequisite skill offer', () => {
expect(engContent).toContain('Prerequisite Skill Offer');
expect(engContent).toContain('/office-hours');
});
test('offer includes graceful decline', () => {
expect(ceoContent).toContain('No worries');
});
test('skills without benefits-from do NOT have prerequisite offer', () => {
const qaContent = fs.readFileSync(path.join(ROOT, 'qa', 'SKILL.md'), 'utf-8');
expect(qaContent).not.toContain('Prerequisite Skill Offer');
});
test('inline invocation — no "another window" language', () => {
expect(ceoContent).not.toContain('another window');
expect(engContent).not.toContain('another window');
});
test('inline invocation — read-and-follow path present', () => {
expect(ceoContent).toContain('office-hours/SKILL.md');
expect(engContent).toContain('office-hours/SKILL.md');
});
test('BENEFITS_FROM delegates to INVOKE_SKILL pattern', () => {
// Should contain the INVOKE_SKILL-style loading prose (not the old manual skip list)
expect(engContent).toContain('Follow its instructions from top to bottom');
expect(engContent).toContain('skipping these sections');
expect(ceoContent).toContain('Follow its instructions from top to bottom');
});
});
// --- {{INVOKE_SKILL}} resolver tests ---
describe('INVOKE_SKILL resolver', () => {
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
test('plan-ceo-review uses INVOKE_SKILL for mid-session office-hours fallback', () => {
// The mid-session detection path should use INVOKE_SKILL-generated prose
expect(ceoContent).toContain('office-hours/SKILL.md');
expect(ceoContent).toContain('Follow its instructions from top to bottom');
});
test('INVOKE_SKILL output includes default skip list', () => {
expect(ceoContent).toContain('Preamble (run first)');
expect(ceoContent).toContain('Telemetry (run last)');
expect(ceoContent).toContain('AskUserQuestion Format');
});
test('INVOKE_SKILL output includes error handling', () => {
expect(ceoContent).toContain('If unreadable');
expect(ceoContent).toContain('Could not load');
});
test('template uses {{INVOKE_SKILL:office-hours}} placeholder', () => {
const tmpl = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md.tmpl'), 'utf-8');
expect(tmpl).toContain('{{INVOKE_SKILL:office-hours}}');
});
});
// --- {{CHANGELOG_WORKFLOW}} resolver tests ---
describe('CHANGELOG_WORKFLOW resolver', () => {
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
test('ship SKILL.md contains changelog workflow', () => {
expect(shipContent).toContain('CHANGELOG (auto-generate)');
expect(shipContent).toContain('git log <base>..HEAD --oneline');
});
test('changelog workflow includes cross-check step', () => {
expect(shipContent).toContain('Cross-check');
expect(shipContent).toContain('Every commit must map to at least one bullet point');
});
test('changelog workflow includes voice guidance', () => {
expect(shipContent).toContain('Lead with what the user can now **do**');
});
test('template uses {{CHANGELOG_WORKFLOW}} placeholder', () => {
const tmpl = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md.tmpl'), 'utf-8');
expect(tmpl).toContain('{{CHANGELOG_WORKFLOW}}');
// Should NOT contain the old inline changelog content
expect(tmpl).not.toContain('Group commits by theme');
});
test('changelog workflow includes keep-changelog format', () => {
expect(shipContent).toContain('### Added');
expect(shipContent).toContain('### Fixed');
});
});
// --- Parameterized resolver infrastructure tests ---
describe('parameterized resolver support', () => {
test('gen-skill-docs regex handles colon-separated args', () => {
// Verify the template containing {{INVOKE_SKILL:office-hours}} was processed
// without leaving unresolved placeholders
const ceoContent = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
expect(ceoContent).not.toMatch(/\{\{INVOKE_SKILL:[^}]+\}\}/);
});
test('templates with parameterized resolvers pass unresolved check', () => {
// All generated SKILL.md files should have no unresolved {{...}} placeholders
const skillDirs = fs.readdirSync(ROOT).filter(d =>
fs.existsSync(path.join(ROOT, d, 'SKILL.md'))
);
for (const dir of skillDirs) {
const content = fs.readFileSync(path.join(ROOT, dir, 'SKILL.md'), 'utf-8');
const unresolved = content.match(/\{\{[A-Z_]+(?::[^}]*)?\}\}/g);
if (unresolved) {
throw new Error(`${dir}/SKILL.md has unresolved placeholders: ${unresolved.join(', ')}`);
}
}
});
});
// --- Preamble routing injection tests ---
describe('preamble routing injection', () => {
const shipContent = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
test('preamble bash checks for routing section in CLAUDE.md', () => {
expect(shipContent).toContain('grep -q "## Skill routing" CLAUDE.md');
expect(shipContent).toContain('HAS_ROUTING');
});
test('preamble bash reads routing_declined config', () => {
expect(shipContent).toContain('routing_declined');
expect(shipContent).toContain('ROUTING_DECLINED');
});
test('preamble includes routing injection AskUserQuestion', () => {
expect(shipContent).toContain('Add routing rules to CLAUDE.md');
expect(shipContent).toContain("I'll invoke skills manually");
});
test('routing injection respects prior decline', () => {
expect(shipContent).toContain('ROUTING_DECLINED');
expect(shipContent).toMatch(/routing_declined.*true/);
});
test('routing injection only fires when all conditions met', () => {
// Must be: HAS_ROUTING=no AND ROUTING_DECLINED=false AND PROACTIVE_PROMPTED=yes
expect(shipContent).toContain('HAS_ROUTING');
expect(shipContent).toContain('ROUTING_DECLINED');
expect(shipContent).toContain('PROACTIVE_PROMPTED');
});
test('routing section content includes key routing rules', () => {
expect(shipContent).toContain('invoke /office-hours');
expect(shipContent).toContain('invoke /investigate');
expect(shipContent).toContain('invoke /ship');
expect(shipContent).toContain('invoke /qa');
});
test('routing section uses renamed checkpoint skills (not stale /checkpoint)', () => {
expect(shipContent).toContain('invoke /context-save');
expect(shipContent).toContain('invoke /context-restore');
expect(shipContent).not.toContain('invoke checkpoint');
});
test('routing section uses soft "when in doubt" policy, not hard "ALWAYS invoke"', () => {
expect(shipContent).toContain('When in doubt, invoke the skill');
expect(shipContent).not.toContain('Do NOT answer directly');
});
});
// --- {{DESIGN_OUTSIDE_VOICES}} resolver tests ---
describe('DESIGN_OUTSIDE_VOICES resolver', () => {
test('plan-design-review contains outside voices section', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Design Outside Voices');
expect(content).toContain('CODEX_AVAILABLE');
expect(content).toContain('LITMUS SCORECARD');
});
test('design-review contains outside voices section', () => {
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Design Outside Voices');
expect(content).toContain('source audit');
});
test('design-consultation contains outside voices section', () => {
const content = fs.readFileSync(path.join(ROOT, 'design-consultation', 'SKILL.md'), 'utf-8');
expect(content).toContain('Design Outside Voices');
expect(content).toContain('design direction');
});
test('branches correctly per skillName — different prompts', () => {
const planContent = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
const consultContent = fs.readFileSync(path.join(ROOT, 'design-consultation', 'SKILL.md'), 'utf-8');
// plan-design-review uses analytical prompt (high reasoning)
expect(planContent).toContain('model_reasoning_effort="high"');
// design-consultation uses creative prompt (medium reasoning)
expect(consultContent).toContain('model_reasoning_effort="medium"');
});
});
// --- {{DESIGN_HARD_RULES}} resolver tests ---
describe('DESIGN_HARD_RULES resolver', () => {
test('plan-design-review Pass 4 contains hard rules', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Design Hard Rules');
expect(content).toContain('Classifier');
expect(content).toContain('MARKETING/LANDING PAGE');
expect(content).toContain('APP UI');
});
test('design-review contains hard rules', () => {
const content = fs.readFileSync(path.join(ROOT, 'design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Design Hard Rules');
});
test('includes all 3 rule sets', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Landing page rules');
expect(content).toContain('App UI rules');
expect(content).toContain('Universal rules');
});
test('references shared AI slop blacklist items', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('3-column feature grid');
expect(content).toContain('Purple/violet/indigo');
});
test('includes OpenAI hard rejection criteria', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Generic SaaS card grid');
expect(content).toContain('Carousel with no narrative purpose');
});
test('includes OpenAI litmus checks', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-design-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Brand/product unmistakable');
expect(content).toContain('premium with all decorative shadows removed');
});
});
// --- Extended DESIGN_SKETCH resolver tests ---
describe('DESIGN_SKETCH extended with outside voices', () => {
const content = fs.readFileSync(path.join(ROOT, 'office-hours', 'SKILL.md'), 'utf-8');
test('contains outside design voices step', () => {
expect(content).toContain('Outside design voices');
});
test('offers opt-in via AskUserQuestion', () => {
expect(content).toContain('outside design perspectives');
});
test('still contains original wireframe steps', () => {
expect(content).toContain('wireframe');
expect(content).toContain('$B goto');
});
});
// --- Extended DESIGN_REVIEW_LITE resolver tests ---
describe('DESIGN_REVIEW_LITE extended with Codex', () => {
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
test('contains Codex design voice block', () => {
expect(content).toContain('Codex design voice');
expect(content).toContain('CODEX (design)');
});
test('still contains original checklist steps', () => {
expect(content).toContain('design-checklist.md');
expect(content).toContain('SCOPE_FRONTEND');
});
});
// ─── Codex Generation Tests ─────────────────────────────────
describe('Codex generation (--host codex)', () => {
const AGENTS_DIR = path.join(ROOT, '.agents', 'skills');
// .agents/ is gitignored (v0.11.2.0) — generate on demand for tests
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex'], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
});
// Dynamic discovery of expected Codex skills: all templates except /codex
// Also excludes skills where .agents/skills/{name} is a symlink back to the repo root
// (vendored dev mode — gen-skill-docs skips these to avoid overwriting Claude SKILL.md)
const CODEX_SKILLS = (() => {
const skills: Array<{ dir: string; codexName: string }> = [];
const isSymlinkLoop = (codexName: string): boolean => {
const agentSkillDir = path.join(ROOT, '.agents', 'skills', codexName);
try {
return fs.realpathSync(agentSkillDir) === fs.realpathSync(ROOT);
} catch { return false; }
};
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
if (!isSymlinkLoop('gstack')) {
skills.push({ dir: '.', codexName: 'gstack' });
}
}
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
if (entry.name === 'codex') continue; // /codex is excluded from Codex output
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
const codexName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
if (isSymlinkLoop(codexName)) continue;
skills.push({ dir: entry.name, codexName });
}
return skills;
})();
test('--host codex generates correct output paths', () => {
for (const skill of CODEX_SKILLS) {
const skillMd = path.join(AGENTS_DIR, skill.codexName, 'SKILL.md');
expect(fs.existsSync(skillMd)).toBe(true);
}
});
test('root gstack bundle has OpenAI metadata for Codex skill browsing', () => {
const rootMetadata = path.join(ROOT, 'agents', 'openai.yaml');
expect(fs.existsSync(rootMetadata)).toBe(true);
const content = fs.readFileSync(rootMetadata, 'utf-8');
expect(content).toContain('display_name: "gstack"');
expect(content).toContain('Use $gstack to locate the bundled gstack skills.');
expect(content).toContain('allow_implicit_invocation: true');
});
test('externalSkillName mapping: root is gstack, others are gstack-{dir}', () => {
// Root → gstack
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack', 'SKILL.md'))).toBe(true);
// Subdirectories → gstack-{dir}
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'))).toBe(true);
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'))).toBe(true);
// gstack-upgrade doesn't double-prefix
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-upgrade', 'SKILL.md'))).toBe(true);
// No double-prefix: gstack-gstack-upgrade must NOT exist
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-gstack-upgrade', 'SKILL.md'))).toBe(false);
});
test('Codex frontmatter has ONLY name + description', () => {
for (const skill of CODEX_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
expect(content.startsWith('---\n')).toBe(true);
const fmEnd = content.indexOf('\n---', 4);
expect(fmEnd).toBeGreaterThan(0);
const frontmatter = content.slice(4, fmEnd);
// Must have name and description
expect(frontmatter).toContain('name:');
expect(frontmatter).toContain('description:');
// Must NOT have allowed-tools, version, or hooks
expect(frontmatter).not.toContain('allowed-tools:');
expect(frontmatter).not.toContain('version:');
expect(frontmatter).not.toContain('hooks:');
}
});
test('all Codex skills have agents/openai.yaml metadata', () => {
for (const skill of CODEX_SKILLS) {
const metadata = path.join(AGENTS_DIR, skill.codexName, 'agents', 'openai.yaml');
expect(fs.existsSync(metadata)).toBe(true);
const content = fs.readFileSync(metadata, 'utf-8');
expect(content).toContain(`display_name: "${skill.codexName}"`);
expect(content).toContain('short_description:');
expect(content).toContain('allow_implicit_invocation: true');
}
});
test('no .claude/skills/ in Codex output', () => {
for (const skill of CODEX_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
expect(content).not.toContain('.claude/skills');
}
});
test('no ~/.claude/ paths in Codex output', () => {
for (const skill of CODEX_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
expect(content).not.toContain('~/.claude/');
}
});
test('/codex skill excluded from Codex output', () => {
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
expect(fs.existsSync(path.join(AGENTS_DIR, 'gstack-codex'))).toBe(false);
});
test('Codex output includes Claude outside-voice skill with read-only boundary', () => {
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-claude', 'SKILL.md'), 'utf-8');
expect(content).toContain('claude -p');
expect(content).toContain('mktemp /tmp/gstack-claude-prompt-');
expect(content).toContain('mktemp /tmp/gstack-claude-diff-');
expect(content).not.toContain('/tmp/gstack-claude-diff-$$');
expect(content).toContain('cat "$PROMPT_FILE" | claude -p');
expect(content).toContain('--disable-slash-commands');
expect(content).toContain('--tools ""');
expect(content).toContain('--allowedTools Read,Grep,Glob');
expect(content).toContain('--disallowedTools Bash,Edit,Write');
expect(content).toContain('is_error');
});
test('Codex review step stripped from Codex-host ship and review', () => {
const shipContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
expect(shipContent).not.toContain('codex review --base');
expect(shipContent).not.toContain('CODEX_REVIEWS');
const reviewContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
expect(reviewContent).not.toContain('codex review --base');
expect(reviewContent).not.toContain('CODEX_REVIEWS');
});
test('--host codex --dry-run freshness', () => {
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run'], {
cwd: ROOT,
stdout: 'pipe',
stderr: 'pipe',
});
expect(result.exitCode).toBe(0);
const output = result.stdout.toString();
// Every Codex skill should be FRESH
for (const skill of CODEX_SKILLS) {
expect(output).toContain(`FRESH: .agents/skills/${skill.codexName}/SKILL.md`);
}
expect(output).not.toContain('STALE');
});
test('--host agents alias produces same output as --host codex', () => {
const codexResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'codex', '--dry-run'], {
cwd: ROOT,
stdout: 'pipe',
stderr: 'pipe',
});
const agentsResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'agents', '--dry-run'], {
cwd: ROOT,
stdout: 'pipe',
stderr: 'pipe',
});
expect(codexResult.exitCode).toBe(0);
expect(agentsResult.exitCode).toBe(0);
// Both should produce the same output (same FRESH lines)
expect(codexResult.stdout.toString()).toBe(agentsResult.stdout.toString());
});
test('multiline descriptions preserved in Codex output', () => {
// office-hours has a multiline description — verify it survives the frontmatter transform
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-office-hours', 'SKILL.md'), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(4, fmEnd);
// Description should span multiple lines (block scalar)
const descLines = frontmatter.split('\n').filter(l => l.startsWith(' '));
expect(descLines.length).toBeGreaterThan(1);
// Verify key phrases survived
expect(frontmatter).toContain('YC Office Hours');
});
test('hook skills have safety prose and no hooks: in frontmatter', () => {
const HOOK_SKILLS = ['gstack-careful', 'gstack-freeze', 'gstack-guard'];
for (const skillName of HOOK_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skillName, 'SKILL.md'), 'utf-8');
// Must have safety advisory prose
expect(content).toContain('Safety Advisory');
// Must NOT have hooks: in frontmatter
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(4, fmEnd);
expect(frontmatter).not.toContain('hooks:');
}
});
test('all Codex SKILL.md files have auto-generated header', () => {
for (const skill of CODEX_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
expect(content).toContain('AUTO-GENERATED from SKILL.md.tmpl');
expect(content).toContain('Regenerate: bun run gen:skill-docs');
}
});
test('Codex preamble resolves runtime assets from repo-local or global gstack roots', () => {
// Check a skill that has a preamble (review is a good candidate)
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('GSTACK_ROOT');
expect(content).toContain('$_ROOT/.agents/skills/gstack');
expect(content).toContain('$GSTACK_BIN/gstack-config');
expect(content).toContain('$GSTACK_ROOT/gstack-upgrade/SKILL.md');
expect(content).not.toContain('~/.codex/skills/gstack/bin/gstack-config get telemetry');
});
// ─── Path rewriting regression tests ─────────────────────────
test('sidecar paths point to .agents/skills/gstack/review/ (not gstack-review/)', () => {
// Regression: gen-skill-docs rewrote .claude/skills/review → .agents/skills/gstack-review
// but setup puts sidecars under .agents/skills/gstack/review/. Must match setup layout.
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
// Correct: references to sidecar files use gstack/review/ path
expect(content).toContain('.agents/skills/gstack/review/checklist.md');
// design-checklist.md is now referenced via Review Army specialist (Claude only, stripped for Codex)
// Wrong: must NOT reference gstack-review/checklist.md (file doesn't exist there)
expect(content).not.toContain('.agents/skills/gstack-review/checklist.md');
});
test('sidecar paths in ship skill point to gstack/review/ for pre-landing review', () => {
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
// Ship references the review checklist in its pre-landing review step
if (content.includes('checklist.md')) {
expect(content).toContain('.agents/skills/gstack/review/');
expect(content).not.toContain('.agents/skills/gstack-review/checklist');
}
});
test('greptile-triage sidecar path is correct', () => {
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
if (content.includes('greptile-triage')) {
expect(content).toContain('.agents/skills/gstack/review/greptile-triage.md');
expect(content).not.toContain('.agents/skills/gstack-review/greptile-triage');
}
});
test('all four path rewrite rules produce correct output', () => {
// Test each of the 4 path rewrite rules individually
const content = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
// Rule 1: ~/.claude/skills/gstack → $GSTACK_ROOT
expect(content).not.toContain('~/.claude/skills/gstack');
expect(content).toContain('$GSTACK_ROOT');
// Rule 2: .claude/skills/gstack → .agents/skills/gstack
expect(content).not.toContain('.claude/skills/gstack');
// Rule 3: .claude/skills/review → .agents/skills/gstack/review
expect(content).not.toContain('.claude/skills/review');
// Rule 4: .claude/skills → .agents/skills (catch-all)
expect(content).not.toContain('.claude/skills');
});
test('path rewrite rules apply to all Codex skills with sidecar references', () => {
// Verify across ALL generated skills, not just review
for (const skill of CODEX_SKILLS) {
const content = fs.readFileSync(path.join(AGENTS_DIR, skill.codexName, 'SKILL.md'), 'utf-8');
// No skill should reference Claude paths
expect(content).not.toContain('~/.claude/skills');
expect(content).not.toContain('.claude/skills');
if (content.includes('gstack-config') || content.includes('gstack-update-check') || content.includes('gstack-telemetry-log')) {
expect(content).toContain('$GSTACK_ROOT');
}
// If a skill references checklist.md, it must use the correct sidecar path
if (content.includes('checklist.md') && !content.includes('design-checklist.md')) {
expect(content).not.toContain('gstack-review/checklist.md');
}
}
});
// ─── Claude output regression guard ─────────────────────────
test('Claude output unchanged: review skill still uses .claude/skills/ paths', () => {
// Codex changes must NOT affect Claude output
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('.claude/skills/review/checklist.md');
expect(content).toContain('~/.claude/skills/gstack');
// Must NOT contain Codex paths
expect(content).not.toContain('.agents/skills');
expect(content).not.toContain('~/.codex/');
});
test('Claude output unchanged: ship skill still uses .claude/skills/ paths', () => {
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
expect(content).toContain('~/.claude/skills/gstack');
expect(content).not.toContain('.agents/skills');
expect(content).not.toContain('~/.codex/');
});
test('Claude output unchanged: all Claude skills have zero Codex paths', () => {
for (const skill of CLAUDE_GENERATED_SKILLS) {
const content = fs.readFileSync(path.join(ROOT, skill.dir, 'SKILL.md'), 'utf-8');
// pair-agent legitimately documents how Codex agents store credentials.
// codex + autoplan document the Codex CLI auth file (~/.codex/auth.json)
// and log path (~/.codex/logs/) — those are user-facing Codex CLI paths,
// not the gstack Codex host install path.
if (skill.dir !== 'pair-agent' && skill.dir !== 'codex' && skill.dir !== 'autoplan') {
expect(content).not.toContain('~/.codex/');
}
// gstack-upgrade legitimately references .agents/skills for cross-platform detection
if (skill.dir !== 'gstack-upgrade') {
expect(content).not.toContain('.agents/skills');
}
}
});
// ─── Design outside voices: Codex host guard ─────────────────
test('codex host produces empty outside voices in design-review', () => {
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-design-review', 'SKILL.md'), 'utf-8');
expect(codexContent).not.toContain('Design Outside Voices');
});
test('codex host does not include Codex design block in ship', () => {
const codexContent = fs.readFileSync(path.join(AGENTS_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
expect(codexContent).not.toContain('Codex design voice');
});
});
// ─── Factory generation tests ────────────────────────────────
describe('Factory generation (--host factory)', () => {
const FACTORY_DIR = path.join(ROOT, '.factory', 'skills');
// Generate Factory output for tests
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory'], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
});
const FACTORY_SKILLS = (() => {
const skills: Array<{ dir: string; factoryName: string }> = [];
const isSymlinkLoop = (name: string): boolean => {
const factorySkillDir = path.join(ROOT, '.factory', 'skills', name);
try { return fs.realpathSync(factorySkillDir) === fs.realpathSync(ROOT); }
catch { return false; }
};
if (fs.existsSync(path.join(ROOT, 'SKILL.md.tmpl'))) {
if (!isSymlinkLoop('gstack')) skills.push({ dir: '.', factoryName: 'gstack' });
}
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
if (entry.name === 'codex') continue;
if (!fs.existsSync(path.join(ROOT, entry.name, 'SKILL.md.tmpl'))) continue;
const factoryName = entry.name.startsWith('gstack-') ? entry.name : `gstack-${entry.name}`;
if (isSymlinkLoop(factoryName)) continue;
skills.push({ dir: entry.name, factoryName });
}
return skills;
})();
test('--host factory generates correct output paths', () => {
for (const skill of FACTORY_SKILLS) {
const skillMd = path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md');
expect(fs.existsSync(skillMd)).toBe(true);
}
});
test('Factory frontmatter has name + description + user-invocable', () => {
for (const skill of FACTORY_SKILLS) {
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(4, fmEnd);
expect(frontmatter).toContain('name:');
expect(frontmatter).toContain('description:');
expect(frontmatter).toContain('user-invocable: true');
expect(frontmatter).not.toContain('allowed-tools:');
expect(frontmatter).not.toContain('preamble-tier:');
expect(frontmatter).not.toContain('sensitive:');
}
});
test('sensitive skills have disable-model-invocation', () => {
const SENSITIVE = ['gstack-ship', 'gstack-land-and-deploy', 'gstack-guard', 'gstack-careful', 'gstack-freeze', 'gstack-unfreeze'];
for (const name of SENSITIVE) {
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(4, fmEnd);
expect(frontmatter).toContain('disable-model-invocation: true');
}
});
test('non-sensitive skills lack disable-model-invocation', () => {
const NON_SENSITIVE = ['gstack-qa', 'gstack-review', 'gstack-investigate', 'gstack-browse'];
for (const name of NON_SENSITIVE) {
const content = fs.readFileSync(path.join(FACTORY_DIR, name, 'SKILL.md'), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(4, fmEnd);
expect(frontmatter).not.toContain('disable-model-invocation');
}
});
test('no .claude/skills/ in Factory output', () => {
for (const skill of FACTORY_SKILLS) {
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
expect(content).not.toContain('.claude/skills');
}
});
test('no ~/.claude/skills/ paths in Factory output', () => {
for (const skill of FACTORY_SKILLS) {
const content = fs.readFileSync(path.join(FACTORY_DIR, skill.factoryName, 'SKILL.md'), 'utf-8');
// ~/.claude/skills should be rewritten, but ~/.claude/plans is legitimate
// (plan directory lookup) and ~/.claude/ in codex prompts is intentional
expect(content).not.toContain('~/.claude/skills');
}
});
test('/codex skill excluded from Factory output', () => {
expect(fs.existsSync(path.join(FACTORY_DIR, 'gstack-codex', 'SKILL.md'))).toBe(false);
expect(fs.existsSync(path.join(FACTORY_DIR, 'gstack-codex'))).toBe(false);
});
test('Factory keeps Codex integration blocks', () => {
// Factory users CAN use Codex second opinions (codex exec is a standalone binary)
const shipContent = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-ship', 'SKILL.md'), 'utf-8');
expect(shipContent).toContain('codex');
});
test('no agents/openai.yaml in Factory output', () => {
for (const skill of FACTORY_SKILLS) {
const yamlPath = path.join(FACTORY_DIR, skill.factoryName, 'agents', 'openai.yaml');
expect(fs.existsSync(yamlPath)).toBe(false);
}
});
test('--host droid alias works', () => {
const factoryResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run'], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
});
const droidResult = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'droid', '--dry-run'], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
});
expect(factoryResult.exitCode).toBe(0);
expect(droidResult.exitCode).toBe(0);
expect(factoryResult.stdout.toString()).toBe(droidResult.stdout.toString());
});
test('--host factory --dry-run freshness', () => {
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'factory', '--dry-run'], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
});
expect(result.exitCode).toBe(0);
const output = result.stdout.toString();
for (const skill of FACTORY_SKILLS) {
expect(output).toContain(`FRESH: .factory/skills/${skill.factoryName}/SKILL.md`);
}
expect(output).not.toContain('STALE');
});
test('Factory preamble uses .factory paths', () => {
const content = fs.readFileSync(path.join(FACTORY_DIR, 'gstack-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('GSTACK_ROOT');
expect(content).toContain('$_ROOT/.factory/skills/gstack');
expect(content).toContain('$GSTACK_BIN/gstack-config');
});
});
// ─── Parameterized host smoke tests (config-driven) ─────────
import { ALL_HOST_CONFIGS, getExternalHosts } from '../hosts/index';
describe('Parameterized host smoke tests', () => {
for (const hostConfig of getExternalHosts()) {
describe(`${hostConfig.displayName} (--host ${hostConfig.name})`, () => {
const hostDir = path.join(ROOT, hostConfig.hostSubdir, 'skills');
test('generates output that exists on disk', () => {
// Generated dir should exist (created by earlier bun run gen:skill-docs --host all)
if (!fs.existsSync(hostDir)) {
// Generate if not already done
Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
});
}
expect(fs.existsSync(hostDir)).toBe(true);
const skills = fs.readdirSync(hostDir).filter(d =>
fs.existsSync(path.join(hostDir, d, 'SKILL.md'))
);
expect(skills.length).toBeGreaterThan(0);
});
test('no .claude/skills path leakage outside repo-root sidecar symlinks', () => {
if (!fs.existsSync(hostDir)) return; // skip if not generated
const skills = fs.readdirSync(hostDir);
for (const skill of skills) {
// Dev installs may mount the repo root at host/skills/gstack as a runtime
// sidecar. The generator skips that symlink loop, so leakage checks should too.
if (isRepoRootSymlink(path.join(hostDir, skill))) continue;
const skillMd = path.join(hostDir, skill, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
// Strip bash blocks (which have legitimate fallback paths)
const noBash = content.replace(/```bash\n[\s\S]*?```/g, '');
const leaks = noBash.split('\n').filter(l => l.includes('.claude/skills'));
if (leaks.length > 0) {
throw new Error(`${skill}: .claude/skills leakage:\n${leaks.slice(0, 3).join('\n')}`);
}
}
});
test('frontmatter has name and description', () => {
if (!fs.existsSync(hostDir)) return;
const skills = fs.readdirSync(hostDir);
for (const skill of skills) {
const skillMd = path.join(hostDir, skill, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
expect(content).toMatch(/^---\n/);
expect(content).toMatch(/^name:\s/m);
expect(content).toMatch(/^description:\s/m);
}
});
test('generates Claude outside-voice skill for external hosts', () => {
const skillMd = path.join(hostDir, 'gstack-claude', 'SKILL.md');
expect(fs.existsSync(skillMd)).toBe(true);
const content = fs.readFileSync(skillMd, 'utf-8');
expect(content).toContain('claude -p');
expect(content).toContain('--disable-slash-commands');
expect(content).toContain('--allowedTools Read,Grep,Glob');
expect(content).toContain('--disallowedTools Bash,Edit,Write');
});
test('--dry-run freshness check passes', () => {
const result = Bun.spawnSync(
['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', hostConfig.name, '--dry-run'],
{ cwd: ROOT, stdout: 'pipe', stderr: 'pipe' }
);
expect(result.exitCode).toBe(0);
const output = result.stdout.toString();
expect(output).not.toContain('STALE');
});
if (hostConfig.generation.skipSkills?.includes('codex')) {
test('/codex skill excluded', () => {
expect(fs.existsSync(path.join(hostDir, 'gstack-codex', 'SKILL.md'))).toBe(false);
});
}
});
}
});
// ─── --host all tests ────────────────────────────────────────
describe('--host all', () => {
test('--host all generates for all registered hosts', () => {
const result = Bun.spawnSync(['bun', 'run', 'scripts/gen-skill-docs.ts', '--host', 'all', '--dry-run'], {
cwd: ROOT, stdout: 'pipe', stderr: 'pipe',
});
expect(result.exitCode).toBe(0);
const output = result.stdout.toString();
// All hosts should appear in output
expect(output).toContain('FRESH: SKILL.md'); // claude
for (const hostConfig of getExternalHosts()) {
expect(output).toContain(`FRESH: ${hostConfig.hostSubdir}/skills/`);
}
});
});
// ─── Setup script validation ─────────────────────────────────
// These tests verify the setup script's install layout matches
// what the generator produces — catching the bug where setup
// installed Claude-format source dirs for Codex users.
describe('setup script validation', () => {
const setupContent = fs.readFileSync(path.join(ROOT, 'setup'), 'utf-8');
test('setup has separate link functions for Claude and Codex', () => {
expect(setupContent).toContain('link_claude_skill_dirs');
expect(setupContent).toContain('link_codex_skill_dirs');
// Old unified function must not exist
expect(setupContent).not.toMatch(/^link_skill_dirs\(\)/m);
});
test('Claude install uses link_claude_skill_dirs', () => {
// The Claude install section (section 4) should use the Claude function
const claudeSection = setupContent.slice(
setupContent.indexOf('# 4. Install for Claude'),
setupContent.indexOf('# 5. Install for Codex')
);
expect(claudeSection).toContain('link_claude_skill_dirs');
expect(claudeSection).not.toContain('link_codex_skill_dirs');
});
test('Codex install uses link_codex_skill_dirs', () => {
// The Codex install section (section 5) should use the Codex function
const codexSection = setupContent.slice(
setupContent.indexOf('# 5. Install for Codex'),
setupContent.indexOf('# 6. Create')
);
expect(codexSection).toContain('create_codex_runtime_root');
expect(codexSection).toContain('link_codex_skill_dirs');
expect(codexSection).not.toContain('link_claude_skill_dirs');
expect(codexSection).not.toContain('_link_or_copy "$GSTACK_DIR" "$CODEX_GSTACK"');
});
test('Codex install prefers repo-local .agents/skills when setup runs from there', () => {
expect(setupContent).toContain('SKILLS_PARENT_BASENAME');
expect(setupContent).toContain('CODEX_REPO_LOCAL=0');
expect(setupContent).toContain('[ "$SKILLS_PARENT_BASENAME" = ".agents" ]');
expect(setupContent).toContain('CODEX_REPO_LOCAL=1');
expect(setupContent).toContain('CODEX_SKILLS="$INSTALL_SKILLS_DIR"');
});
test('setup separates install path from source path for symlinked repo-local installs', () => {
expect(setupContent).toContain('INSTALL_GSTACK_DIR=');
expect(setupContent).toContain('SOURCE_GSTACK_DIR=');
expect(setupContent).toContain('INSTALL_SKILLS_DIR=');
expect(setupContent).toContain('CODEX_GSTACK="$INSTALL_GSTACK_DIR"');
expect(setupContent).toContain('link_codex_skill_dirs "$SOURCE_GSTACK_DIR" "$CODEX_SKILLS"');
});
test('Codex installs always create sidecar runtime assets for the real skill target', () => {
expect(setupContent).toContain('if [ "$INSTALL_CODEX" -eq 1 ]; then');
expect(setupContent).toContain('create_agents_sidecar "$SOURCE_GSTACK_DIR"');
});
test('link_codex_skill_dirs reads from .agents/skills/', () => {
// The Codex link function must reference .agents/skills for generated Codex skills
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('.agents/skills');
expect(fnBody).toContain('gstack*');
});
test('link_claude_skill_dirs creates real directories with absolute SKILL.md symlinks', () => {
// Claude links should be real directories with absolute SKILL.md symlinks
// to ensure Claude Code discovers them as top-level skills (not nested under gstack/)
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('mkdir -p "$target"');
// v1.36.0.0: routes through _link_or_copy helper for Windows fallback (cp on MSYS2/Git Bash).
expect(fnBody).toContain('_link_or_copy "$gstack_dir/$dir_name/SKILL.md" "$target/SKILL.md"');
});
// REGRESSION: cleanup functions must handle both old symlinks AND new real-directory pattern
test('cleanup functions handle real directories with symlinked SKILL.md', () => {
// cleanup_old_claude_symlinks must detect and remove real dirs with SKILL.md symlinks
const cleanupOldStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
const cleanupOldEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up old', cleanupOldStart));
const cleanupOldBody = setupContent.slice(cleanupOldStart, cleanupOldEnd);
expect(cleanupOldBody).toContain('-d "$old_target"');
expect(cleanupOldBody).toContain('-L "$old_target/SKILL.md"');
expect(cleanupOldBody).toContain('rm -rf "$old_target"');
// cleanup_prefixed_claude_symlinks must also handle the new pattern
const cleanupPrefixedStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
const cleanupPrefixedEnd = setupContent.indexOf('}', setupContent.indexOf('cleaned up prefixed', cleanupPrefixedStart));
const cleanupPrefixedBody = setupContent.slice(cleanupPrefixedStart, cleanupPrefixedEnd);
expect(cleanupPrefixedBody).toContain('-d "$prefixed_target"');
expect(cleanupPrefixedBody).toContain('-L "$prefixed_target/SKILL.md"');
expect(cleanupPrefixedBody).toContain('rm -rf "$prefixed_target"');
});
// REGRESSION: link function must upgrade old directory symlinks
test('link_claude_skill_dirs removes old directory symlinks before creating real dirs', () => {
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
// Must check for and remove old symlinks before mkdir
expect(fnBody).toContain('if [ -L "$target" ]');
expect(fnBody).toContain('rm -f "$target"');
});
test('setup supports --host auto|claude|codex|kiro|opencode', () => {
expect(setupContent).toContain('--host');
expect(setupContent).toContain('claude|codex|kiro|factory|opencode|auto');
});
test('auto mode detects claude, codex, kiro, and opencode binaries', () => {
expect(setupContent).toContain('command -v claude');
expect(setupContent).toContain('command -v codex');
expect(setupContent).toContain('command -v kiro-cli');
expect(setupContent).toContain('command -v opencode');
});
// T1: Sidecar skip guard — prevents .agents/skills/gstack from being linked as a skill
test('link_codex_skill_dirs skips the gstack sidecar directory', () => {
const fnStart = setupContent.indexOf('link_codex_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('[ "$skill_name" = "gstack" ] && continue');
});
// T2: Dynamic $GSTACK_ROOT paths in generated Codex preambles
test('generated Codex preambles use dynamic GSTACK_ROOT paths', () => {
const codexSkillDir = path.join(ROOT, '.agents', 'skills', 'gstack-ship');
if (!fs.existsSync(codexSkillDir)) return; // skip if .agents/ not generated
const content = fs.readFileSync(path.join(codexSkillDir, 'SKILL.md'), 'utf-8');
expect(content).toContain('GSTACK_ROOT=');
expect(content).toContain('$GSTACK_BIN/');
});
test('setup supports --host kiro with install section and sed rewrites', () => {
expect(setupContent).toContain('INSTALL_KIRO=');
expect(setupContent).toContain('kiro-cli');
expect(setupContent).toContain('KIRO_SKILLS=');
expect(setupContent).toContain('~/.kiro/skills/gstack');
});
test('setup supports --host opencode with install section and OpenCode skill path vars', () => {
expect(setupContent).toContain('INSTALL_OPENCODE=');
expect(setupContent).toContain('OPENCODE_SKILLS="$HOME/.config/opencode/skills"');
expect(setupContent).toContain('OPENCODE_GSTACK="$OPENCODE_SKILLS/gstack"');
});
test('setup installs OpenCode skills into a nested gstack runtime root', () => {
expect(setupContent).toContain('create_opencode_runtime_root');
expect(setupContent).toContain('.opencode/skills');
expect(setupContent).toContain('review/specialists');
expect(setupContent).toContain('qa/templates');
expect(setupContent).toContain('qa/references');
expect(setupContent).toContain('dx-hall-of-fame.md');
});
test('create_agents_sidecar links runtime assets', () => {
// Sidecar must link bin, browse, review, qa
const fnStart = setupContent.indexOf('create_agents_sidecar()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('bin');
expect(fnBody).toContain('browse');
expect(fnBody).toContain('review');
expect(fnBody).toContain('qa');
});
test('create_codex_runtime_root exposes only runtime assets', () => {
const fnStart = setupContent.indexOf('create_codex_runtime_root()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('done', setupContent.indexOf('review/', fnStart)));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('gstack/SKILL.md');
expect(fnBody).toContain('browse/dist');
expect(fnBody).toContain('browse/bin');
expect(fnBody).toContain('gstack-upgrade/SKILL.md');
// Review runtime assets (individual files, not the whole dir)
expect(fnBody).toContain('checklist.md');
expect(fnBody).toContain('design-checklist.md');
expect(fnBody).toContain('greptile-triage.md');
expect(fnBody).toContain('TODOS-format.md');
expect(fnBody).not.toContain('_link_or_copy "$gstack_dir" "$codex_gstack"');
});
test('direct Codex installs are migrated out of ~/.codex/skills/gstack', () => {
expect(setupContent).toContain('migrate_direct_codex_install');
expect(setupContent).toContain('$HOME/.gstack/repos/gstack');
expect(setupContent).toContain('avoid duplicate skill discovery');
});
// --- Symlink prefix tests (PR #503) ---
test('link_claude_skill_dirs applies gstack- prefix by default', () => {
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('SKILL_PREFIX');
expect(fnBody).toContain('link_name="gstack-$skill_name"');
});
test('link_claude_skill_dirs preserves already-prefixed dirs', () => {
const fnStart = setupContent.indexOf('link_claude_skill_dirs()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('linked[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
// gstack-* dirs should keep their name (e.g., gstack-upgrade stays gstack-upgrade)
expect(fnBody).toContain('gstack-*) link_name="$skill_name"');
});
test('setup supports --no-prefix flag', () => {
expect(setupContent).toContain('--no-prefix');
expect(setupContent).toContain('SKILL_PREFIX=0');
});
test('cleanup_old_claude_symlinks removes only gstack-pointing symlinks', () => {
expect(setupContent).toContain('cleanup_old_claude_symlinks');
const fnStart = setupContent.indexOf('cleanup_old_claude_symlinks()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
// Should check readlink before removing
expect(fnBody).toContain('readlink');
expect(fnBody).toContain('gstack/*');
// Should skip already-prefixed dirs
expect(fnBody).toContain('gstack-*) continue');
});
test('cleanup runs before link when prefix is enabled', () => {
// In the Claude install section, cleanup should happen before linking
const claudeInstallSection = setupContent.slice(
setupContent.indexOf('INSTALL_CLAUDE'),
setupContent.lastIndexOf('link_claude_skill_dirs')
);
expect(claudeInstallSection).toContain('cleanup_old_claude_symlinks');
});
// --- Persistent config + interactive prompt tests ---
test('setup reads skill_prefix from config', () => {
expect(setupContent).toContain('get skill_prefix');
expect(setupContent).toContain('GSTACK_CONFIG');
});
test('setup supports --prefix flag', () => {
expect(setupContent).toContain('--prefix)');
expect(setupContent).toContain('SKILL_PREFIX=1; SKILL_PREFIX_FLAG=1');
});
test('--prefix and --no-prefix persist to config', () => {
expect(setupContent).toContain('set skill_prefix');
});
test('interactive prompt shows when no config', () => {
expect(setupContent).toContain('Short names');
expect(setupContent).toContain('Namespaced');
expect(setupContent).toContain('Choice [1/2]');
});
test('non-TTY defaults to flat names', () => {
// Should check if stdin is a TTY before prompting
expect(setupContent).toContain('-t 0');
});
test('cleanup_prefixed_claude_symlinks exists and uses readlink', () => {
expect(setupContent).toContain('cleanup_prefixed_claude_symlinks');
const fnStart = setupContent.indexOf('cleanup_prefixed_claude_symlinks()');
const fnEnd = setupContent.indexOf('}', setupContent.indexOf('removed[@]}', fnStart));
const fnBody = setupContent.slice(fnStart, fnEnd);
expect(fnBody).toContain('readlink');
expect(fnBody).toContain('gstack-$skill_name');
});
test('reverse cleanup runs before link when prefix is disabled', () => {
const claudeInstallSection = setupContent.slice(
setupContent.indexOf('INSTALL_CLAUDE'),
setupContent.lastIndexOf('link_claude_skill_dirs')
);
expect(claudeInstallSection).toContain('cleanup_prefixed_claude_symlinks');
});
test('welcome message references SKILL_PREFIX', () => {
// gstack-upgrade is always called gstack-upgrade (it's the actual dir name)
// but the welcome section should exist near the prefix logic
expect(setupContent).toContain('Run /gstack-upgrade anytime');
});
});
describe('discover-skills hidden directory filtering', () => {
test('discoverTemplates skips dot-prefixed directories', () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-discover-'));
try {
// Create a hidden dir with a template (should be excluded)
fs.mkdirSync(path.join(tmpDir, '.hidden'), { recursive: true });
fs.writeFileSync(path.join(tmpDir, '.hidden', 'SKILL.md.tmpl'), '---\nname: evil\n---\ntest');
// Create a visible dir with a template (should be included)
fs.mkdirSync(path.join(tmpDir, 'visible'), { recursive: true });
fs.writeFileSync(path.join(tmpDir, 'visible', 'SKILL.md.tmpl'), '---\nname: good\n---\ntest');
const { discoverTemplates } = require('../scripts/discover-skills');
const results = discoverTemplates(tmpDir);
const dirs = results.map((r: { tmpl: string }) => r.tmpl);
expect(dirs).toContain('visible/SKILL.md.tmpl');
expect(dirs).not.toContain('.hidden/SKILL.md.tmpl');
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
});
describe('telemetry', () => {
test('generated SKILL.md contains telemetry start block', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('_TEL_START');
expect(content).toContain('_SESSION_ID');
expect(content).toContain('TELEMETRY:');
expect(content).toContain('TEL_PROMPTED:');
expect(content).toContain('gstack-config get telemetry');
});
test('generated SKILL.md contains telemetry opt-in prompt', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('.telemetry-prompted');
expect(content).toContain('Help gstack get better');
expect(content).toContain('gstack-config set telemetry community');
expect(content).toContain('gstack-config set telemetry anonymous');
expect(content).toContain('gstack-config set telemetry off');
});
test('generated SKILL.md contains telemetry epilogue', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('Telemetry (run last)');
expect(content).toContain('gstack-telemetry-log');
expect(content).toContain('_TEL_END');
expect(content).toContain('_TEL_DUR');
expect(content).toContain('SKILL_NAME');
expect(content).toContain('OUTCOME');
expect(content).toContain('PLAN MODE EXCEPTION');
});
test('generated SKILL.md contains pending marker handling', () => {
const content = fs.readFileSync(path.join(ROOT, 'SKILL.md'), 'utf-8');
expect(content).toContain('.pending');
expect(content).toContain('_pending_finalize');
});
test('telemetry blocks appear in all skill files that use PREAMBLE', () => {
const skills = ['qa', 'ship', 'review', 'plan-ceo-review', 'plan-eng-review', 'retro'];
for (const skill of skills) {
const skillPath = path.join(ROOT, skill, 'SKILL.md');
if (fs.existsSync(skillPath)) {
const content = fs.readFileSync(skillPath, 'utf-8');
expect(content).toContain('_TEL_START');
expect(content).toContain('Telemetry (run last)');
}
}
});
});
describe('community fixes wave', () => {
// Helper to get all generated SKILL.md files
function getAllSkillMds(): Array<{ name: string; content: string }> {
const results: Array<{ name: string; content: string }> = [];
const rootPath = path.join(ROOT, 'SKILL.md');
if (fs.existsSync(rootPath)) {
results.push({ name: 'root', content: fs.readFileSync(rootPath, 'utf-8') });
}
for (const entry of fs.readdirSync(ROOT, { withFileTypes: true })) {
if (!entry.isDirectory() || entry.name.startsWith('.') || entry.name === 'node_modules') continue;
const skillPath = path.join(ROOT, entry.name, 'SKILL.md');
if (fs.existsSync(skillPath)) {
results.push({ name: entry.name, content: fs.readFileSync(skillPath, 'utf-8') });
}
}
return results;
}
// #594 — Discoverability: every SKILL.md.tmpl description contains "gstack"
test('every SKILL.md.tmpl description contains "gstack"', () => {
for (const skill of ALL_SKILLS) {
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
const content = fs.readFileSync(tmplPath, 'utf-8');
const desc = extractDescription(content);
expect(desc.toLowerCase()).toContain('gstack');
}
});
// #594 — Discoverability: first line of each description is under 120 chars
test('every SKILL.md.tmpl description first line is under 120 chars', () => {
for (const skill of ALL_SKILLS) {
const tmplPath = skill.dir === '.' ? path.join(ROOT, 'SKILL.md.tmpl') : path.join(ROOT, skill.dir, 'SKILL.md.tmpl');
const content = fs.readFileSync(tmplPath, 'utf-8');
const desc = extractDescription(content);
const firstLine = desc.split('\n')[0];
expect(firstLine.length).toBeLessThanOrEqual(120);
}
});
// #573 — Feature signals: ship/SKILL.md contains feature signal detection
test('ship/SKILL.md contains feature signal detection in Step 4', () => {
const content = fs.readFileSync(path.join(ROOT, 'ship', 'SKILL.md'), 'utf-8');
expect(content.toLowerCase()).toContain('feature signal');
});
// #510 — Context warnings: no SKILL.md contains "running low on context"
test('no generated SKILL.md contains "running low on context"', () => {
const skills = getAllSkillMds();
for (const { name, content } of skills) {
expect(content).not.toContain('running low on context');
}
});
// #510 — Context warnings: plan-eng-review has explicit anti-warning
test('plan-eng-review/SKILL.md contains "Do not preemptively warn"', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-eng-review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Do not preemptively warn');
});
// #474 — Safety Net: no SKILL.md uses find with -delete
test('no generated SKILL.md contains find with -delete flag', () => {
const skills = getAllSkillMds();
for (const { name, content } of skills) {
// Match find commands that use -delete (but not prose mentioning the word "delete")
const lines = content.split('\n');
for (const line of lines) {
if (line.includes('find ') && line.includes('-delete')) {
throw new Error(`${name}/SKILL.md contains find with -delete: ${line.trim()}`);
}
}
}
});
// #467 — Telemetry: preamble JSONL writes are gated by telemetry setting
test('preamble JSONL writes are inside telemetry conditional', () => {
const preamble = fs.readFileSync(path.join(ROOT, 'scripts/resolvers/preamble.ts'), 'utf-8');
// Find all skill-usage.jsonl write lines
const lines = preamble.split('\n');
for (let i = 0; i < lines.length; i++) {
if (lines[i].includes('skill-usage.jsonl') && lines[i].includes('>>')) {
// Look backwards for a telemetry conditional within 5 lines
let foundConditional = false;
for (let j = i - 1; j >= Math.max(0, i - 5); j--) {
if (lines[j].includes('_TEL') && lines[j].includes('off')) {
foundConditional = true;
break;
}
}
expect(foundConditional).toBe(true);
}
}
});
});
describe('codex commands must not use inline $(git rev-parse --show-toplevel) for cwd', () => {
// Regression test: inline $(git rev-parse --show-toplevel) in codex exec -C
// or codex review without cd evaluates in whatever cwd the background shell
// inherits, which may be a different project in Conductor workspaces.
// The fix is to resolve _REPO_ROOT eagerly at the top of each bash block.
// Scan all source files that could contain codex commands
// Use Bun.Glob to avoid ELOOP from .claude/skills/gstack symlink back to ROOT
const tmplGlob = new Bun.Glob('**/*.tmpl');
const sourceFiles = [
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
.filter(f => f.endsWith('.ts'))
.map(f => `scripts/resolvers/${f}`),
'scripts/gen-skill-docs.ts',
];
test('no codex exec command uses inline $(git rev-parse --show-toplevel) in -C flag', () => {
const violations: string[] = [];
for (const rel of sourceFiles) {
const abs = path.join(ROOT, rel);
if (!fs.existsSync(abs)) continue;
const content = fs.readFileSync(abs, 'utf-8');
const lines = content.split('\n');
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
violations.push(`${rel}:${i + 1}`);
}
}
}
expect(violations).toEqual([]);
});
test('no generated SKILL.md has codex exec with inline $(git rev-parse --show-toplevel) in -C flag', () => {
const violations: string[] = [];
const skillMdGlob = new Bun.Glob('**/SKILL.md');
const skillMdFiles = Array.from(skillMdGlob.scanSync({ cwd: ROOT, followSymlinks: false }));
for (const rel of skillMdFiles) {
const abs = path.join(ROOT, rel);
if (!fs.existsSync(abs)) continue;
const content = fs.readFileSync(abs, 'utf-8');
const lines = content.split('\n');
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
if (line.includes('codex exec') && line.includes('-C') && line.includes('$(git rev-parse --show-toplevel)')) {
violations.push(`${rel}:${i + 1}`);
}
}
}
expect(violations).toEqual([]);
});
test('codex review commands must be preceded by cd "$_REPO_ROOT" (no -C support)', () => {
// codex review does not support -C, so the pattern must be:
// _REPO_ROOT=$(git rev-parse --show-toplevel) || { ... }
// cd "$_REPO_ROOT"
// codex review ...
// NOT: codex review ... with inline $(git rev-parse --show-toplevel)
const allFiles = [
...Array.from(tmplGlob.scanSync({ cwd: ROOT, followSymlinks: false })),
...Array.from(new Bun.Glob('**/SKILL.md').scanSync({ cwd: ROOT, followSymlinks: false })),
...fs.readdirSync(path.join(ROOT, 'scripts/resolvers'))
.filter(f => f.endsWith('.ts'))
.map(f => `scripts/resolvers/${f}`),
'scripts/gen-skill-docs.ts',
];
const violations: string[] = [];
for (const rel of allFiles) {
const abs = path.join(ROOT, rel);
if (!fs.existsSync(abs)) continue;
const content = fs.readFileSync(abs, 'utf-8');
const lines = content.split('\n');
for (let i = 0; i < lines.length; i++) {
const line = lines[i];
// Skip non-executable lines (markdown table cells, prose references)
if (line.includes('|') && line.includes('`/codex review`')) continue;
if (line.includes('`codex review`')) continue;
// Check for codex review with inline $(git rev-parse)
if (line.includes('codex review') && line.includes('$(git rev-parse --show-toplevel)')) {
violations.push(`${rel}:${i + 1} — inline git rev-parse in codex review`);
}
}
}
expect(violations).toEqual([]);
});
test('codex review commands pass diff scope through prompt, not --base', () => {
const checkedFiles = [
'codex/SKILL.md.tmpl',
'codex/SKILL.md',
'scripts/resolvers/review.ts',
'review/SKILL.md',
'ship/SKILL.md',
];
for (const rel of checkedFiles) {
const content = fs.readFileSync(path.join(ROOT, rel), 'utf-8');
expect(content).not.toContain('--base <base> -c \'model_reasoning_effort="high"\'');
expect(content).toContain('Run git diff origin/<base>...HEAD 2>/dev/null || git diff <base>...HEAD');
}
});
});
// ─── Learnings + Confidence Resolver Tests ─────────────────────
describe('LEARNINGS_SEARCH resolver', () => {
const SEARCH_SKILLS = ['review', 'ship', 'plan-eng-review', 'investigate', 'office-hours', 'plan-ceo-review'];
for (const skill of SEARCH_SKILLS) {
test(`${skill} generated SKILL.md contains learnings search`, () => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).toContain('Prior Learnings');
expect(content).toContain('gstack-learnings-search');
});
}
test('learnings search includes cross-project config check', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('cross_project_learnings');
expect(content).toContain('--cross-project');
});
test('learnings search includes AskUserQuestion for first-time cross-project opt-in', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Enable cross-project learnings');
expect(content).toContain('project-scoped only');
});
test('learnings search mentions prior learning applied display format', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Prior learning applied');
});
});
describe('LEARNINGS_LOG resolver', () => {
const LOG_SKILLS = ['review', 'retro', 'investigate'];
for (const skill of LOG_SKILLS) {
test(`${skill} generated SKILL.md contains learnings log`, () => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).toContain('Capture Learnings');
expect(content).toContain('gstack-learnings-log');
});
}
test('learnings log documents all type values', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
for (const type of ['pattern', 'pitfall', 'preference', 'architecture', 'tool']) {
expect(content).toContain(type);
}
});
test('learnings log documents all source values', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
for (const source of ['observed', 'user-stated', 'inferred', 'cross-model']) {
expect(content).toContain(source);
}
});
test('learnings log includes files field for staleness detection', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('"files"');
expect(content).toContain('staleness detection');
});
});
describe('CONFIDENCE_CALIBRATION resolver', () => {
const CONFIDENCE_SKILLS = ['review', 'ship', 'plan-eng-review', 'cso'];
for (const skill of CONFIDENCE_SKILLS) {
test(`${skill} generated SKILL.md contains confidence calibration`, () => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).toContain('Confidence Calibration');
expect(content).toContain('confidence score');
});
}
test('confidence calibration includes scoring rubric with all tiers', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('9-10');
expect(content).toContain('7-8');
expect(content).toContain('5-6');
expect(content).toContain('3-4');
expect(content).toContain('1-2');
});
test('confidence calibration includes display rules', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('Show normally');
expect(content).toContain('Suppress from main report');
});
test('confidence calibration includes finding format example', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('[P1] (confidence:');
expect(content).toContain('SQL injection');
});
test('confidence calibration includes calibration learning feedback loop', () => {
const content = fs.readFileSync(path.join(ROOT, 'review', 'SKILL.md'), 'utf-8');
expect(content).toContain('calibration event');
expect(content).toContain('Log the corrected pattern');
});
test('skills without confidence calibration do NOT contain it', () => {
// office-hours and retro do NOT use confidence calibration
for (const skill of ['office-hours', 'retro']) {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
expect(content).not.toContain('## Confidence Calibration');
}
});
});
describe('gen-skill-docs prefix warning (#620/#578)', () => {
const { execSync } = require('child_process');
test('warns about skill_prefix when config has prefix=true', () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
try {
// Create a fake ~/.gstack/config.yaml with skill_prefix: true
const fakeHome = tmpDir;
const fakeGstack = path.join(fakeHome, '.gstack');
fs.mkdirSync(fakeGstack, { recursive: true });
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: true\n');
const output = execSync('bun run scripts/gen-skill-docs.ts', {
cwd: ROOT,
env: { ...process.env, HOME: fakeHome },
encoding: 'utf-8',
timeout: 30000,
});
expect(output).toContain('skill_prefix is true');
expect(output).toContain('gstack-relink');
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
test('no warning when skill_prefix is false or absent', () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gstack-prefix-warn-'));
try {
const fakeHome = tmpDir;
const fakeGstack = path.join(fakeHome, '.gstack');
fs.mkdirSync(fakeGstack, { recursive: true });
fs.writeFileSync(path.join(fakeGstack, 'config.yaml'), 'skill_prefix: false\n');
const output = execSync('bun run scripts/gen-skill-docs.ts', {
cwd: ROOT,
env: { ...process.env, HOME: fakeHome },
encoding: 'utf-8',
timeout: 30000,
});
expect(output).not.toContain('skill_prefix is true');
} finally {
fs.rmSync(tmpDir, { recursive: true, force: true });
}
});
});
describe('voice-triggers processing', () => {
const { extractVoiceTriggers, processVoiceTriggers } = require('../scripts/gen-skill-docs') as {
extractVoiceTriggers: (content: string) => string[];
processVoiceTriggers: (content: string) => string;
};
test('extractVoiceTriggers parses valid YAML list', () => {
const content = `---\nname: cso\ndescription: |\n Security audit.\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
const triggers = extractVoiceTriggers(content);
expect(triggers).toEqual(['see-so', 'security review']);
});
test('extractVoiceTriggers returns [] when no field present', () => {
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
expect(extractVoiceTriggers(content)).toEqual([]);
});
test('processVoiceTriggers appends voice triggers to description', () => {
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n - "security review"\n---\nBody`;
const result = processVoiceTriggers(content);
expect(result).toContain('Voice triggers (speech-to-text aliases): "see-so", "security review".');
});
test('processVoiceTriggers strips voice-triggers field from output', () => {
const content = `---\nname: cso\ndescription: |\n Security audit. (gstack)\nvoice-triggers:\n - "see-so"\n---\nBody`;
const result = processVoiceTriggers(content);
expect(result).not.toContain('voice-triggers:');
});
test('processVoiceTriggers returns content unchanged when no voice-triggers', () => {
const content = `---\nname: qa\ndescription: |\n QA testing.\n---\nBody`;
expect(processVoiceTriggers(content)).toBe(content);
});
test('generated CSO SKILL.md contains voice triggers in description', () => {
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
expect(content).toContain('"see-so"');
expect(content).toContain('Voice triggers (speech-to-text aliases):');
});
test('generated CSO SKILL.md does NOT contain raw voice-triggers field', () => {
const content = fs.readFileSync(path.join(ROOT, 'cso', 'SKILL.md'), 'utf-8');
const fmEnd = content.indexOf('\n---', 4);
const frontmatter = content.slice(0, fmEnd);
expect(frontmatter).not.toContain('voice-triggers:');
});
});
describe('plan-mode-info resolver (handshake-replacement)', () => {
const REVIEW_SKILLS = [
'plan-ceo-review',
'plan-eng-review',
'plan-design-review',
'plan-devex-review',
];
// Header for the vestigial handshake that was removed. If it ever reappears,
// someone accidentally re-introduced the resolver.
const HANDSHAKE_MARKER = '## Plan Mode Handshake';
// Header for the new plan-mode-info section (previously lived at the tail
// of completion-status.ts; now hoisted to position 1 of the preamble).
const PLAN_MODE_INFO_MARKER = '## Skill Invocation During Plan Mode';
test('vestigial handshake is absent from all generated Claude SKILL.md files', () => {
// Scan every generated SKILL.md under ROOT (top-level directory per skill).
// Using fs.readdirSync + filter instead of a glob so we catch any skill
// that gets added later without updating this list.
const entries = fs.readdirSync(ROOT, { withFileTypes: true });
let checked = 0;
for (const entry of entries) {
if (!entry.isDirectory()) continue;
const skillMd = path.join(ROOT, entry.name, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
expect(content, `handshake marker in ${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
checked++;
}
expect(checked).toBeGreaterThan(0);
});
test('vestigial handshake is absent from non-Claude host outputs when present on disk', () => {
// Non-Claude hosts render to hostSubdirs (.agents/, .openclaw/, etc). The
// plan-mode-info resolver has no host-scoping — all hosts get the new
// section, none get the old handshake. Scan all candidate host dirs.
const hostDirs = ['.agents', '.openclaw', '.opencode', '.factory', '.hermes', '.kiro', '.cursor', '.slate'];
let checked = 0;
for (const host of hostDirs) {
const skillsRoot = path.join(ROOT, host, 'skills');
if (!fs.existsSync(skillsRoot)) continue;
const entries = fs.readdirSync(skillsRoot, { withFileTypes: true });
for (const entry of entries) {
if (!entry.isDirectory()) continue;
const skillMd = path.join(skillsRoot, entry.name, 'SKILL.md');
if (!fs.existsSync(skillMd)) continue;
const content = fs.readFileSync(skillMd, 'utf-8');
expect(content, `handshake marker in ${host}/skills/${entry.name}/SKILL.md`).not.toContain(HANDSHAKE_MARKER);
checked++;
}
}
if (checked === 0) {
// eslint-disable-next-line no-console
console.warn(
'plan-mode-info: no non-Claude host outputs found for cross-host absence check — ' +
'run `bun run gen:skill-docs --host all` to populate',
);
}
});
test.each(REVIEW_SKILLS)(
'%s/SKILL.md contains the new plan-mode-info section near the top',
(skill) => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
const idx = content.indexOf(PLAN_MODE_INFO_MARKER);
expect(idx).toBeGreaterThan(0);
// Position 1 in preamble composition = within the first ~300 lines.
// Roughly translates to first ~15KB of text.
expect(idx).toBeLessThan(15_000);
},
);
test('plan-mode-info is wired BEFORE generateUpgradeCheck in preamble', () => {
const content = fs.readFileSync(
path.join(ROOT, 'plan-ceo-review', 'SKILL.md'),
'utf-8',
);
const planModeIdx = content.indexOf(PLAN_MODE_INFO_MARKER);
const upgradeIdx = content.indexOf('UPGRADE_AVAILABLE');
expect(planModeIdx).toBeGreaterThan(0);
expect(upgradeIdx).toBeGreaterThan(0);
expect(planModeIdx).toBeLessThan(upgradeIdx);
});
test('0C-bis STOP block present in plan-ceo-review/SKILL.md', () => {
const content = fs.readFileSync(path.join(ROOT, 'plan-ceo-review', 'SKILL.md'), 'utf-8');
const presentIdx = content.indexOf('Present these approach options via AskUserQuestion');
const preludeIdx = content.indexOf('### 0D-prelude');
expect(presentIdx).toBeGreaterThan(0);
expect(preludeIdx).toBeGreaterThan(presentIdx);
const between = content.slice(presentIdx, preludeIdx);
expect(between).toContain('**STOP.**');
expect(between).toContain('Do NOT proceed to Step 0D or 0F until the user responds to 0C-bis');
});
});
// GSTACK REVIEW REPORT report-at-bottom contract — verifies the prompt-text
// fix in scripts/resolvers/review.ts (the load-bearing change for the
// "report not at bottom of plan in plan mode" bug). The bug is in the
// prompt's contradictory write-flow instructions, not in observable
// runtime behavior we can cheaply gate in CI. Verifying the prompt text
// directly is the deterministic equivalent of the regression test the
// PTY harness can't reliably drive (autoplan needs auto-progression of
// AskUserQuestions to reach the report-write step, which the harness
// doesn't support today).
describe('GSTACK REVIEW REPORT delete-then-append flow', () => {
const PLAN_REVIEW_SKILLS = [
'plan-ceo-review',
'plan-design-review',
'plan-devex-review',
'plan-eng-review',
];
for (const skill of PLAN_REVIEW_SKILLS) {
test(`${skill}/SKILL.md prescribes delete-then-append, not in-place replace`, () => {
const content = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
// The new (correct) instruction must be present.
expect(content).toContain('delete-then-append flow');
expect(content).toContain('never mid-file');
expect(content).toContain('Do NOT replace the section in place');
// The old contradictory bullets must be gone. The signature phrase
// from the buggy prompt was 'replace it entirely using the Edit tool'
// which is what allowed mid-file reports to stay mid-file.
expect(content).not.toContain('replace it** entirely using the Edit tool');
expect(content).not.toContain('If it was found mid-file, move it');
});
}
test('scripts/resolvers/review.ts source has the rewritten flow', () => {
const src = fs.readFileSync(path.join(ROOT, 'scripts', 'resolvers', 'review.ts'), 'utf-8');
expect(src).toContain('delete-then-append flow');
expect(src).toContain('never mid-file');
expect(src).toContain('Do NOT replace the section in place');
// Old contradictory bullets are gone from the source resolver.
expect(src).not.toContain('replace it** entirely using the Edit tool');
expect(src).not.toContain('If it was found mid-file, move it');
});
});
describe('LEARNINGS_SEARCH resolver: query parameter', () => {
// Lazy-load resolver and types after describe block to keep test file self-contained.
const { generateLearningsSearch } = require('../scripts/resolvers/learnings');
const { HOST_PATHS } = require('../scripts/resolvers/types');
const claudeCtx = {
skillName: 'test',
tmplPath: 'test/SKILL.md.tmpl',
host: 'claude',
paths: HOST_PATHS.claude,
};
const codexCtx = { ...claudeCtx, host: 'codex', paths: HOST_PATHS.codex };
test('no args → bash does not contain --query (backwards-compat)', () => {
const out = generateLearningsSearch(claudeCtx);
expect(out).not.toContain('--query');
});
test('claude host + query=foo bar → both cross-project and project-scoped branches contain --query', () => {
const out = generateLearningsSearch(claudeCtx, ['query=foo bar']);
// Both branches of the if/else must carry the flag.
const lines = out.split('\n').filter(l => l.includes('gstack-learnings-search'));
expect(lines.length).toBeGreaterThanOrEqual(2);
for (const line of lines) {
expect(line).toContain('--query "foo bar"');
}
});
test('codex host + query=foo bar → codex bash variant contains --query', () => {
const out = generateLearningsSearch(codexCtx, ['query=foo bar']);
expect(out).toContain('--query "foo bar"');
expect(out).toContain('$GSTACK_BIN/gstack-learnings-search');
});
test('empty value query= → bash does not contain --query (locked semantics: falls through)', () => {
const claudeOut = generateLearningsSearch(claudeCtx, ['query=']);
expect(claudeOut).not.toContain('--query');
const codexOut = generateLearningsSearch(codexCtx, ['query=']);
expect(codexOut).not.toContain('--query');
});
test('shell-injection chars in query= → throws at gen-time (defense in depth)', () => {
for (const bad of ['$(whoami)', '`cmd`', 'a;b', 'a&b', 'a"b', 'a\\b', 'foo$x']) {
expect(() => generateLearningsSearch(claudeCtx, [`query=${bad}`])).toThrow(/alphanumeric/);
}
});
});
describe('EXIT PLAN MODE GATE placement', () => {
// Fresh skill list — do NOT reuse REVIEW_SKILLS upstream (3 entries, missing plan-devex).
const planSkills = ['plan-eng-review', 'plan-ceo-review', 'plan-design-review', 'plan-devex-review'];
// Strip fenced code blocks before matching headings — PLAN_FILE_REVIEW_REPORT
// already contains `## GSTACK REVIEW REPORT` inside a markdown example fence,
// and the gate text itself shows `## GSTACK REVIEW REPORT` inside a fence too.
const stripFences = (md: string) => md.replace(/```[\s\S]*?```/g, '');
test('gate is the terminal ## heading in every plan-* review SKILL.md', () => {
for (const skill of planSkills) {
const md = fs.readFileSync(path.join(ROOT, skill, 'SKILL.md'), 'utf-8');
const stripped = stripFences(md);
const headings = [...stripped.matchAll(/^## .+$/gm)].map(m => m[0]);
const lastH2 = headings.at(-1);
expect(lastH2, `${skill}/SKILL.md last ## heading (fences stripped)`).toBe('## EXIT PLAN MODE GATE (BLOCKING)');
expect(md, `${skill}/SKILL.md gate body`).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation');
}
});
test('codex/SKILL.md contains gate (mid-file per D5; Step 2B/2C follow)', () => {
const codex = fs.readFileSync(path.join(ROOT, 'codex', 'SKILL.md'), 'utf-8');
expect(codex).toContain('## EXIT PLAN MODE GATE (BLOCKING)');
expect(codex).toContain('Failing this gate and calling ExitPlanMode anyway is a contract violation');
});
});