2 Commits

Author	SHA1	Message	Date
Garry Tan	3bf43766d5	v1.38.0.0 fix wave: Windows install hardening + Unicode sanitization at server egress (4 community PRs) (#1505) ... * fix(browse): single-point Unicode sanitization at server egress Add sanitizeLoneSurrogates (regex-based UTF-16 lone-half cleaner) and sanitizeReplacer (JSON.stringify replacer that runs the cleaner on every string field during encoding). Split handleCommandInternal into handleCommandInternalImpl (raw) plus a thin sanitizing wrapper. The wrapper applies sanitizeLoneSurrogates to cr.result so both single-command (handleCommand line 1034) and batch-loop (line 1966) egress paths inherit it. Inline INVARIANT comment near the wrapper documents the architectural constraint. Both SSE producers (activity feed at /activity/stream and inspector stream) stringify with sanitizeReplacer. Post-stringify regex is ineffective on those paths because JSON.stringify has already converted the lone surrogate into the escape sequence "\\\\uD800" before any regex could match it; the replacer runs during stringify on the raw string value, so the substitution lands. Originated from @realcarsonterry PR #1463 (handleCommand-only wrap). Architectural lift to handleCommandInternal + SSE coverage authored on this branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(setup): _link_or_copy helper for Windows file-copy fallback On Windows without Developer Mode (MSYS2/Git Bash), plain ln -snf silently creates a frozen file copy that doesn't refresh on git pull. Skill files become stale after every upgrade. Add a _link_or_copy SRC DST helper near IS_WINDOWS detection (line ~33). It auto-dispatches: on Unix it preserves ln -snf semantics, on Windows it copies (cp -R for directories, cp -f for files). When the source is a Unix-style name-only alias that doesn't resolve on disk (the connect-chrome → gstack/open-gstack-browser pattern), the helper returns 0 silently on Windows rather than aborting setup under set -e. Rewrite all 42 prior ln -snf call sites to route through the helper: link_claude_skill_dirs (line 437), team-claude install paths (lines 556, 581, 592), Codex host adapter block (lines 618-640), Factory host adapter block (lines 658-678), OpenCode host adapter block (lines 696-731), Kiro host adapter block (lines 939-953), plus migration and alias sites. Add _print_windows_copy_note_once helper and call it from link_claude_skill_dirs after any linking work completes so Windows users see one user-visible note explaining they must re-run ./setup after every git pull. Extend cleanup_old_claude_symlinks and cleanup_prefixed_claude_symlinks with a Windows branch: when the target is a real directory containing a real-file SKILL.md (no symlink to readlink), and IS_WINDOWS=1, treat the name-matched directory as gstack-managed and remove it. This makes --prefix / --no-prefix flips work on Windows instead of leaving stale copies behind. Originated from @realcarsonterry PR #1462 (1 of 42 sites). Helper extraction, 42-site rewrite, alias-resolution edge case, and Windows cleanup compat authored on this branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(docs): rename stale gbrain_sync_mode to artifacts_sync_mode + register /document-generate Five stale gstack-config references in docs/ pointed to the deprecated gbrain_sync_mode key (renamed to artifacts_sync_mode in v1.27.0.0): - docs/gbrain-sync.md: lines 62, 110, 111, 173 - docs/gbrain-sync-errors.md: lines 26, 203 Users following the docs would set a key that gstack-brain-sync no longer reads, silently breaking artifacts sync. Originated from @realcarsonterry PR #1461 (verbatim). Also register /document-generate in AGENTS.md (Operational + memory table) and docs/skills.md (skill index). The skill shipped in v1.35.0.0 but the doc-inventory cross-check in test/skill-validation.test.ts was failing because neither file mentioned it. Allowlist the new test/docs-config-keys.test.ts file in test/no-stale-gstack-brain-refs.test.ts — it intentionally lists the deprecated keys in its DEPRECATED_KEYS denylist (defending the rename). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci(windows): migrate windows-free-tests to paid faster runner + register wave tests Move the Windows free-test job from GitHub-hosted windows-latest to Blacksmith's paid Windows runner (blacksmith-2vcpu-windows-2022). Spin-up drops from ~60s to ~10s and Bun installs land 3-4x faster. The label can swap to namespace-profile-windows or ubicloud-windows-* if this repo's Blacksmith installation isn't configured. Register the four new wave tests in the workflow's curated test list: - browse/test/server-sanitize-surrogates.test.ts - test/setup-windows-fallback.test.ts - test/build-script-shell-compat.test.ts - test/docs-config-keys.test.ts These tests cover the Windows-hardening surface that this wave ships (sanitizer wiring, _link_or_copy helper, build-script subshells, doc- config drift), so they need to run on Windows where the bug shapes actually manifest. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test: wave coverage for sanitizer, link_or_copy, build script, doc drift Four new test files (29 cases total): browse/test/server-sanitize-surrogates.test.ts: - 11 unit cases for sanitizeLoneSurrogates (passthrough, valid pair, lone high/low mid-string, trailing/leading lone, adjacent doubles, pair-then-lone, lone-then-pair, empty) - 2 bug-repro tests pinning the regression intent (UTF-8 round-trip, JSON.parse round-trip with codepoint assertion) - 4 wiring invariants asserting the architectural choke points stay intact (handleCommandInternalImpl rename, central sanitization line, sanitizeReplacer function exists, SSE producers stringify with replacer) Function extracted from server.ts via regex + eval'd in test scope so no production-code export is needed. test/setup-windows-fallback.test.ts: - Static invariant (D7): zero raw `ln` calls outside the _link_or_copy helper body and comments - Helper-existence assertions - 4-cell behavior matrix (file/dir × Windows/Unix) via awk-style helper extraction + bash -c sourcing - Windows-note printer registration check Mirrors test/setup-conductor-worktree.test.ts patterns. test/build-script-shell-compat.test.ts: - Regex assertion that package.json scripts.* contain no bash brace groups (Bun-Windows-hostile) - Subshell-precedence check for `.version` redirects Strips single-quoted strings before regexing so embedded JS code inside echo '...' doesn't false-positive. test/docs-config-keys.test.ts: - DEPRECATED_KEYS denylist scanned across docs/**/*.md - Round-trip test for `gstack-config get artifacts_sync_mode` Defends the v1.27.0.0 rename from doc drift. Updates to two existing tests: - test/setup-conductor-worktree.test.ts: expect `_link_or_copy` instead of `ln -snf` at the Conductor-worktree guard call site - test/gen-skill-docs.test.ts: same swap at three assertion sites (Codex section, Claude link_claude_skill_dirs body, Codex link_codex_skill_dirs body) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore: bump v1.38.0.0 + build-script subshells + CHANGELOG VERSION 1.35.0.0 → 1.38.0.0 (MINOR). PR #1500 (lyon-v2) claimed v1.37.0.0 ahead of this branch; v1.38.0.0 is the next free MINOR slot per bin/gstack-next-version queue check. Workspace-aware ship rule applies — queue-advancing past a claimed version within the same bump level is explicitly permitted. package.json build script: three `{ git rev-parse HEAD ...; }` brace groups → `( git rev-parse HEAD ... )` subshells. Bun's Windows shell parser doesn't grok bash brace groups; subshells are POSIX-universal. Originated from @realcarsonterry PR #1460. CHANGELOG entry covers the full wave: - Windows install hardening (42-site _link_or_copy + cleanup compat) - Unicode sanitization architecture (handleCommandInternal + SSE replacer) - Build script POSIX-shell compat (subshells) - Doc rename (gbrain_sync_mode → artifacts_sync_mode) - Windows CI on paid faster runner - 4 new wave tests (29 cases) Frames each item as a current system property, not a fix narrative. Credits @realcarsonterry for PRs #1460, #1461, #1462, #1463 (the seed of the wave). Scope expansion to all 42 setup sites, every server egress path, Windows CI migration, and codex-flagged P0/P1 fixes (connect-chrome alias on Windows, SSE replacer, prefix-cleanup Windows compat) authored on this branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: post-ship sync for v1.38.0.0 Document the two architectural invariants that landed in v1.38.0.0 in their persistent homes (not just CHANGELOG): - README Windows section: add the `./setup` re-run-after-git-pull requirement that `_print_windows_copy_note_once` shows at runtime. - CONTRIBUTING "Things to know": add the no-raw-`ln` invariant for contributors editing `setup`, with the test that enforces it. - ARCHITECTURE: new "Unicode sanitization at server egress" section between Shell injection prevention and Prompt injection defense, with egress table (HTTP/batch/SSE) and the post-stringify-regex rationale. - CLAUDE.md: cross-references for both invariants, matching the v1.6.0.0 dual-listener pattern (each constraint says which files to read before editing and which test pins it). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci(windows): use windows-latest-8-cores instead of unregistered Blacksmith label actionlint failed PR #1505 because `blacksmith-2vcpu-windows-2022` isn't in the repo's approved runner-label list (actionlint.yaml only registers `ubicloud-standard-2`, and Ubicloud doesn't ship a Windows pool). Switch to GitHub's paid larger Windows runner `windows-latest-8-cores` — 4x the cores of the free `windows-latest` at the larger-runner billing rate, no new third-party CI provider, no actionlint config changes. CHANGELOG: replace "Blacksmith" / "blacksmith-2vcpu-windows-2022" / "~6x faster spin-up" claims with the actual choice (8 cores vs 4, paid larger runner). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci(windows): switch from windows-latest-8-cores to ubicloud-standard-2-windows `windows-latest-8-cores` sat queued indefinitely because the GitHub larger-runner billing isn't enabled at the org level — the "Queued — Waiting to run this check" status surfaced on PR #1505 with no progress for the whole CI run. Switch to Ubicloud Windows runners (`ubicloud-standard-2-windows`) so Windows CI uses the same provider as the existing Linux evals (`ubicloud-standard-2`). Billing stays under one account instead of two. Register the new label in actionlint.yaml alongside the existing ubicloud-standard-2 entry so actionlint doesn't reject it as unknown. CHANGELOG entry updated: runner row reflects the actual provider chosen, "Itemized changes" mentions the actionlint.yaml registration, and the narrative paragraph documents why `windows-latest-8-cores` failed first. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci: migrate all workflows to Ubicloud (Linux + Windows, 8-core) Switch every `runs-on` in this repo to Ubicloud so CI has a single billing surface, consistent capacity, and 4x more cores on the workloads that were previously stuck on free `ubuntu-latest` (2 cores). Windows uses Ubicloud's Windows pool too — `ubicloud-standard-8-windows` — so the queued-forever problem with GitHub's `windows-latest-8-cores` paid larger runner (org-level larger-runner billing not enabled) goes away. Workflows touched (9): - evals.yml, evals-periodic.yml, ci-image.yml — bump default + matrix from `ubicloud-standard-2` to `ubicloud-standard-8`. The one matrix entry that was already on -8 stays. - windows-free-tests.yml — `ubicloud-standard-2-windows` → `ubicloud-standard-8-windows`. - make-pdf-gate.yml — matrix `ubuntu-latest` → `ubicloud-standard-8`. macOS entry preserved; the poppler-install `if: matrix.os` conditional swaps to match the new label. - actionlint.yml, pr-title-sync.yml, skill-docs.yml, version-gate.yml — `ubuntu-latest` → `ubicloud-standard-8`. .github/actionlint.yaml registers all four Ubicloud labels in one place: - ubicloud-standard-2 - ubicloud-standard-8 - ubicloud-standard-2-windows (the v1.38.0.0 windows-free-tests target) - ubicloud-standard-8-windows (this PR's windows-free-tests target) Removed the duplicate `actionlint.yaml` at the repo root that I accidentally created in the prior commit — actionlint only reads `.github/actionlint.yaml`, so the root file was dead weight. CHANGELOG entry updated: a single "all Ubicloud" sentence in the narrative plus a metrics-row covering the runner pool change, and the itemized line expanded to enumerate the 9 affected workflows. The previously-orphaned "Itemized changes" line about just `windows-free-tests.yml` is replaced. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci(windows): revert to free `windows-latest` Ubicloud doesn't ship Windows runners — confirmed via their docs. The `ubicloud-standard-*-windows` labels I added do not exist and were causing `windows-free-tests` to sit "Queued — Waiting to run this check" forever (GitHub Actions can't tell a typoed label from a self-hosted runner that's about to register; it just waits). Three prior Windows-runner attempts all failed for different reasons: - `blacksmith-2vcpu-windows-2022` — Blacksmith app not installed on the org - `windows-latest-8-cores` — GitHub paid larger-runner billing not enabled - `ubicloud-standard-2/8-windows` — Ubicloud doesn't offer Windows at all The free `windows-latest` runner (4 cores, ~60s spin-up, $0) is the one path that actually runs. The wave-coverage Windows tests are <30s of real work; total job time stays under 2 minutes. Cleaned up `.github/actionlint.yaml` to drop the bogus `ubicloud-standard-*-windows` entries — kept only the two real Linux labels. CHANGELOG: split the runner-pool row into Linux (migrated to Ubicloud-8) vs Windows (stays on free windows-latest), with the why on each. Itemized line for windows-free-tests rewritten to reflect the actual outcome. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(windows): skip Unix-only cases on Windows runner windows-free-tests on GitHub free windows-latest fails three cases that depend on Unix tooling the runner doesn't have: 1. `setup-windows-fallback.test.ts` behavior matrix — IS_WINDOWS=0 cells assert `ln -snf` produces a real symlink. On Windows-without-Developer- Mode (which the free `windows-latest` runner is), `ln -snf` silently creates a file copy. That's literally the bug `_link_or_copy` exists to work around, so the assertion can never pass there. Skip the whole describe block on win32. The static-invariant test (zero raw `ln` outside the helper body) above the matrix still runs and pins the shape the Windows install relies on. 2. `docs-config-keys.test.ts` round-trip — spawnSync(`bin/gstack-config`) on Windows doesn't read the bash shebang and fails to exec. Skip on win32; the deprecated-key denylist test in the same file still runs and is the actual invariant defending the v1.27.0.0 rename at the doc layer. Use `describe.skipIf(process.platform === 'win32', ...)` and `test.skipIf(process.platform === 'win32', ...)`. Tests still run on macOS and Linux unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-05-14 21:19:58 -07:00
Garry Tan	d0782c4c4d	feat(v1.4.0.0): /make-pdf — markdown to publication-quality PDFs (#1086) ... * feat(browse): full $B pdf flag contract + tab-scoped load-html/js/pdf Grow $B pdf from a 2-line wrapper (hard-coded A4) into a real PDF engine frontend so make-pdf can shell out to it without duplicating Playwright: - pdf: --format, --width/--height, --margins, --margin-*, --header-template, --footer-template, --page-numbers, --tagged, --outline, --print-background, --prefer-css-page-size, --toc. Mutex rules enforced. --from-file <json> dodges Windows argv limits (8191 char CreateProcess cap). - load-html: add --from-file <json> mode for large inline HTML. Size + magic byte checks still apply to the inline content, not the payload file path. - newtab: add --json returning {"tabId":N,"url":...} for programmatic use. - cli: extract --tab-id flag and route as body.tabId to the HTTP layer so parallel callers can target specific tabs without racing on the active tab (makes make-pdf's per-render tab isolation possible). - --toc: non-fatal 3s wait for window.__pagedjsAfterFired. Paged.js ships later; v1 renders TOC statically via the markdown renderer. Codex round 2 flagged these P0 issues during plan review. All resolved. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(resolvers): add MAKE_PDF_SETUP + makePdfDir host paths Skill templates can now embed {{MAKE_PDF_SETUP}} to resolve $P to the make-pdf binary via the same discovery order as $B / $D: env override (MAKE_PDF_BIN), local skill root, global install, or PATH. Mirrors the pattern established by generateBrowseSetup() and generateDesignSetup() in scripts/resolvers/design.ts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(make-pdf): new /make-pdf skill + orchestrator binary Turn markdown into publication-quality PDFs. $P generate input.md out.pdf produces a PDF with 1in margins, intelligent page breaks, page numbers, running header, CONFIDENTIAL footer, and curly quotes/em dashes — all on Helvetica so copy-paste extraction works ("S ai li ng" bug avoided). Architecture (per Codex round 2): markdown → render.ts (marked + sanitize + smartypants) → orchestrator → $B newtab --json → $B load-html --tab-id → $B js (poll Paged.js) → $B pdf --tab-id → $B closetab browseClient.ts shells out to the compiled browse CLI rather than duplicating Playwright. --tab-id isolation per render means parallel $P generate calls don't race on the active tab. try/finally tab cleanup survives Paged.js timeouts, browser crashes, and output-path failures. Features in v1: --cover left-aligned cover page (eyebrow + title + hairline rule) --toc clickable static TOC (Paged.js page numbers deferred) --watermark <text> diagonal DRAFT/CONFIDENTIAL layer --no-chapter-breaks opt out of H1-starts-new-page --page-numbers "N of M" footer (default on) --tagged --outline accessible PDF + bookmark outline (default on) --allow-network opt in to external image loading (default off for privacy) --quiet --verbose stderr control Design decisions locked from the /plan-design-review pass: - Helvetica everywhere (Chromium emits single-word Tj operators for system fonts; bundled webfonts emit per-glyph and break extraction). - Left-aligned body, flush-left paragraphs, no text-indent, 12pt gap. - Cover shares 1in margins with body pages; no flexbox-center, no inset padding. - The reference HTMLs at .context/designs/*.html are the implementation source of truth for print-css.ts. Tests (56 unit + 1 E2E combined-features gate): - smartypants: code/URL-safe, verified against 10 fixtures - sanitizer: strips <script>/<iframe>/on*/javascript: URLs - render: HTML assembly, CJK fallback, cover/TOC/chapter wrap - print-css: all @page rules, margin variants, watermark - pdftotext: normalize()+copyPasteGate() cross-OS tolerance - browseClient: binary resolution + typed error propagation - combined-features gate (P0): 2-chapter fixture with smartypants + hyphens + ligatures + bold/italic + inline code + lists + blockquote passes through PDF → pdftotext → expected.txt diff Deferred to Phase 4 (future PR): Paged.js vendored for accurate TOC page numbers, highlight.js for syntax highlighting, drop caps, pull quotes, two-column, CMYK, watermark visual-diff acceptance. Plan: .context/ceo-plans/2026-04-19-perfect-pdf-generator.md References: .context/designs/make-pdf-*.html Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(build): wire make-pdf into build/test/setup/bin + add marked dep - package.json: compile make-pdf/dist/pdf as part of bun run build; add "make-pdf" to bin entry; include make-pdf/test/ in the free test pass; add marked@18.0.2 as a dep (markdown parser, ~40KB). - setup: add make-pdf/dist/pdf to the Apple Silicon codesign loop. - .gitignore: add make-pdf/dist/ (matches browse/dist/ and design/dist/). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(make-pdf): matrix copy-paste gate on Ubuntu + macOS Runs the combined-features P0 gate on pull requests that touch make-pdf/ or browse's PDF surface. Installs poppler (macOS) / poppler-utils (Ubuntu) per OS. Windows deferred to tolerant mode (Xpdf / Poppler-Windows extraction variance not yet calibrated against the normalized comparator — Codex round 2 #18). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(skills): regenerate SKILL.md for make-pdf addition + browse pdf flags bun run gen:skill-docs picks up: - the new /make-pdf skill (make-pdf/SKILL.md) - updated browse command descriptions for 'pdf', 'load-html', 'newtab' reflecting the new flag contract and --from-file mode Source of truth stays the .tmpl files + COMMAND_DESCRIPTIONS; these are regenerated artifacts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(tests): repair stale test expectations + emit _EXPLAIN_LEVEL / _QUESTION_TUNING from preamble Three pre-existing test failures on main were blocking /ship: - test/skill-validation.test.ts "Step 3.4 test coverage audit" expected the literal strings "CODE PATH COVERAGE" and "USER FLOW COVERAGE" which were removed when the Step 7 coverage diagram was compressed. Updated assertions to check the stable `Code paths:` / `User flows:` labels that still ship. - test/skill-validation.test.ts "ship step numbering" allowed-substeps list didn't include 15.0 (WIP squash) and 15.1 (bisectable commits) which were added for continuous checkpoint mode. Extended the allowlist. - test/writing-style-resolver.test.ts and test/plan-tune.test.ts expected `_EXPLAIN_LEVEL` and `_QUESTION_TUNING` bash variables in the preamble but generate-preamble-bash.ts had been refactored and those lines were dropped. Without them, downstream skills can't read `explain_level` or `question_tuning` config at runtime — terse mode and /plan-tune features were silently broken. Added the two bash echo blocks back to generatePreambleBash and refreshed the golden-file fixtures to match. All three preamble-related golden baselines (claude/codex/factory) are synchronized with the new output. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v1.4.0.0) New /make-pdf skill + $P binary. Turn any markdown file into a publication-quality PDF. Default output is a 1in-margin Helvetica letter with page numbers in the footer. `--cover` adds a left-aligned cover page, `--toc` generates a clickable table of contents, `--watermark DRAFT` overlays a diagonal watermark. Copy-paste extraction from the PDF produces clean words, not "S a i l i n g" spaced out letter by letter. CI gate (macOS + Ubuntu) runs a combined- features fixture through pdftotext on every PR. make-pdf shells out to browse rather than duplicating Playwright. $B pdf grew into a real PDF engine with full flag contract (--format, --margins, --header-template, --footer-template, --page-numbers, --tagged, --outline, --toc, --tab-id, --from-file). $B load-html and $B js gained --tab-id. $B newtab --json returns structured output. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(changelog): rewrite v1.4.0.0 headline — positive voice, no VC framing The original headline led with "a PDF you wouldn't be embarrassed to send to a VC": double-negative voice and audience-too-narrow. /make-pdf works for essays, letters, memos, reports, proposals, and briefs. Framing the whole release around founders-to-investors misses the wider audience. New headline: "Turn any markdown file into a PDF that looks finished." New tagline: "This one reads like a real essay or a real letter." Positive voice. Broader aperture. Same energy. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-20 13:20:30 +08:00