mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-16 01:02:13 +08:00
* fix(browse): single-point Unicode sanitization at server egress Add sanitizeLoneSurrogates (regex-based UTF-16 lone-half cleaner) and sanitizeReplacer (JSON.stringify replacer that runs the cleaner on every string field during encoding). Split handleCommandInternal into handleCommandInternalImpl (raw) plus a thin sanitizing wrapper. The wrapper applies sanitizeLoneSurrogates to cr.result so both single-command (handleCommand line 1034) and batch-loop (line 1966) egress paths inherit it. Inline INVARIANT comment near the wrapper documents the architectural constraint. Both SSE producers (activity feed at /activity/stream and inspector stream) stringify with sanitizeReplacer. Post-stringify regex is ineffective on those paths because JSON.stringify has already converted the lone surrogate into the escape sequence "\\\\uD800" before any regex could match it; the replacer runs during stringify on the raw string value, so the substitution lands. Originated from @realcarsonterry PR #1463 (handleCommand-only wrap). Architectural lift to handleCommandInternal + SSE coverage authored on this branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(setup): _link_or_copy helper for Windows file-copy fallback On Windows without Developer Mode (MSYS2/Git Bash), plain ln -snf silently creates a frozen file copy that doesn't refresh on git pull. Skill files become stale after every upgrade. Add a _link_or_copy SRC DST helper near IS_WINDOWS detection (line ~33). It auto-dispatches: on Unix it preserves ln -snf semantics, on Windows it copies (cp -R for directories, cp -f for files). When the source is a Unix-style name-only alias that doesn't resolve on disk (the connect-chrome → gstack/open-gstack-browser pattern), the helper returns 0 silently on Windows rather than aborting setup under set -e. Rewrite all 42 prior ln -snf call sites to route through the helper: link_claude_skill_dirs (line 437), team-claude install paths (lines 556, 581, 592), Codex host adapter block (lines 618-640), Factory host adapter block (lines 658-678), OpenCode host adapter block (lines 696-731), Kiro host adapter block (lines 939-953), plus migration and alias sites. Add _print_windows_copy_note_once helper and call it from link_claude_skill_dirs after any linking work completes so Windows users see one user-visible note explaining they must re-run ./setup after every git pull. Extend cleanup_old_claude_symlinks and cleanup_prefixed_claude_symlinks with a Windows branch: when the target is a real directory containing a real-file SKILL.md (no symlink to readlink), and IS_WINDOWS=1, treat the name-matched directory as gstack-managed and remove it. This makes --prefix / --no-prefix flips work on Windows instead of leaving stale copies behind. Originated from @realcarsonterry PR #1462 (1 of 42 sites). Helper extraction, 42-site rewrite, alias-resolution edge case, and Windows cleanup compat authored on this branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(docs): rename stale gbrain_sync_mode to artifacts_sync_mode + register /document-generate Five stale gstack-config references in docs/ pointed to the deprecated gbrain_sync_mode key (renamed to artifacts_sync_mode in v1.27.0.0): - docs/gbrain-sync.md: lines 62, 110, 111, 173 - docs/gbrain-sync-errors.md: lines 26, 203 Users following the docs would set a key that gstack-brain-sync no longer reads, silently breaking artifacts sync. Originated from @realcarsonterry PR #1461 (verbatim). Also register /document-generate in AGENTS.md (Operational + memory table) and docs/skills.md (skill index). The skill shipped in v1.35.0.0 but the doc-inventory cross-check in test/skill-validation.test.ts was failing because neither file mentioned it. Allowlist the new test/docs-config-keys.test.ts file in test/no-stale-gstack-brain-refs.test.ts — it intentionally lists the deprecated keys in its DEPRECATED_KEYS denylist (defending the rename). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci(windows): migrate windows-free-tests to paid faster runner + register wave tests Move the Windows free-test job from GitHub-hosted windows-latest to Blacksmith's paid Windows runner (blacksmith-2vcpu-windows-2022). Spin-up drops from ~60s to ~10s and Bun installs land 3-4x faster. The label can swap to namespace-profile-windows or ubicloud-windows-* if this repo's Blacksmith installation isn't configured. Register the four new wave tests in the workflow's curated test list: - browse/test/server-sanitize-surrogates.test.ts - test/setup-windows-fallback.test.ts - test/build-script-shell-compat.test.ts - test/docs-config-keys.test.ts These tests cover the Windows-hardening surface that this wave ships (sanitizer wiring, _link_or_copy helper, build-script subshells, doc- config drift), so they need to run on Windows where the bug shapes actually manifest. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test: wave coverage for sanitizer, link_or_copy, build script, doc drift Four new test files (29 cases total): browse/test/server-sanitize-surrogates.test.ts: - 11 unit cases for sanitizeLoneSurrogates (passthrough, valid pair, lone high/low mid-string, trailing/leading lone, adjacent doubles, pair-then-lone, lone-then-pair, empty) - 2 bug-repro tests pinning the regression intent (UTF-8 round-trip, JSON.parse round-trip with codepoint assertion) - 4 wiring invariants asserting the architectural choke points stay intact (handleCommandInternalImpl rename, central sanitization line, sanitizeReplacer function exists, SSE producers stringify with replacer) Function extracted from server.ts via regex + eval'd in test scope so no production-code export is needed. test/setup-windows-fallback.test.ts: - Static invariant (D7): zero raw `ln` calls outside the _link_or_copy helper body and comments - Helper-existence assertions - 4-cell behavior matrix (file/dir × Windows/Unix) via awk-style helper extraction + bash -c sourcing - Windows-note printer registration check Mirrors test/setup-conductor-worktree.test.ts patterns. test/build-script-shell-compat.test.ts: - Regex assertion that package.json scripts.* contain no bash brace groups (Bun-Windows-hostile) - Subshell-precedence check for `.version` redirects Strips single-quoted strings before regexing so embedded JS code inside echo '...' doesn't false-positive. test/docs-config-keys.test.ts: - DEPRECATED_KEYS denylist scanned across docs/**/*.md - Round-trip test for `gstack-config get artifacts_sync_mode` Defends the v1.27.0.0 rename from doc drift. Updates to two existing tests: - test/setup-conductor-worktree.test.ts: expect `_link_or_copy` instead of `ln -snf` at the Conductor-worktree guard call site - test/gen-skill-docs.test.ts: same swap at three assertion sites (Codex section, Claude link_claude_skill_dirs body, Codex link_codex_skill_dirs body) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore: bump v1.38.0.0 + build-script subshells + CHANGELOG VERSION 1.35.0.0 → 1.38.0.0 (MINOR). PR #1500 (lyon-v2) claimed v1.37.0.0 ahead of this branch; v1.38.0.0 is the next free MINOR slot per bin/gstack-next-version queue check. Workspace-aware ship rule applies — queue-advancing past a claimed version within the same bump level is explicitly permitted. package.json build script: three `{ git rev-parse HEAD ...; }` brace groups → `( git rev-parse HEAD ... )` subshells. Bun's Windows shell parser doesn't grok bash brace groups; subshells are POSIX-universal. Originated from @realcarsonterry PR #1460. CHANGELOG entry covers the full wave: - Windows install hardening (42-site _link_or_copy + cleanup compat) - Unicode sanitization architecture (handleCommandInternal + SSE replacer) - Build script POSIX-shell compat (subshells) - Doc rename (gbrain_sync_mode → artifacts_sync_mode) - Windows CI on paid faster runner - 4 new wave tests (29 cases) Frames each item as a current system property, not a fix narrative. Credits @realcarsonterry for PRs #1460, #1461, #1462, #1463 (the seed of the wave). Scope expansion to all 42 setup sites, every server egress path, Windows CI migration, and codex-flagged P0/P1 fixes (connect-chrome alias on Windows, SSE replacer, prefix-cleanup Windows compat) authored on this branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: post-ship sync for v1.38.0.0 Document the two architectural invariants that landed in v1.38.0.0 in their persistent homes (not just CHANGELOG): - README Windows section: add the `./setup` re-run-after-git-pull requirement that `_print_windows_copy_note_once` shows at runtime. - CONTRIBUTING "Things to know": add the no-raw-`ln` invariant for contributors editing `setup`, with the test that enforces it. - ARCHITECTURE: new "Unicode sanitization at server egress" section between Shell injection prevention and Prompt injection defense, with egress table (HTTP/batch/SSE) and the post-stringify-regex rationale. - CLAUDE.md: cross-references for both invariants, matching the v1.6.0.0 dual-listener pattern (each constraint says which files to read before editing and which test pins it). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci(windows): use windows-latest-8-cores instead of unregistered Blacksmith label actionlint failed PR #1505 because `blacksmith-2vcpu-windows-2022` isn't in the repo's approved runner-label list (actionlint.yaml only registers `ubicloud-standard-2`, and Ubicloud doesn't ship a Windows pool). Switch to GitHub's paid larger Windows runner `windows-latest-8-cores` — 4x the cores of the free `windows-latest` at the larger-runner billing rate, no new third-party CI provider, no actionlint config changes. CHANGELOG: replace "Blacksmith" / "blacksmith-2vcpu-windows-2022" / "~6x faster spin-up" claims with the actual choice (8 cores vs 4, paid larger runner). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci(windows): switch from windows-latest-8-cores to ubicloud-standard-2-windows `windows-latest-8-cores` sat queued indefinitely because the GitHub larger-runner billing isn't enabled at the org level — the "Queued — Waiting to run this check" status surfaced on PR #1505 with no progress for the whole CI run. Switch to Ubicloud Windows runners (`ubicloud-standard-2-windows`) so Windows CI uses the same provider as the existing Linux evals (`ubicloud-standard-2`). Billing stays under one account instead of two. Register the new label in actionlint.yaml alongside the existing ubicloud-standard-2 entry so actionlint doesn't reject it as unknown. CHANGELOG entry updated: runner row reflects the actual provider chosen, "Itemized changes" mentions the actionlint.yaml registration, and the narrative paragraph documents why `windows-latest-8-cores` failed first. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci: migrate all workflows to Ubicloud (Linux + Windows, 8-core) Switch every `runs-on` in this repo to Ubicloud so CI has a single billing surface, consistent capacity, and 4x more cores on the workloads that were previously stuck on free `ubuntu-latest` (2 cores). Windows uses Ubicloud's Windows pool too — `ubicloud-standard-8-windows` — so the queued-forever problem with GitHub's `windows-latest-8-cores` paid larger runner (org-level larger-runner billing not enabled) goes away. Workflows touched (9): - evals.yml, evals-periodic.yml, ci-image.yml — bump default + matrix from `ubicloud-standard-2` to `ubicloud-standard-8`. The one matrix entry that was already on -8 stays. - windows-free-tests.yml — `ubicloud-standard-2-windows` → `ubicloud-standard-8-windows`. - make-pdf-gate.yml — matrix `ubuntu-latest` → `ubicloud-standard-8`. macOS entry preserved; the poppler-install `if: matrix.os` conditional swaps to match the new label. - actionlint.yml, pr-title-sync.yml, skill-docs.yml, version-gate.yml — `ubuntu-latest` → `ubicloud-standard-8`. .github/actionlint.yaml registers all four Ubicloud labels in one place: - ubicloud-standard-2 - ubicloud-standard-8 - ubicloud-standard-2-windows (the v1.38.0.0 windows-free-tests target) - ubicloud-standard-8-windows (this PR's windows-free-tests target) Removed the duplicate `actionlint.yaml` at the repo root that I accidentally created in the prior commit — actionlint only reads `.github/actionlint.yaml`, so the root file was dead weight. CHANGELOG entry updated: a single "all Ubicloud" sentence in the narrative plus a metrics-row covering the runner pool change, and the itemized line expanded to enumerate the 9 affected workflows. The previously-orphaned "Itemized changes" line about just `windows-free-tests.yml` is replaced. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * ci(windows): revert to free `windows-latest` Ubicloud doesn't ship Windows runners — confirmed via their docs. The `ubicloud-standard-*-windows` labels I added do not exist and were causing `windows-free-tests` to sit "Queued — Waiting to run this check" forever (GitHub Actions can't tell a typoed label from a self-hosted runner that's about to register; it just waits). Three prior Windows-runner attempts all failed for different reasons: - `blacksmith-2vcpu-windows-2022` — Blacksmith app not installed on the org - `windows-latest-8-cores` — GitHub paid larger-runner billing not enabled - `ubicloud-standard-2/8-windows` — Ubicloud doesn't offer Windows at all The free `windows-latest` runner (4 cores, ~60s spin-up, $0) is the one path that actually runs. The wave-coverage Windows tests are <30s of real work; total job time stays under 2 minutes. Cleaned up `.github/actionlint.yaml` to drop the bogus `ubicloud-standard-*-windows` entries — kept only the two real Linux labels. CHANGELOG: split the runner-pool row into Linux (migrated to Ubicloud-8) vs Windows (stays on free windows-latest), with the why on each. Itemized line for windows-free-tests rewritten to reflect the actual outcome. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test(windows): skip Unix-only cases on Windows runner windows-free-tests on GitHub free windows-latest fails three cases that depend on Unix tooling the runner doesn't have: 1. `setup-windows-fallback.test.ts` behavior matrix — IS_WINDOWS=0 cells assert `ln -snf` produces a real symlink. On Windows-without-Developer- Mode (which the free `windows-latest` runner is), `ln -snf` silently creates a file copy. That's literally the bug `_link_or_copy` exists to work around, so the assertion can never pass there. Skip the whole describe block on win32. The static-invariant test (zero raw `ln` outside the helper body) above the matrix still runs and pins the shape the Windows install relies on. 2. `docs-config-keys.test.ts` round-trip — spawnSync(`bin/gstack-config`) on Windows doesn't read the bash shebang and fails to exec. Skip on win32; the deprecated-key denylist test in the same file still runs and is the actual invariant defending the v1.27.0.0 rename at the doc layer. Use `describe.skipIf(process.platform === 'win32', ...)` and `test.skipIf(process.platform === 'win32', ...)`. Tests still run on macOS and Linux unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
193 lines
6.7 KiB
Markdown
193 lines
6.7 KiB
Markdown
# Cross-machine memory with GBrain sync
|
||
|
||
gstack writes a lot of useful state to `~/.gstack/` — learnings, retros, CEO
|
||
plans, design docs, developer profile. By default, all of that dies when you
|
||
switch laptops. **GBrain sync** pushes a curated subset to a private git
|
||
repo so your memory follows you across machines and becomes indexable by
|
||
GBrain.
|
||
|
||
## What you get
|
||
|
||
- Work on machine A, pick up seamlessly on machine B.
|
||
- Your learnings, plans, and designs are visible in GBrain (if you use it).
|
||
- A clean off-ramp (`gstack-brain-uninstall`) that never touches your data.
|
||
- No daemon, no system service, no background process.
|
||
|
||
## What does NOT leave your machine
|
||
|
||
By design, these stay local even when sync is on:
|
||
|
||
- Credentials: `.auth.json`, `auth-token.json`, `sidebar-sessions/`,
|
||
`security/device-salt`, consumer tokens in `config.yaml`
|
||
- Machine-specific state: Chromium profiles, ONNX model weights,
|
||
caches, eval-cache, CDP-profile, one-time prompt markers
|
||
(`.welcome-seen`, `.telemetry-prompted`, `.vendoring-warned-*`, etc.)
|
||
- Question-preferences: per-machine UX preferences
|
||
(`question-preferences.json`, `question-log.jsonl`, `question-events.jsonl`).
|
||
|
||
The exact allowlist lives in `~/.gstack/.brain-allowlist`. The CLI manages
|
||
it; you can append your own entries below the marker line.
|
||
|
||
## First-run setup (30–90 seconds)
|
||
|
||
```bash
|
||
gstack-brain-init
|
||
```
|
||
|
||
The command:
|
||
|
||
1. Turns `~/.gstack/` into a git repo.
|
||
2. Asks for a remote URL (default: `gh repo create --private
|
||
gstack-brain-$USER`). Any git remote works — GitHub, GitLab, Gitea,
|
||
self-hosted.
|
||
3. Pushes an initial commit with just the config.
|
||
4. Writes `~/.gstack-brain-remote.txt` (URL-only, no secrets —
|
||
safe to copy to another machine).
|
||
5. Wires the gstack-brain repo into your local gbrain as a federated
|
||
source (via `gbrain sources add` + `git worktree`) so `gbrain search`
|
||
can index your synced learnings, plans, and designs. Implementation
|
||
lives in `bin/gstack-gbrain-source-wireup`. The old
|
||
`gstack-brain-reader add --ingest-url ...` HTTP path was removed in
|
||
v1.15.1.0 — it depended on a `/ingest-repo` endpoint gbrain never
|
||
shipped.
|
||
|
||
After init, the **next skill you run** will ask you ONE question about
|
||
privacy mode:
|
||
|
||
- **Everything allowlisted (recommended)**: learnings, reviews, plans,
|
||
designs, retros, timelines, and developer profile all sync.
|
||
- **Only artifacts**: plans, designs, retros, learnings — skip
|
||
behavioral data (timelines, developer profile).
|
||
- **Decline**: keep everything local. You can turn sync on later with
|
||
`gstack-config set artifacts_sync_mode full`.
|
||
|
||
Your answer is persisted. You won't be asked again.
|
||
|
||
## Cross-machine workflow
|
||
|
||
On machine A: run `gstack-brain-init` once. That's it — every skill
|
||
invocation now drains the sync queue at its start and end boundaries
|
||
(~200–800 ms network pause per skill).
|
||
|
||
On machine B:
|
||
|
||
1. Copy `~/.gstack-brain-remote.txt` from machine A to machine B
|
||
(password manager, dotfile repo, USB stick — your call).
|
||
2. Run any gstack skill. The preamble sees the URL file and prints:
|
||
```
|
||
BRAIN_SYNC: brain repo detected: <url>
|
||
BRAIN_SYNC: run 'gstack-brain-restore' to pull your cross-machine memory
|
||
```
|
||
3. Run `gstack-brain-restore`. That clones the repo, rehydrates your
|
||
learnings/plans/retros, and re-registers the git merge drivers.
|
||
4. Re-enter consumer tokens (they're machine-local and NOT synced —
|
||
`gstack-config set gbrain_token <your-token>`).
|
||
5. Next skill: your yesterday-on-machine-A learning surfaces. That's the
|
||
magical moment.
|
||
|
||
## Status, health, and queue depth
|
||
|
||
```bash
|
||
gstack-brain-sync --status
|
||
```
|
||
|
||
Shows: last successful push, pending queue depth, any sync blocks, and the
|
||
current privacy mode.
|
||
|
||
Every skill run prints a `BRAIN_SYNC:` line near the top of the preamble
|
||
output. Scan it for problems.
|
||
|
||
## Privacy modes in detail
|
||
|
||
| Mode | What syncs |
|
||
|------|------------|
|
||
| `off` | Nothing (default). |
|
||
| `artifacts-only` | Plans, designs, retros, learnings, reviews. Skips timelines + developer-profile. |
|
||
| `full` | Everything in the allowlist, including behavioral state. |
|
||
|
||
Change anytime with:
|
||
```bash
|
||
gstack-config set artifacts_sync_mode full
|
||
gstack-config set artifacts_sync_mode off
|
||
```
|
||
|
||
## Secret protection
|
||
|
||
Every commit is scanned for credential-shaped content before it leaves
|
||
your machine. Blocked patterns include:
|
||
|
||
- AWS access keys (`AKIA…`)
|
||
- GitHub tokens (`ghp_`, `gho_`, `ghu_`, `ghs_`, `ghr_`, `github_pat_`)
|
||
- OpenAI keys (`sk-…`)
|
||
- PEM blocks (`-----BEGIN …-----`)
|
||
- JWTs (`eyJ…`)
|
||
- Bearer tokens in JSON (`"authorization": "…"`, `"api_key": "…"`, etc.)
|
||
|
||
If a scan hits, sync stops, the queue is preserved, and your preamble
|
||
prints:
|
||
|
||
```
|
||
BRAIN_SYNC: blocked: <pattern-family>:<snippet>
|
||
```
|
||
|
||
To remediate:
|
||
|
||
1. Review the offending file.
|
||
2. If the match is a false positive on content you explicitly want to
|
||
sync, run `gstack-brain-sync --skip-file <path>` to permanently
|
||
exclude that path.
|
||
3. Otherwise, edit the file to remove the secret and re-run any skill.
|
||
|
||
There's a defense-in-depth hook at `~/.gstack/.git/hooks/pre-commit` that
|
||
runs the same scan if you manually `git commit` against the repo.
|
||
|
||
## Two-machine conflicts
|
||
|
||
If you write on machine A and machine B the same day, both will push
|
||
append commits. Git's default would conflict at the file tail, but the
|
||
`.jsonl` and markdown files are registered with custom merge drivers:
|
||
|
||
- JSONL files use a sort-and-dedup driver that orders appends by ISO
|
||
timestamp (falls back to SHA-256 hash of each line for determinism).
|
||
- Markdown artifacts (retros, plans, designs) use a union merge driver
|
||
that concatenates both sides.
|
||
|
||
You shouldn't see conflict prompts. If you do (a real semantic conflict,
|
||
like two machines editing the same plan), git will stop and prompt.
|
||
|
||
## Cross-machine pull cadence
|
||
|
||
The preamble runs `git fetch` + `git merge --ff-only` once per 24 hours
|
||
(cached via `~/.gstack/.brain-last-pull`). You don't need to think about
|
||
this — it happens automatically at the first skill invocation each day.
|
||
|
||
## Uninstall
|
||
|
||
```bash
|
||
gstack-brain-uninstall
|
||
```
|
||
|
||
This:
|
||
|
||
- Removes `~/.gstack/.git/` and all `.brain-*` config files.
|
||
- Clears `artifacts_sync_mode` in `gstack-config`.
|
||
- Does NOT touch your learnings, plans, retros, or developer profile.
|
||
|
||
Add `--delete-remote` to also delete the private GitHub repo (GitHub only,
|
||
uses `gh repo delete`).
|
||
|
||
Re-init anytime with `gstack-brain-init`.
|
||
|
||
## Troubleshooting
|
||
|
||
See [gbrain-sync-errors.md](gbrain-sync-errors.md) for an index of every
|
||
error message gstack-brain may print, with problem / cause / fix for each.
|
||
|
||
## Under the hood
|
||
|
||
For the architectural decisions behind this feature (allowlist vs
|
||
denylist, daemon vs preamble-boundary sync, JSONL merge driver, privacy
|
||
stop-gate), see the
|
||
[approved plan](../system-instruction-you-are-working-jaunty-kahn.md) in
|
||
the gstack plans directory.
|