mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-18 18:32:28 +08:00
feat: safety hook skills + skill usage telemetry (v0.7.1) (#189)
* feat: add /careful, /freeze, /guard, /unfreeze safety hook skills Four new on-demand skills using Claude Code's PreToolUse hooks: - /careful: warns before destructive commands (rm -rf, DROP TABLE, force-push, etc.) - /freeze: blocks file edits outside a specified directory - /guard: composes both into one command - /unfreeze: clears freeze boundary without ending session Pure bash hook scripts with Python fallback for JSON edge cases. Safe exceptions for build artifacts (node_modules, dist, .next, etc.). Hook fire telemetry logs pattern name only (never command content). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add skill usage telemetry to preamble TemplateContext system passes skill name through resolver pipeline so each generated SKILL.md gets its own name baked into the telemetry line. Appends to ~/.gstack/analytics/skill-usage.jsonl on every invocation. Covers 14 preamble-using skills + 4 hook skills (inline telemetry). JSONL format: {"skill":"ship","ts":"...","repo":"my-project"} Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add analytics CLI for skill usage stats bun run analytics reads ~/.gstack/analytics/skill-usage.jsonl and shows top skills, per-repo breakdown, hook fire stats, and daily timeline. Supports --period 7d/30d/all. Handles missing/empty/malformed data. 22 unit tests cover parsing, filtering, formatting, and edge cases. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add skills-used-this-week to /retro Retro Step 2 now reads skill-usage.jsonl and shows which gstack skills were used during the retro window. Follows the same pattern as the Greptile signal and Backlog Health metrics — read file, filter by date, aggregate, present. Skips silently if no analytics data exists. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add hook script and telemetry tests 32 unit tests for check-careful.sh covering all 8 destructive patterns, safe exceptions, Python fallback, and malformed input handling. 7 unit tests for check-freeze.sh covering boundary enforcement, trailing slash edge case, and missing state file. Telemetry tests verify per-skill name correctness in generated output. Adds careful/freeze/guard/unfreeze/document-release to ALL_SKILLS. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump version to 0.6.5 + changelog + mark TODOs shipped Safety hook skills and skill usage telemetry shipped. Analytics CLI and /retro integration included. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: /debug auto-freezes edits to the module being debugged Add PreToolUse hooks (Edit/Write) to debug/SKILL.md.tmpl that reference the existing freeze/bin/check-freeze.sh. After Phase 1 investigation, /debug locks edits to the narrowest affected directory. Graceful degradation: if freeze script is unavailable, scope lock is skipped. Users can run /unfreeze to remove the restriction. Deferred 6 enhancements to TODOS.md, gated on telemetry showing the freeze hook actually fires in real debugging sessions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
47
TODOS.md
47
TODOS.md
@@ -506,34 +506,37 @@ Shipped as v0.5.0 on main. Includes `/plan-design-review` (report-only design au
|
||||
|
||||
## Safety & Observability
|
||||
|
||||
### On-demand hook skills (/careful, /freeze, /guard)
|
||||
### On-demand hook skills (/careful, /freeze, /guard) — SHIPPED
|
||||
|
||||
**What:** Three new skills that use Claude Code's session-scoped PreToolUse hooks to add safety guardrails on demand.
|
||||
~~**What:** Three new skills that use Claude Code's session-scoped PreToolUse hooks to add safety guardrails on demand.~~
|
||||
|
||||
**Why:** Anthropic's internal skill best practices recommend on-demand hooks for safety. Claude Code already handles destructive command permissions, but these add an explicit opt-in layer for high-risk sessions (touching prod, debugging live systems).
|
||||
Shipped as `/careful`, `/freeze`, `/guard`, and `/unfreeze` in v0.6.5. Includes hook fire-rate telemetry (pattern name only, no command content) and inline skill activation telemetry.
|
||||
|
||||
**Skills:**
|
||||
- `/careful` — PreToolUse hook on Bash tool. Warns (not blocks) before destructive commands: `rm -rf`, `DROP TABLE`, `git push --force`, `git reset --hard`, `kubectl delete`, `docker system prune`. Uses `permissionDecision: "ask"` so user can override.
|
||||
- `/freeze` — PreToolUse hook on Edit/Write tools. Restricts file edits to a user-specified directory. Great for debugging without accidentally "fixing" unrelated code.
|
||||
- `/guard` — meta-skill composing `/careful` + `/freeze` into one command.
|
||||
### Skill usage telemetry — SHIPPED
|
||||
|
||||
**Implementation notes:** Use `${CLAUDE_SKILL_DIR}` (not `${SKILL_DIR}`) for script paths in hook commands. Pure bash JSON parsing (no jq dependency). Freeze dir storage: `${CLAUDE_PLUGIN_DATA}/freeze-dir.txt` with `~/.gstack/freeze-dir.txt` fallback. Ensure trailing `/` on freeze dir paths to prevent `/src` matching `/src-old`.
|
||||
~~**What:** Track which skills get invoked, how often, from which repo.~~
|
||||
|
||||
**Effort:** M (human) / S (CC)
|
||||
Shipped in v0.6.5. TemplateContext in gen-skill-docs.ts bakes skill name into preamble telemetry line. Analytics CLI (`bun run analytics`) for querying. /retro integration shows skills-used-this-week.
|
||||
|
||||
### /debug scoped debugging enhancements (gated on telemetry)
|
||||
|
||||
**What:** Six enhancements to /debug auto-freeze, contingent on telemetry showing the freeze hook actually fires in real debugging sessions.
|
||||
|
||||
**Why:** /debug v0.7.1 auto-freezes edits to the module being debugged. If telemetry shows the hook fires often, these enhancements make the experience smarter. If it never fires, the problem wasn't real and these aren't worth building.
|
||||
|
||||
**Context:** All items are prose additions to `debug/SKILL.md.tmpl`. No new scripts.
|
||||
|
||||
**Items:**
|
||||
1. Stack trace auto-detection for freeze directory (parse deepest app frame)
|
||||
2. Freeze boundary widening (ask to widen instead of hard-block when hitting boundary)
|
||||
3. Post-fix auto-unfreeze + full test suite run
|
||||
4. Debug instrumentation cleanup (tag with DEBUG-TEMP, remove before commit)
|
||||
5. Debug session persistence (~/.gstack/debug-sessions/ — save investigation for reuse)
|
||||
6. Investigation timeline in debug report (hypothesis log with timing)
|
||||
|
||||
**Effort:** M (all 6 combined)
|
||||
**Priority:** P3
|
||||
**Depends on:** None
|
||||
|
||||
### Skill usage telemetry
|
||||
|
||||
**What:** Track which skills get invoked, how often, from which repo.
|
||||
|
||||
**Why:** Enables finding undertriggering skills and measuring adoption. Anthropic uses a PreToolUse hook for this; simpler approach is appending JSONL from the preamble.
|
||||
|
||||
**Context:** Add to `generatePreamble()` in `scripts/gen-skill-docs.ts`. Append to `~/.gstack/analytics/skill-usage.jsonl` with skill name, timestamp, and repo name. `mkdir -p` ensures the directory exists.
|
||||
|
||||
**Effort:** S (human) / S (CC)
|
||||
**Priority:** P3
|
||||
**Depends on:** None
|
||||
**Depends on:** Telemetry data showing freeze hook fires in real /debug sessions
|
||||
|
||||
## Completed
|
||||
|
||||
|
||||
Reference in New Issue
Block a user