Commit Graph

2 Commits

Author SHA1 Message Date
Garry Tan
db3b6412b9 feat: one-way door classifier (belt-and-suspenders safety fallback)
scripts/one-way-doors.ts — secondary keyword-pattern classifier that catches
destructive questions even when the registry doesn't have an entry for them.

The registry's door_type field (from scripts/question-registry.ts) is the
PRIMARY safety gate. This classifier is the fallback for ad-hoc question_ids
that agents generate at runtime.

Classification priority:
  1. Registry lookup by question_id → use declared door_type
  2. Skill:category fallback (cso:approval, land-and-deploy:approval)
  3. Keyword pattern match against question_summary
  4. Default: treat as two-way (safer to log the miss than auto-decide unsafely)

Covers 21 destructive patterns across:
  - File system (rm -rf, delete, wipe, purge, truncate)
  - Database (drop table/database/schema, delete from)
  - Git/VCS (force-push, reset --hard, checkout --, branch -D)
  - Deploy/infra (kubectl delete, terraform destroy, rollback)
  - Credentials (revoke/reset/rotate API key|token|secret|password)
  - Architecture (breaking change, schema migration, data model change)

7 new tests in test/plan-tune.test.ts covering: registry-first lookup,
unknown-id fallthrough, keyword matching on destructive phrasings including
embedded filler words ("rotate the API key"), skill-category fallback,
benign questions defaulting to two-way, pattern-list non-empty.

27 pass, 0 fail. 1270 expect() calls.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 06:12:39 +08:00
Garry Tan
2c0c95099f test: registry schema + safety + coverage tests (gate tier)
20 tests validating the question registry:

Schema (7 tests):
- Every entry has required fields
- All ids are kebab-case and start with their skill name
- No duplicate ids
- Categories are from the allowed set
- door_type is one-way | two-way
- Options arrays are well-formed
- Descriptions are short and single-line

Helpers (5 tests):
- getQuestion returns entry for known id, undefined for unknown
- getOneWayDoorIds includes destructive questions, excludes two-way
- getAllRegisteredIds count matches QUESTIONS keys
- getRegistryStats totals are internally consistent

One-way door safety (2 tests):
- Every critical question (test failure, SQL safety, LLM trust boundary,
  security scan, merge confirm, rollback, fix apply, premise revise,
  arch finding, privacy gate, user challenge) is declared one-way
- At least 10 one-way doors exist (catches regression if declarations
  are accidentally dropped)

Registry breadth (3 tests):
- 11 high-volume skills each have >= 1 registered question
- Preamble one-time prompts are registered
- /plan-tune's own questions are registered

Signal map references (1 test):
- signal_key values are typed kebab-case strings

Template coverage (2 tests, informational):
- AskUserQuestion usage across templates is non-trivial (>20)
- Registry spans >= 10 skills

20 pass, 0 fail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 06:10:57 +08:00