ソースを参照

Document skill description trap: descriptions override flowcharts

Testing revealed that skill descriptions summarizing workflow cause
Claude to follow the description instead of reading the skill body.

- A description saying "code review between tasks" caused ONE review
- The flowchart clearly showed TWO reviews (spec compliance + quality)
- Minimal description ("Use when...") correctly deferred to flowchart

Updated writing-skills with:
- "Description = When to Use, NOT What the Skill Does" section
- Cautionary tale about this actual failure
- Examples of good (triggers only) vs bad (workflow summary) descriptions

Updated subagent-driven-development:
- Removed workflow summary from description
- Now just: "Use when executing implementation plans..."

Updated test runner:
- Added --dangerously-skip-permissions for automated testing
Jesse Vincent 10 ヶ月 前
親
コミット
2a19be0b78

+ 1 - 1
skills/subagent-driven-development/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: subagent-driven-development
-description: Use when executing implementation plans with independent tasks in the current session - dispatches fresh subagent for each task with two-stage review (spec compliance then code quality) between tasks
+description: Use when executing implementation plans with independent tasks in the current session
 ---
 
 # Subagent-Driven Development

+ 32 - 6
skills/writing-skills/SKILL.md

@@ -95,15 +95,16 @@ skills/
 - Only two fields supported: `name` and `description`
 - Max 1024 characters total
 - `name`: Use letters, numbers, and hyphens only (no parentheses, special chars)
-- `description`: Third-person, includes BOTH what it does AND when to use it
+- `description`: Third-person, describes ONLY when to use (NOT what it does)
   - Start with "Use when..." to focus on triggering conditions
   - Include specific symptoms, situations, and contexts
+  - **NEVER summarize the skill's process or workflow** (see CSO section for why)
   - Keep under 500 characters if possible
 
 ```markdown
 ---
 name: Skill-Name-With-Hyphens
-description: Use when [specific triggering conditions and symptoms] - [what the skill does and how it helps, written in third person]
+description: Use when [specific triggering conditions and symptoms]
 ---
 
 # Skill Name
@@ -143,7 +144,31 @@ Concrete results
 
 **Purpose:** Claude reads description to decide which skills to load for a given task. Make it answer: "Should I read this skill right now?"
 
-**Format:** Start with "Use when..." to focus on triggering conditions, then explain what it does
+**Format:** Start with "Use when..." to focus on triggering conditions
+
+**CRITICAL: Description = When to Use, NOT What the Skill Does**
+
+The description should ONLY describe triggering conditions. Do NOT summarize the skill's process or workflow in the description.
+
+**Why this matters:** Testing revealed that when a description summarizes the skill's workflow, Claude may follow the description instead of reading the full skill content. A description saying "code review between tasks" caused Claude to do ONE review, even though the skill's flowchart clearly showed TWO reviews (spec compliance then code quality).
+
+When the description was changed to just "Use when executing implementation plans with independent tasks" (no workflow summary), Claude correctly read the flowchart and followed the two-stage review process.
+
+**The trap:** Descriptions that summarize workflow create a shortcut Claude will take. The skill body becomes documentation Claude skips.
+
+```yaml
+# ❌ BAD: Summarizes workflow - Claude may follow this instead of reading skill
+description: Use when executing plans - dispatches subagent per task with code review between tasks
+
+# ❌ BAD: Too much process detail
+description: Use for TDD - write test first, watch it fail, write minimal code, refactor
+
+# ✅ GOOD: Just triggering conditions, no workflow summary
+description: Use when executing implementation plans with independent tasks in the current session
+
+# ✅ GOOD: Triggering conditions only
+description: Use when implementing any feature or bugfix, before writing implementation code
+```
 
 **Content:**
 - Use concrete triggers, symptoms, and situations that signal this skill applies
@@ -151,6 +176,7 @@ Concrete results
 - Keep triggers technology-agnostic unless the skill itself is technology-specific
 - If skill is technology-specific, make that explicit in the trigger
 - Write in third person (injected into system prompt)
+- **NEVER summarize the skill's process or workflow**
 
 ```yaml
 # ❌ BAD: Too abstract, vague, doesn't include when to use
@@ -162,11 +188,11 @@ description: I can help you with async tests when they're flaky
 # ❌ BAD: Mentions technology but skill isn't specific to it
 description: Use when tests use setTimeout/sleep and are flaky
 
-# ✅ GOOD: Starts with "Use when", describes problem, then what it does
-description: Use when tests have race conditions, timing dependencies, or pass/fail inconsistently - replaces arbitrary timeouts with condition polling for reliable async tests
+# ✅ GOOD: Starts with "Use when", describes problem, no workflow
+description: Use when tests have race conditions, timing dependencies, or pass/fail inconsistently
 
 # ✅ GOOD: Technology-specific skill with explicit trigger
-description: Use when using React Router and handling authentication redirects - provides patterns for protected routes and auth state management
+description: Use when using React Router and handling authentication redirects
 ```
 
 ### 2. Keyword Coverage

+ 2 - 1
tests/subagent-driven-dev/run-test.sh

@@ -71,10 +71,11 @@ echo ""
 
 # Run claude and capture output
 # Using stream-json to get token usage stats
-# Run from project directory so .claude/settings.local.json is picked up
+# --dangerously-skip-permissions for automated testing (subagents don't inherit parent settings)
 cd "$OUTPUT_DIR/project"
 claude -p "$PROMPT" \
   --plugin-dir "$PLUGIN_DIR" \
+  --dangerously-skip-permissions \
   --output-format stream-json \
   > "$LOG_FILE" 2>&1 || true