Garry Tan 901ad53aa5 docs: add E2E eval failure blame protocol ...

"Not related to our changes" is an extraordinary claim that requires
extraordinary proof. When evals fail during /ship:

1. Run the same eval on main — prove it fails there too
2. If it passes on main, it IS your change — trace the blame
3. If you can't verify, say "unverified" not "pre-existing"

Added to CLAUDE.md and as a comment in skill-e2e.test.ts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-16 10:21:46 -05:00

..

fixtures

feat: contributor mode, session awareness, recommendation format (#90)

2026-03-16 01:45:50 -05:00

helpers

merge: integrate origin/main (v0.4.0, v0.4.1) into team-supabase-store

2026-03-16 07:49:27 -05:00

gen-skill-docs.test.ts

feat: contributor mode, session awareness, recommendation format (#90)

2026-03-16 01:45:50 -05:00

lib-dashboard-queries.test.ts

feat: add dashboard query functions — pure transforms for team analytics

2026-03-16 02:43:52 -05:00

lib-dashboard-ui.test.ts

feat: add shared team dashboard, regression alerts, weekly digest edge functions

2026-03-16 02:44:47 -05:00

lib-eval-cache.test.ts

feat: add SHA-based eval caching with EVAL_CACHE=0 bypass

2026-03-15 09:39:26 -05:00

lib-eval-cli.test.ts

feat: add CLI leaderboard, refactor formatTeamSummary to use dashboard-queries

2026-03-16 02:44:12 -05:00

lib-eval-cost.test.ts

feat: add eval format validation, tier selection, cost tracking

2026-03-15 09:39:18 -05:00

lib-eval-format.test.ts

feat: add eval format validation, tier selection, cost tracking

2026-03-15 09:39:18 -05:00

lib-eval-tier.test.ts

feat: add eval format validation, tier selection, cost tracking

2026-03-15 09:39:18 -05:00

lib-eval-trend.test.ts

feat: add eval:trend CLI for per-test pass rate tracking

2026-03-15 16:47:41 -05:00

lib-llm-summarize.test.ts

feat: add push-transcript CLI, show sessions, interactive setup, 36 tests

2026-03-16 00:15:26 -05:00

lib-sync-config.test.ts

feat: hook eval-store sync, use shared utils, add 30 lib tests

2026-03-15 02:02:54 -05:00

lib-sync-show.test.ts

feat: add push-transcript CLI, show sessions, interactive setup, 36 tests

2026-03-16 00:15:26 -05:00

lib-sync.test.ts

feat: hook eval-store sync, use shared utils, add 30 lib tests

2026-03-15 02:02:54 -05:00

lib-team-admin.test.ts

feat: add team admin CLI + migration 007 (settings, cooldowns, create_team RPC)

2026-03-16 02:44:24 -05:00

lib-transcript-sync.test.ts

feat: add push-transcript CLI, show sessions, interactive setup, 36 tests

2026-03-16 00:15:26 -05:00

lib-util.test.ts

feat: add listEvalFiles, loadEvalResults, formatTimestamp to lib/util.ts

2026-03-15 09:39:09 -05:00

skill-e2e.test.ts

docs: add E2E eval failure blame protocol

2026-03-16 10:21:46 -05:00

skill-llm-eval.test.ts

feat: wire eval-cache + eval-tier into LLM judge, pin E2E model

2026-03-15 16:47:35 -05:00

skill-parser.test.ts

feat: SKILL.md template system, 3-tier testing, DX tools (v0.3.3) (#41)

2026-03-13 21:08:12 -07:00

skill-validation.test.ts

test: add deterministic contributor mode preamble validation

2026-03-16 09:44:23 -05:00