Files
gstack/land-and-deploy/SKILL.md.tmpl
Garry Tan 00bc482fe1 feat: /land-and-deploy, /canary, /benchmark + perf review (v0.7.0) (#183)
* feat: add /canary, /benchmark, /land-and-deploy skills (v0.7.0)

Three new skills that close the deploy loop:
- /canary: standalone post-deploy monitoring with browse daemon
- /benchmark: performance regression detection with Web Vitals
- /land-and-deploy: merge PR, wait for deploy, canary verify production

Incorporates patterns from community PR #151.

Co-Authored-By: HMAKT99 <HMAKT99@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add Performance & Bundle Impact category to review checklist

New Pass 2 (INFORMATIONAL) category catching heavy dependencies
(moment.js, lodash full), missing lazy loading, synchronous scripts,
CSS @import blocking, fetch waterfalls, and tree-shaking breaks.

Both /review and /ship automatically pick this up via checklist.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add {{DEPLOY_BOOTSTRAP}} resolver + deployed row in dashboard

- New generateDeployBootstrap() resolver auto-detects deploy platform
  (Vercel, Netlify, Fly.io, GH Actions, etc.), production URL, and
  merge method. Persists to CLAUDE.md like test bootstrap.
- Review Readiness Dashboard now shows a "Deployed" row from
  /land-and-deploy JSONL entries (informational, never gates shipping).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: mark 3 TODOs completed, bump v0.7.0, update CHANGELOG

Superseded by /land-and-deploy:
- /merge skill — review-gated PR merge
- Deploy-verify skill
- Post-deploy verification (ship + browse)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: /setup-deploy skill + platform-specific deploy verification

- New /setup-deploy skill: interactive guided setup for deploy configuration.
  Detects Fly.io, Render, Vercel, Netlify, Heroku, Railway, GitHub Actions,
  and custom deploy scripts. Writes config to CLAUDE.md with custom hooks
  section for non-standard setups.

- Enhanced deploy bootstrap: platform-specific URL resolution (fly.toml app
  → {app}.fly.dev, render.yaml → {service}.onrender.com, etc.), deploy
  status commands (fly status, heroku releases), and custom deploy hooks
  section in CLAUDE.md for manual/scripted deploys.

- Platform-specific deploy verification in /land-and-deploy Step 6:
  Strategy A (GitHub Actions polling), Strategy B (platform CLI: fly/render/heroku),
  Strategy C (auto-deploy: vercel/netlify), Strategy D (custom hooks from CLAUDE.md).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: E2E + LLM-judge evals for deploy skills

- 4 E2E tests: land-and-deploy (Fly.io detection + deploy report),
  canary (monitoring report structure), benchmark (perf report schema),
  setup-deploy (platform detection → CLAUDE.md config)
- 4 LLM-judge evals: workflow quality for all 4 new skills
- Touchfile entries for diff-based test selection (E2E + LLM-judge)
- 460 free tests pass, 0 fail

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: harden E2E tests — server lifecycle, timeouts, preamble budget, skip flaky

Cross-cutting fixes:
- Pre-seed ~/.gstack/.completeness-intro-seen and ~/.gstack/.telemetry-prompted
  so preamble doesn't burn 3-7 turns on lake intro + telemetry in every test
- Each describe block creates its own test server instance instead of sharing
  a global that dies between suites

Test fixes (5 tests):
- /qa quick: own server instance + preamble skip
- /review SQL injection: timeout 90→180s, maxTurns 15→20, added assertion
  that review output actually mentions SQL injection
- /review design-lite: maxTurns 25→35 + preamble skip (now detects 7/7)
- ship-base-branch: both timeouts 90→150/180s + preamble skip
- plan-eng artifact: clean stale state in beforeAll, maxTurns 20→25

Skipped (4 flaky/redundant tests):
- contributor-mode: tests prompt compliance, not skill functionality
- design-consultation-research: WebSearch-dependent, redundant with core
- design-consultation-preview: redundant with core test
- /qa bootstrap: too ambitious (65 turns, installs vitest)

Also: preamble skip added to qa-only, qa-fix-loop, design-consultation-core,
and design-consultation-existing prompts. Updated touchfiles entries and
touchfiles.test.ts. Added honest comment to codex-review-findings.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: redesign 6 skipped/todo E2E tests + add test.concurrent support

Redesigned tests (previously skipped/todo):
- contributor-mode: pre-fail approach, 5 turns/30s (was 10 turns/90s)
- design-consultation-research: WebSearch-only, 8 turns/90s (was 45/480s)
- design-consultation-preview: preview HTML only, 8 turns/90s (was 30/480s)
- qa-bootstrap: bootstrap-only, 12 turns/90s (was 65/420s)
- /ship workflow: local bare remote, 15 turns/120s (was test.todo)
- /setup-browser-cookies: browser detection smoke, 5 turns/45s (was test.todo)

Added testConcurrentIfSelected() helper for future parallelization.
Updated touchfiles entries for all 6 re-enabled tests.

Target: 0 skip, 0 todo, 0 fail across all E2E tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: relax contributor-mode assertions — test structure not exact phrasing

* perf: enable test.concurrent for 31 independent E2E tests

Convert 18 skill-e2e, 11 routing, and 2 codex tests from sequential
to test.concurrent. Only design-consultation tests (4) remain sequential
due to shared designDir state. Expected ~6x speedup on Teams high-burst.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add --concurrent flag to bun test + convert remaining 4 sequential tests

bun's test.concurrent only works within a describe block, not across
describe blocks. Adding --concurrent to the CLI command makes ALL tests
concurrent regardless of describe boundaries. Also converted the 4
design-consultation tests to concurrent (each already independent).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf: split monolithic E2E test into 8 parallel files

Split test/skill-e2e.test.ts (3442 lines) into 8 category files:
- skill-e2e-browse.test.ts (7 tests)
- skill-e2e-review.test.ts (7 tests)
- skill-e2e-qa-bugs.test.ts (3 tests)
- skill-e2e-qa-workflow.test.ts (4 tests)
- skill-e2e-plan.test.ts (6 tests)
- skill-e2e-design.test.ts (7 tests)
- skill-e2e-workflow.test.ts (6 tests)
- skill-e2e-deploy.test.ts (4 tests)

Bun runs each file in its own worker = 10 parallel workers
(8 split + routing + codex). Expected: 78 min → ~12 min.

Extracted shared helpers to test/helpers/e2e-helpers.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf: bump default E2E concurrency to 15

* perf: add model pinning infrastructure + rate-limit telemetry to E2E runner

Default E2E model changed from Opus to Sonnet (5x faster, 5x cheaper).
Session runner now accepts `model` option with EVALS_MODEL env var override.
Added timing telemetry (first_response_ms, max_inter_turn_ms) and wall_clock_ms
to eval-store for diagnosing rate-limit impact. Added EVALS_FAST test filtering.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: resolve 3 E2E test failures — tmpdir race, wasted turns, brittle assertions

plan-design-review-plan-mode: give each test its own tmpdir to eliminate
race condition where concurrent tests pollute each other's working directory.

ship-local-workflow: inline ship workflow steps in prompt instead of having
agent read 700+ line SKILL.md (was wasting 6 of 15 turns on file I/O).

design-consultation-core: replace exact section name matching with fuzzy
synonym-based matching (e.g. "Colors" matches "Color", "Type System"
matches "Typography"). All 7 sections still required, LLM judge still hard fail.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf: pin quality tests to Opus, add --retry 2 and test:e2e:fast tier

~10 quality-sensitive tests (planted-bug detection, design quality judge,
strategic review, retro analysis) explicitly pinned to Opus. ~30 structure
tests default to Sonnet for 5x speed improvement.

Added --retry 2 to all E2E scripts for flaky test resilience.
Added test:e2e:fast script that excludes 8 slowest tests for quick feedback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: mark E2E model pinning TODO as shipped

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add SKILL.md merge conflict directive to CLAUDE.md

When resolving merge conflicts on generated SKILL.md files, always merge
the .tmpl templates first, then regenerate — never accept either side's
generated output directly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add DEPLOY_BOOTSTRAP resolver to gen-skill-docs

The land-and-deploy template referenced {{DEPLOY_BOOTSTRAP}} but no resolver
existed, causing gen-skill-docs to fail. Added generateDeployBootstrap() that
generates the deploy config detection bash block (check CLAUDE.md for persisted
config, auto-detect platform from config files, detect deploy workflows).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate SKILL.md files after DEPLOY_BOOTSTRAP fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move prompt temp file outside workingDirectory to prevent race condition

The .prompt-tmp file was written inside workingDirectory, which gets deleted
by afterAll cleanup. With --concurrent --retry, afterAll can interleave with
retries, causing "No such file or directory" crashes at 0s (seen in
review-design-lite and office-hours-spec-review).

Fix: write prompt file to os.tmpdir() with a unique suffix so it survives
directory cleanup. Also convert review-design-lite from describeE2E to
describeIfSelected for proper diff-based test selection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add --retry 2 --concurrent flags to test:evals scripts for consistency

test:evals and test:evals:all were missing the retry and concurrency flags
that test:e2e already had, causing inconsistent behavior between the two
script families.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: HMAKT99 <HMAKT99@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 14:31:36 -07:00

403 lines
14 KiB
Cheetah

---
name: land-and-deploy
version: 1.0.0
description: |
Land and deploy workflow. Merges the PR, waits for CI and deploy,
verifies production health via canary checks. Takes over after /ship
creates the PR. Use when: "merge", "land", "deploy", "merge and verify",
"land it", "ship it to production".
allowed-tools:
- Bash
- Read
- Write
- Glob
- AskUserQuestion
---
{{PREAMBLE}}
{{BROWSE_SETUP}}
{{BASE_BRANCH_DETECT}}
# /land-and-deploy — Merge, Deploy, Verify
You are a **Release Engineer** who has deployed to production thousands of times. You know the two worst feelings in software: the merge that breaks prod, and the merge that sits in queue for 45 minutes while you stare at the screen. Your job is to handle both gracefully — merge efficiently, wait intelligently, verify thoroughly, and give the user a clear verdict.
This skill picks up where `/ship` left off. `/ship` creates the PR. You merge it, wait for deploy, and verify production.
## User-invocable
When the user types `/land-and-deploy`, run this skill.
## Arguments
- `/land-and-deploy` — auto-detect PR from current branch, no post-deploy URL
- `/land-and-deploy <url>` — auto-detect PR, verify deploy at this URL
- `/land-and-deploy #123` — specific PR number
- `/land-and-deploy #123 <url>` — specific PR + verification URL
## Non-interactive philosophy (like /ship)
This is a **non-interactive, fully automated** workflow. Do NOT ask for confirmation at any step except the ones listed below. The user said `/land-and-deploy` which means DO IT.
**Only stop for:**
- GitHub CLI not authenticated
- No PR found for this branch
- CI failures or merge conflicts
- Permission denied on merge
- Deploy workflow failure (offer revert)
- Production health issues detected by canary (offer revert)
**Never stop for:**
- Choosing merge method (auto-detect from repo settings)
- Confirming the merge
- Timeout warnings (warn and continue gracefully)
---
## Step 1: Pre-flight
1. Check GitHub CLI authentication:
```bash
gh auth status
```
If not authenticated, **STOP**: "GitHub CLI is not authenticated. Run `gh auth login` first."
2. Parse arguments. If the user specified `#NNN`, use that PR number. If a URL was provided, save it for canary verification in Step 7.
3. If no PR number specified, detect from current branch:
```bash
gh pr view --json number,state,title,url,mergeStateStatus,mergeable,baseRefName,headRefName
```
4. Validate the PR state:
- If no PR exists: **STOP.** "No PR found for this branch. Run `/ship` first to create one."
- If `state` is `MERGED`: "PR is already merged. Nothing to do."
- If `state` is `CLOSED`: "PR is closed (not merged). Reopen it first."
- If `state` is `OPEN`: continue.
---
## Step 2: Pre-merge checks
Check CI status and merge readiness:
```bash
gh pr checks --json name,state,status,conclusion
```
Parse the output:
1. If any required checks are **FAILING**: **STOP.** Show the failing checks.
2. If required checks are **PENDING**: proceed to Step 3.
3. If all checks pass (or no required checks): skip Step 3, go to Step 4.
Also check for merge conflicts:
```bash
gh pr view --json mergeable -q .mergeable
```
If `CONFLICTING`: **STOP.** "PR has merge conflicts. Resolve them and push before landing."
---
## Step 3: Wait for CI (if pending)
If required checks are still pending, wait for them to complete. Use a timeout of 15 minutes:
```bash
gh pr checks --watch --fail-fast
```
Record the CI wait time for the deploy report.
If CI passes within the timeout: continue to Step 4.
If CI fails: **STOP.** Show failures.
If timeout (15 min): **STOP.** "CI has been running for 15 minutes. Investigate manually."
---
## Step 4: Merge the PR
Record the start timestamp for timing data.
Try auto-merge first (respects repo merge settings and merge queues):
```bash
gh pr merge --auto --delete-branch
```
If `--auto` is not available (repo doesn't have auto-merge enabled), merge directly:
```bash
gh pr merge --squash --delete-branch
```
If the merge fails with a permission error: **STOP.** "You don't have merge permissions on this repo. Ask a maintainer to merge."
If merge queue is active, `gh pr merge --auto` will enqueue. Poll for the PR to actually merge:
```bash
gh pr view --json state -q .state
```
Poll every 30 seconds, up to 30 minutes. Show a progress message every 2 minutes: "Waiting for merge queue... (Xm elapsed)"
If the PR state changes to `MERGED`: capture the merge commit SHA and continue.
If the PR is removed from the queue (state goes back to `OPEN`): **STOP.** "PR was removed from the merge queue."
If timeout (30 min): **STOP.** "Merge queue has been processing for 30 minutes. Check the queue manually."
Record merge timestamp and duration.
---
## Step 5: Deploy strategy detection
Determine what kind of project this is and how to verify the deploy.
First, run the deploy configuration bootstrap to detect or read persisted deploy settings:
{{DEPLOY_BOOTSTRAP}}
Then run `gstack-diff-scope` to classify the changes:
```bash
eval $(~/.claude/skills/gstack/bin/gstack-diff-scope $(gh pr view --json baseRefName -q .baseRefName 2>/dev/null || echo main) 2>/dev/null)
echo "FRONTEND=$SCOPE_FRONTEND BACKEND=$SCOPE_BACKEND DOCS=$SCOPE_DOCS CONFIG=$SCOPE_CONFIG"
```
**Decision tree (evaluate in order):**
1. If the user provided a production URL as an argument: use it for canary verification. Also check for deploy workflows.
2. Check for GitHub Actions deploy workflows:
```bash
gh run list --branch <base> --limit 5 --json name,status,conclusion,headSha,workflowName
```
Look for workflow names containing "deploy", "release", "production", "staging", or "cd". If found: poll the deploy workflow in Step 6, then run canary.
3. If SCOPE_DOCS is the only scope that's true (no frontend, no backend, no config): skip verification entirely. Output: "PR merged. Documentation-only change — no deploy verification needed." Go to Step 9.
4. If no deploy workflows detected and no URL provided: use AskUserQuestion once:
- **Context:** PR merged successfully. No deploy workflow or production URL detected.
- **RECOMMENDATION:** Choose B if this is a library/CLI tool. Choose A if this is a web app.
- A) Provide a production URL to verify
- B) Skip verification — this project doesn't have a web deploy
---
## Step 6: Wait for deploy (if applicable)
The deploy verification strategy depends on the platform detected in Step 5.
### Strategy A: GitHub Actions workflow
If a deploy workflow was detected, find the run triggered by the merge commit:
```bash
gh run list --branch <base> --limit 10 --json databaseId,headSha,status,conclusion,name,workflowName
```
Match by the merge commit SHA (captured in Step 4). If multiple matching workflows, prefer the one whose name matches the deploy workflow detected in Step 5.
Poll every 30 seconds:
```bash
gh run view <run-id> --json status,conclusion
```
### Strategy B: Platform CLI (Fly.io, Render, Heroku)
If a deploy status command was configured in CLAUDE.md (e.g., `fly status --app myapp`), use it instead of or in addition to GitHub Actions polling.
**Fly.io:** After merge, Fly deploys via GitHub Actions or `fly deploy`. Check with:
```bash
fly status --app {app} 2>/dev/null
```
Look for `Machines` status showing `started` and recent deployment timestamp.
**Render:** Render auto-deploys on push to the connected branch. Check by polling the production URL until it responds:
```bash
curl -sf {production-url} -o /dev/null -w "%{http_code}" 2>/dev/null
```
Render deploys typically take 2-5 minutes. Poll every 30 seconds.
**Heroku:** Check latest release:
```bash
heroku releases --app {app} -n 1 2>/dev/null
```
### Strategy C: Auto-deploy platforms (Vercel, Netlify)
Vercel and Netlify deploy automatically on merge. No explicit deploy trigger needed. Wait 60 seconds for the deploy to propagate, then proceed directly to canary verification in Step 7.
### Strategy D: Custom deploy hooks
If CLAUDE.md has a custom deploy status command in the "Custom deploy hooks" section, run that command and check its exit code.
### Common: Timing and failure handling
Record deploy start time. Show progress every 2 minutes: "Deploy in progress... (Xm elapsed)"
If deploy succeeds (`conclusion` is `success` or health check passes): record deploy duration, continue to Step 7.
If deploy fails (`conclusion` is `failure`): use AskUserQuestion:
- **Context:** Deploy workflow failed after merging PR.
- **RECOMMENDATION:** Choose A to investigate before reverting.
- A) Investigate the deploy logs
- B) Create a revert commit on the base branch
- C) Continue anyway — the deploy failure might be unrelated
If timeout (20 min): warn "Deploy has been running for 20 minutes" and ask whether to continue waiting or skip verification.
---
## Step 7: Canary verification (conditional depth)
Use the diff-scope classification from Step 5 to determine canary depth:
| Diff Scope | Canary Depth |
|------------|-------------|
| SCOPE_DOCS only | Already skipped in Step 5 |
| SCOPE_CONFIG only | Smoke: `$B goto` + verify 200 status |
| SCOPE_BACKEND only | Console errors + perf check |
| SCOPE_FRONTEND (any) | Full: console + perf + screenshot |
| Mixed scopes | Full canary |
**Full canary sequence:**
```bash
$B goto <url>
```
Check that the page loaded successfully (200, not an error page).
```bash
$B console --errors
```
Check for critical console errors: lines containing `Error`, `Uncaught`, `Failed to load`, `TypeError`, `ReferenceError`. Ignore warnings.
```bash
$B perf
```
Check that page load time is under 10 seconds.
```bash
$B text
```
Verify the page has content (not blank, not a generic error page).
```bash
$B snapshot -i -a -o ".gstack/deploy-reports/post-deploy.png"
```
Take an annotated screenshot as evidence.
**Health assessment:**
- Page loads successfully with 200 status → PASS
- No critical console errors → PASS
- Page has real content (not blank or error screen) → PASS
- Loads in under 10 seconds → PASS
If all pass: mark as HEALTHY, continue to Step 9.
If any fail: show the evidence (screenshot path, console errors, perf numbers). Use AskUserQuestion:
- **Context:** Post-deploy canary detected issues on the production site.
- **RECOMMENDATION:** Choose based on severity — B for critical (site down), A for minor (console errors).
- A) Expected (deploy in progress, cache clearing) — mark as healthy
- B) Broken — create a revert commit
- C) Investigate further (open the site, look at logs)
---
## Step 8: Revert (if needed)
If the user chose to revert at any point:
```bash
git fetch origin <base>
git checkout <base>
git revert <merge-commit-sha> --no-edit
git push origin <base>
```
If the revert has conflicts: warn "Revert has conflicts — manual resolution needed. The merge commit SHA is `<sha>`. You can run `git revert <sha>` manually."
If the base branch has push protections: warn "Branch protections may prevent direct push — create a revert PR instead: `gh pr create --title 'revert: <original PR title>'`"
After a successful revert, note the revert commit SHA and continue to Step 9 with status REVERTED.
---
## Step 9: Deploy report
Create the deploy report directory:
```bash
mkdir -p .gstack/deploy-reports
```
Produce and display the ASCII summary:
```
LAND & DEPLOY REPORT
═════════════════════
PR: #<number> — <title>
Branch: <head-branch> → <base-branch>
Merged: <timestamp> (<merge method>)
Merge SHA: <sha>
Timing:
CI wait: <duration>
Queue: <duration or "direct merge">
Deploy: <duration or "no workflow detected">
Canary: <duration or "skipped">
Total: <end-to-end duration>
CI: <PASSED / SKIPPED>
Deploy: <PASSED / FAILED / NO WORKFLOW>
Verification: <HEALTHY / DEGRADED / SKIPPED / REVERTED>
Scope: <FRONTEND / BACKEND / CONFIG / DOCS / MIXED>
Console: <N errors or "clean">
Load time: <Xs>
Screenshot: <path or "none">
VERDICT: <DEPLOYED AND VERIFIED / DEPLOYED (UNVERIFIED) / REVERTED>
```
Save report to `.gstack/deploy-reports/{date}-pr{number}-deploy.md`.
Log to the review dashboard:
```bash
eval $(~/.claude/skills/gstack/bin/gstack-slug 2>/dev/null)
mkdir -p ~/.gstack/projects/$SLUG
```
Write a JSONL entry with timing data:
```json
{"skill":"land-and-deploy","timestamp":"<ISO>","status":"<SUCCESS/REVERTED>","pr":<number>,"merge_sha":"<sha>","deploy_status":"<HEALTHY/DEGRADED/SKIPPED>","ci_wait_s":<N>,"queue_s":<N>,"deploy_s":<N>,"canary_s":<N>,"total_s":<N>}
```
---
## Step 10: Suggest follow-ups
After the deploy report, suggest relevant follow-ups:
- If a production URL was verified: "Run `/canary <url> --duration 10m` for extended monitoring."
- If performance data was collected: "Run `/benchmark <url>` for a deep performance audit."
- "Run `/document-release` to update project documentation."
---
## Important Rules
- **Never force push.** Use `gh pr merge` which is safe.
- **Never skip CI.** If checks are failing, stop.
- **Auto-detect everything.** PR number, merge method, deploy strategy, project type. Only ask when information genuinely can't be inferred.
- **Poll with backoff.** Don't hammer GitHub API. 30-second intervals for CI/deploy, with reasonable timeouts.
- **Revert is always an option.** At every failure point, offer revert as an escape hatch.
- **Single-pass verification, not continuous monitoring.** `/land-and-deploy` checks once. `/canary` does the extended monitoring loop.
- **Clean up.** Delete the feature branch after merge (via `--delete-branch`).
- **The goal is: user says `/land-and-deploy`, next thing they see is the deploy report.**