feat(skills): add proof-movie and demo-movie evidence recipes

skills/agentic-end-to-end-testing/recording-a-proof-movie.md

@@ -0,0 +1,137 @@
+# Recording a Proof Movie (ffmpeg + avfoundation)
+
+Produce a watchable `.mp4`/`.mov` that proves an e2e run happened, that a
+reviewer can audit and re-derive, and whose hashes match the raw artifacts it
+renders. This is the fallback-that-is-actually-better when OS screen capture
+is permission-blocked (macOS returns wallpaper-only frames): render the movie
+from the real run's log instead of fighting the OS for pixels.
+
+## Try the real capture first — refuse to fake it
+
+```bash
+# probe capture devices
+/opt/homebrew/bin/ffmpeg -f avfoundation -list_devices true -i ""
+
+# short validation grab, then extract frame 1 and LOOK at it
+/opt/homebrew/bin/ffmpeg -y -hide_banner -f avfoundation -framerate 15 -capture_cursor 1 \
+  -t 2 -i '<screen-index>:none' -vf scale=1280:-2 -pix_fmt yuv420p /tmp/cap-validate.mp4
+/opt/homebrew/bin/ffmpeg -y -hide_banner -i /tmp/cap-validate.mp4 -frames:v 1 /tmp/cap-validate.png
+```
+
+If the frame is just wallpaper (app window missing), Screen Recording is
+blocked for this process. **Do not ship it.** Say so explicitly and switch to
+the rendered evidence reel below. `screencapture -x out.png` has the same
+limitation; `screencapture -x -l <windowID> out.png` can grab a single window
+if you can resolve its CoreGraphics window id.
+
+## Run the real gate as the evidence source
+
+Wrap the actual e2e test/command so the log carries machine-checkable
+markers. Use `bash`, not `zsh` — zsh's read-only `$status` injects a spurious
+error *after* a passing run and pollutes the movie.
+
+```bash
+bash -o pipefail -c '
+  printf "MANUAL_E2E_KIND=<name>\n";
+  printf "STARTED_AT="; date -u +%Y-%m-%dT%H:%M:%SZ;
+  <the real e2e command>;             # e.g. xcodebuild test-without-building ... -resultBundlePath ...
+  rc=$?;
+  printf "FINISHED_AT="; date -u +%Y-%m-%dT%H:%M:%SZ;
+  printf "EXIT_STATUS=%s\n" "$rc"; exit "$rc"
+' 2>&1 | tee <evidence-dir>/run.log
+```
+
+## Snapshot external state before and after
+
+If the run touches a remote host or a shared tmux, snapshot it identically
+pre- and post-run and diff. Equal snapshots prove the run left no residue.
+
+```bash
+ssh <host> 'date -Is; tmux list-sessions -F "#{session_name}|#{session_windows}|attached=#{session_attached}"; \
+  ps -eo pid=,args= | awk "/<helper>/ {print}"; find /tmp -maxdepth 1 -name "<sock-glob>" | wc -l' \
+  | tee <evidence-dir>/pre-snapshot.txt
+# ... run gate ...  then repeat with SNAPSHOT_KIND=post => post-snapshot.txt ; assert they match
+```
+
+## Render the reel from the log
+
+Draw 1920x1080 RGB frames from the log and snapshots (title / exact command
+shape / result / before-after diff / evidence bundle) and stream
+`img.tobytes()` into a single ffmpeg pipe. Keep it in a saved
+`generate_*_movie.py` so it is re-runnable and auditable — don't leave it as a
+one-shot heredoc for anything you'll repeat.
+
+```python
+from PIL import Image, ImageDraw, ImageFont
+import subprocess
+
+W, H, FPS = 1920, 1080, 15
+SANS = ImageFont.truetype('/System/Library/Fonts/Helvetica.ttc', 42)   # macOS system fonts
+MONO = ImageFont.truetype('/System/Library/Fonts/Menlo.ttc', 24)
+
+cmd = [
+    '/opt/homebrew/bin/ffmpeg', '-y', '-hide_banner',
+    '-f', 'rawvideo', '-pix_fmt', 'rgb24', '-s', f'{W}x{H}', '-r', str(FPS), '-i', '-',
+    '-an', '-c:v', 'libx264', '-preset', 'medium', '-crf', '20', '-pix_fmt', 'yuv420p',
+    '-movflags', '+faststart', 'out.mov',
+]
+proc = subprocess.Popen(cmd, stdin=subprocess.PIPE)
+for frame_count, render in scenes:            # scenes = [(nframes, render_fn), ...]
+    denom = max(1, frame_count - 1)
+    for i in range(frame_count):
+        proc.stdin.write(render(i / denom).tobytes())   # render() -> PIL RGB Image, W x H
+proc.stdin.close()
+if proc.wait() != 0:
+    raise SystemExit('ffmpeg failed')
+```
+
+## Verify the encoding with ffprobe
+
+```bash
+/opt/homebrew/bin/ffprobe -v error \
+  -show_entries format=duration,size \
+  -show_entries stream=codec_name,width,height,nb_frames \
+  -of default=noprint_wrappers=1 out.mov
+# expect e.g. codec_name=h264, width=1920, height=1080, real duration/nb_frames
+```
+
+## Extract frames, build a contact sheet, and look at it
+
+```bash
+mkdir -p frame-checks
+for t in 00:00:03 00:00:24 00:00:45 00:01:04; do
+  /opt/homebrew/bin/ffmpeg -y -hide_banner -ss "$t" -i out.mov \
+    -frames:v 1 -update 1 "frame-checks/${t//:/-}.png"
+done
+# PIL: paste the extracted frames (resized) into a 2xN contact-sheet.png, labeled by timestamp
+```
+
+Then actually view `contact-sheet.png` (and any suspect full-size frame) to
+confirm the text is legible. If a panel overflows or a frame is unreadable,
+fix the generator and regenerate — do not ship an unreadable reel.
+
+## Hash the bundle
+
+```bash
+shasum -a 256 out.mov frame-checks/contact-sheet.png run.log > SHA256SUMS
+shasum -a 256 -c SHA256SUMS
+```
+
+If you later fix anything the movie renders (a wrong timestamp, a stale test
+selector, a log line), **regenerate the movie and re-hash**. A hash that no
+longer matches the log is a lie.
+
+## Non-negotiables
+
+- Never present a wallpaper-only or blank capture as evidence. Disclose the
+  OS limitation and render an auditable reel instead — say so plainly; that
+  pivot is the honest outcome, not a fallback to apologize for.
+- The raw log and pre/post snapshots live *next to* the movie. The movie is
+  derived from them, not a substitute for them.
+- `ffprobe` confirms the container is real; the contact sheet plus a human
+  view of it confirms it's legible. Neither alone is sufficient.
+- `SHA256SUMS` covers the movie, the contact sheet, and the log — regenerate
+  it whenever any source artifact changes.
+- Keep the working tree clean: isolate scratch paths, snapshot/clean external
+  state, and don't commit evidence artifacts unless the repo already tracks
+  that kind of evidence.

skills/agentic-end-to-end-testing/rendering-a-demo-movie.md

@@ -0,0 +1,133 @@
+# Rendering a Demo Movie (browser-composited)
+
+Turn a real, running app into a short titled/captioned demo `.mp4` whose
+frames are genuine screenshots of the product — not mockups — and verify the
+output is actually correct before handing it over. Needs a running instance
+of the app, a browser-automation tool that can navigate, run JS (`eval`), set
+a viewport, and screenshot to a path, plus `ffmpeg`/`ffprobe`, and a scratch
+dir such as `/tmp/app-movie/`.
+
+## Step 1 — capture real scene frames from the live app
+
+Set a fixed viewport, then per scene: navigate/interact via JS to compose the
+shot, screenshot to `frame-NN.png`, and **read the PNG back to confirm** the
+shot is what you intended. No fixed fps — one deliberate screenshot per scene
+beat.
+
+```
+use_browser: {"action":"navigate","payload":"http://localhost:<port>/"}
+use_browser: {"action":"screenshot","payload":{"path":"/tmp/app-movie/frame-01.png"}}
+# ...navigate/eval to set up each subsequent scene, screenshot frame-02..frame-NN
+```
+
+## Step 2 — composite title/caption/end cards in the browser
+
+Prefer this over ffmpeg `drawtext`, which is fragile: on macOS-under-sandbox,
+`textfile=` reliably fails with `Either text, a valid file, a timecode or
+text source must be provided` (even with absolute paths), while a trivial
+inline `text=Foo` may work. Don't fight it. Render cards as HTML and
+screenshot them — you also get real fonts, `<b>` accents, and CSS layout for
+free.
+
+`card.html` (param-driven: title / end / image+caption-bar):
+
+```html
+<!doctype html>
+<meta charset="utf-8">
+<style>
+  body { margin:0; width:1400px; height:960px; overflow:hidden;
+         font-family:Georgia,serif; background:#faf8f4; }
+  .frame { width:1400px; height:900px; display:block; }        /* the app screenshot */
+  .bar   { width:1400px; height:60px; background:#2a2722; color:#faf8f4;
+           display:flex; align-items:center; justify-content:center;
+           font-size:26px; letter-spacing:.02em; }             /* caption strip */
+  .bar b { color:#e8b04a; font-weight:normal; }
+  .title { height:960px; display:flex; flex-direction:column;
+           align-items:center; justify-content:center; gap:24px; }
+  .title h1 { font-size:120px; margin:0; color:#b3422f; font-weight:normal; }
+  .title p  { font-size:40px; margin:0; color:#44403a; }
+  .title.dark { background:#2a2722; } .title.dark p { color:#faf8f4; }
+  .title.dark p.accent { color:#b3422f; font-size:30px; }
+</style>
+<body><script>
+  const q = new URLSearchParams(location.search);
+  if (q.get("mode") === "title") {
+    document.body.innerHTML = '<div class="title"><h1>App Name</h1><p>one-line tagline</p></div>';
+  } else if (q.get("mode") === "end") {
+    document.body.innerHTML = '<div class="title dark"><p>deployed to production · <date></p><p class="accent">App Name — org</p></div>';
+  } else {
+    document.body.innerHTML = '<img class="frame" src="' + q.get("img") + '"><div class="bar">' + q.get("cap") + '</div>';
+  }
+</script></body>
+```
+
+Drive it (name cards so a lexical glob orders them title → scenes → end:
+`card-00` … `card-07` … `card-99`):
+
+```
+use_browser: {"action":"set_viewport","payload":{"width":1400,"height":960}}
+use_browser: {"action":"navigate","payload":"file:///tmp/app-movie/card.html?mode=title"}
+use_browser: {"action":"screenshot","payload":{"path":"/tmp/app-movie/card-00.png"}}
+# per scene: define a helper once, then swap innerHTML and screenshot:
+use_browser: {"action":"eval","payload":"window.__setCard=(img,cap)=>{document.body.innerHTML='<img class=\"frame\" src=\"'+img+'\"><div class=\"bar\">'+cap+'</div>';return img;}; __setCard('frame-01.png','The scene resolves — it lands in <b>New state</b>')"}
+use_browser: {"action":"screenshot","payload":{"path":"/tmp/app-movie/card-01.png"}}
+# ...repeat __setCard + screenshot for frame-02..frame-07 -> card-02..card-07
+use_browser: {"action":"navigate","payload":"file:///tmp/app-movie/card.html?mode=end"}
+use_browser: {"action":"screenshot","payload":{"path":"/tmp/app-movie/card-99.png"}}
+```
+
+## Step 3 — concatenate the cards
+
+Pure image concat, no drawtext. `-framerate 1/3` holds each card 3 seconds;
+the `card-*` glob orders them.
+
+```bash
+cd /tmp/app-movie && \
+ffmpeg -y -loglevel error -framerate 1/3 -pattern_type glob -i 'card-*.png' \
+  -vf "scale=1400:960" -r 30 -pix_fmt yuv420p ~/Desktop/app-demo.mp4 && \
+ffprobe -v error -show_entries format=duration -of csv=p=0 ~/Desktop/app-demo.mp4
+# 9 cards -> 27.000000
+```
+
+## Step 4 — verify the artifact (do not skip)
+
+Extract a mid-movie frame and actually look at it; duration/size are
+necessary but not sufficient. This is the step that catches a scene
+screenshotted mid-scroll (half-blank) before it ships.
+
+```bash
+ffmpeg -y -loglevel error -ss 13 -i ~/Desktop/app-demo.mp4 -frames:v 1 /tmp/app-movie/check.png
+# then Read check.png; if a scene is wrong, re-capture just that frame-NN,
+# recompose its card-NN.png, and re-run Step 3.
+```
+
+## If you must use ffmpeg drawtext (failed under sandbox — kept for reference)
+
+This is the approach that **FAILED** under macOS sandbox (`textfile=`
+unreadable). Inline `text=` may still work for short labels; per-scene
+captions letterbox the shot and draw text into the padding:
+
+```bash
+FONT=/System/Library/Fonts/Helvetica.ttc
+# title card (lavfi solid color + two inline drawtext)
+ffmpeg -y -loglevel error -f lavfi -i "color=c=0xfaf8f4:s=1400x960:d=3" \
+  -vf "drawtext=fontfile=$FONT:text='App Name':fontsize=110:fontcolor=0xb3422f:x=(w-text_w)/2:y=360,drawtext=fontfile=$FONT:text='one-line tagline':fontsize=42:fontcolor=0x44403a:x=(w-text_w)/2:y=510" \
+  -r 30 -pix_fmt yuv420p seg-00.mp4
+# a captioned scene: scale to 1400x900, pad 60px dark bar, caption in the bar
+ffmpeg -y -loglevel error -loop 1 -i frame-01.png -t 3 \
+  -vf "scale=1400:900,pad=1400:960:0:0:color=0x2a2722,drawtext=fontfile=$FONT:text='caption text':fontsize=30:fontcolor=0xfaf8f4:x=(w-text_w)/2:y=918" \
+  -r 30 -pix_fmt yuv420p seg-01.mp4
+# concat demuxer
+for f in seg-*.mp4; do echo "file '$f'"; done > list.txt
+ffmpeg -y -loglevel error -f concat -safe 0 -i list.txt -c copy ~/Desktop/app-demo.mp4
+```
+
+## Why the browser-composited path wins
+
+- Real product screenshots as scenes are unfakeable — an honest "show it
+  off."
+- No dependency on ffmpeg font rendering, the flaky part; cards get real
+  fonts, rich markup (`<b>` accents), and CSS layout.
+- Deterministic ordering via zero-padded `card-NN.png` filenames plus glob.
+- The extract-a-frame-and-read-it check in Step 4 is the honesty gate: it is
+  how a bad frame gets caught instead of shipped.