test(movie): keep one portable test suite

The Python suite ported the three shell tests' assertions so they run on
Windows; both copies were kept and the README mapped one to the other. Keep
the portable suite. Drop the one-shot Windows acceptance driver and its
browser fixture, which produced evidence rather than regressions, and the
never-implemented reserved suite names.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

tests/proving-it-works-with-a-movie/README.md

@@ -1,6 +1,6 @@
 # Portable movie regressions
 
-Run an implemented suite from the repository root:
+Run a suite from the repository root:
 
 ```sh
 uv run --script tests/proving-it-works-with-a-movie/run-tests.py --suite assembly
@@ -11,69 +11,13 @@ uv run --script tests/proving-it-works-with-a-movie/run-tests.py --suite all
 
 By default, unavailable external capabilities are reported as skips. Add
 `--require-capabilities` when the selected environment is required to provide
-them; any skip then makes the run fail. Only `shells` and `routes` are reserved
-for later tasks and currently fail explicitly instead of reporting empty success.
+them; any skip then makes the run fail.
 
 The assembly sine wave is only a synthetic timing fixture. The narration drift
 inputs exercise text comparison only. Neither is speech/ASR acceptance.
 
-## Imported assertion equivalence
-
-| Bash suite | Imported assertion | Python test |
-| --- | --- | --- |
-| `test-assemble.sh` | `assemble runs` | `AssemblyRegression.test_narration_padding_and_offsets` |
-| `test-assemble.sh` | `segment = max(narration, visuals)` (8s +/- 0.4s) | `AssemblyRegression.test_narration_padding_and_offsets` |
-| `test-assemble.sh` | `offsets.json places the narrated scene` (2s +/- 0.3s) | `AssemblyRegression.test_narration_padding_and_offsets` |
-| `test-assemble.sh` | `cues start at the scene's real offset, not zero` (first cue start equals the measured body offset within 1ms) | `AssemblyRegression.test_narration_padding_and_offsets` |
-| `test-check-movie.sh` | `front-loaded action is rejected` | `CheckerRegression.test_front_loaded_action_is_rejected` |
-| `test-check-movie.sh` | `paced + subtitles is accepted` | `CheckerRegression.test_paced_with_subtitles_is_accepted` |
-| `test-check-movie.sh` | `narrated without subtitles rejected` | `CheckerRegression.test_narrated_without_subtitles_is_rejected` |
-| `test-check-movie.sh` | `subtitles that stop early rejected` | `CheckerRegression.test_subtitles_that_stop_early_are_rejected` |
-| `test-check-movie.sh` | `subtitle check is opt-outable` | `CheckerRegression.test_subtitle_check_is_opt_outable` |
-| `test-check-movie.sh` | `a still with audio is rejected` | `CheckerRegression.test_still_with_audio_is_rejected` |
-| `test-check-movie.sh` | `missing narration is rejected` | `CheckerRegression.test_missing_narration_is_rejected` |
-| `test-check-movie.sh` | `silent movie passes when unnarrated` | `CheckerRegression.test_silent_movie_passes_when_unnarrated` |
-| `test-check-movie.sh` | `a contact sheet is always written` | `CheckerRegression.test_contact_sheet_is_always_written` |
-| `test-narrate.sh` | `mispronounced jargon passes` | `NarrationDriftRegression.test_mispronounced_jargon_passes` |
-| `test-narrate.sh` | `exact transcript passes` | `NarrationDriftRegression.test_exact_transcript_passes` |
-| `test-narrate.sh` | `a dropped clause fails` | `NarrationDriftRegression.test_dropped_clause_fails` |
-| `test-narrate.sh` | `an invented preamble fails` | `NarrationDriftRegression.test_invented_preamble_fails` |
-| `test-narrate.sh` | `an empty clip fails` | `NarrationDriftRegression.test_empty_clip_fails` |
-
-The portable subtitle assertion intentionally strengthens the Bash suite's
-whole-file regex: it parses the first cue start and compares it with the actual
-assembly offset at SRT's millisecond precision.
-
-### Final native Windows workflow fixture
-
-With native uv, Python 3.12+, FFmpeg/ffprobe (libass), Chrome or Edge, and ttyd
-prepared, run from each invoking shell and record that same shell. Record
-its version/PID before invoking uv; `--shell` alone is not invocation evidence.
-
-```text
-uv run --script tests/proving-it-works-with-a-movie/run-windows-acceptance.py --work "PATH/movie O'Brien λ & [take]" --shell powershell51 --phase prepare
-```
-
-Repeat with `powershell7` or `gitbash` in a separate work directory. The
-fixture clicks a real local browser counter through CDP, captures its states,
-and prepares BOM-bearing UTF-8/CRLF scenes with a card, image, frames, and a
-stereo source-tone movie. Use the native recording-a-terminal.md recipe with
-`fixtures/terminal_app.py`, keeping its command alive across two separate
-control calls/takes. Keep the second take running through `q`, the command's
-successful result, and a readable completion hold.
-
-```text
-uv run --script tests/proving-it-works-with-a-movie/run-windows-acceptance.py --work "PATH/movie O'Brien λ & [take]" --shell powershell51 --phase finish --take-one "PATH/session/take-one" --take-two "PATH/session/take-two"
-```
-
-Pass exported frame directories, not `samples/`. `finish` runs all five
-public tools, fresh local narration and a new-output cached-model repeat,
-then checks each final audio interval against its narration or known source
-tone. Tool stdout/stderr, arguments/statuses, source hashes, the contact
-sheet, and rendered-audio results go under `work/evidence`. Inspect actual
-pixels, hard captions, timing, and sound before declaring acceptance. The
-source tone is a labeled non-speech fixture, not TTS evidence.
-
-Use the native quoting/path recipes in the skill, remove cloud keys only
-from the test process, and exclude `llm` from its PATH. Retain final media
-outside disposable SDD scratch; do not commit large generated artifacts.
+The `processes` and `terminal` suites need native Windows with ttyd and Chrome
+or Edge available; elsewhere they skip. Set `MOVIE_TEST_SHELL` to
+`powershell51`, `powershell7`, or `gitbash` to choose the recorded shell, and
+`MOVIE_TEST_SHELL_EXE` and `MOVIE_TEST_TTYD` to name those executables when
+they are not on PATH.

tests/proving-it-works-with-a-movie/fixtures/browser.html

@@ -1,5 +0,0 @@
-<!doctype html><meta charset="utf-8"><title>Local counter proof</title>
-<style>body{margin:80px;background:#122438;color:#fff;font:36px system-ui}button{font:inherit;padding:20px 40px;background:#ffcf66;border:0;border-radius:12px}output{display:block;font-size:160px;margin:40px 0}small{font-size:26px}</style>
-<h1>Local counter — real clicks</h1><button onclick="count.value=Number(count.value)+1">Add one</button><output id="count">0</output><small>This page runs from a local file. Each click updates its own state.</small>
-<i id="cursor" style="position:fixed;display:none;width:24px;height:24px;border:4px solid #ff4081;border-radius:50%;pointer-events:none;transform:translate(-50%,-50%)"></i>
-<script>document.addEventListener('mousemove',e=>{cursor.style.display='block';cursor.style.left=e.clientX+'px';cursor.style.top=e.clientY+'px'});</script>

tests/proving-it-works-with-a-movie/run-tests.py

@@ -21,18 +21,12 @@ IMPLEMENTED_SUITES = {
     "subtitles": "test_subtitles.py",
     "terminal": "test_terminal.py",
 }
-RESERVED_SUITES = {
-    "shells",
-    "routes",
-}
-
-
 def parse_args() -> argparse.Namespace:
     parser = argparse.ArgumentParser()
     parser.add_argument(
         "--suite",
         required=True,
-        choices=[*IMPLEMENTED_SUITES, *sorted(RESERVED_SUITES), "all"],
+        choices=[*IMPLEMENTED_SUITES, "all"],
     )
     parser.add_argument("--require-capabilities", action="store_true")
     return parser.parse_args()
@@ -40,13 +34,6 @@ def parse_args() -> argparse.Namespace:
 
 def main() -> int:
     args = parse_args()
-    if args.suite in RESERVED_SUITES:
-        print(
-            f"suite {args.suite!r} is reserved but not implemented",
-            file=sys.stderr,
-        )
-        return 2
-
     patterns = (
         list(IMPLEMENTED_SUITES.values())
         if args.suite == "all"

tests/proving-it-works-with-a-movie/run-windows-acceptance.py

@@ -1,221 +0,0 @@
-# /// script
-# requires-python = ">=3.12"
-# dependencies = ["pyyaml", "websocket-client==1.9.0"]
-# ///
-"""Prepare real browser scenes, then finish a movie with two exported terminal takes."""
-import argparse
-import array
-import base64
-import hashlib
-import importlib.util
-import json
-import math
-import os
-from pathlib import Path
-import shutil
-import subprocess
-import sys
-import time
-import wave
-
-from fixtures import duration, load_script, run_tool, _run_ffmpeg
-
-REPO = Path(__file__).resolve().parents[2]
-S = REPO / "skills/proving-it-works-with-a-movie"
-
-
-def write_json(path, value, *, bom=False):
-    path.write_bytes(("\ufeff" if bom else "").encode("utf-8") +
-                     (json.dumps(value, ensure_ascii=False, indent=2) + "\n").replace("\n", "\r\n").encode("utf-8"))
-
-
-def recorder_module():
-    sys.path.insert(0, str(S / "scripts"))
-    spec = importlib.util.spec_from_file_location("acceptance_recorder", S / "examples/film-terminal.py")
-    module = importlib.util.module_from_spec(spec)
-    spec.loader.exec_module(module)
-    return module
-
-
-def capture_browser(work):
-    """Click the local application through the recorder's retained CDP client."""
-    rec = recorder_module()
-    browser = load_script("browser_tools").find_browser(None)
-    if not browser:
-        raise RuntimeError("Chrome or Edge is required")
-    frames = work / "browser"
-    frames.mkdir()
-    shutil.copyfile(Path(__file__).parent / "fixtures/browser.html", work / "browser.html")
-    port = rec.free_port()
-    job = rec.WindowsJob()
-    cdp = None
-    samples = []
-    try:
-        job.spawn([browser, "--headless=new", "--no-first-run", "--no-default-browser-check",
-                   f"--user-data-dir={work / 'browser-profile'}", "--remote-debugging-address=127.0.0.1",
-                   f"--remote-debugging-port={port}", "about:blank"], work, work / "browser.log")
-        deadline = time.monotonic() + 20
-        while time.monotonic() < deadline:
-            try:
-                page = next(p for p in rec.http_json(f"http://127.0.0.1:{port}/json/list") if p["type"] == "page")
-                cdp = rec.CDP(page["webSocketDebuggerUrl"], lambda event: None, work / "browser-cdp.jsonl")
-                break
-            except (OSError, StopIteration):
-                time.sleep(.1)
-        if cdp is None:
-            raise TimeoutError("fixture browser did not start")
-        cdp.call("Emulation.setDeviceMetricsOverride", {"width": 1600, "height": 900, "deviceScaleFactor": 1, "mobile": False})
-        cdp.call("Page.navigate", {"url": (work / "browser.html").resolve().as_uri()})
-        deadline = time.monotonic() + 10
-        while time.monotonic() < deadline:
-            found = cdp.call("Runtime.evaluate", {"expression": "!!document.querySelector('button')", "returnByValue": True})
-            if found["result"].get("value"):
-                break
-            time.sleep(.1)
-        else:
-            raise TimeoutError("fixture page did not load")
-        position = cdp.call("Runtime.evaluate", {"expression": "(()=>{const r=document.querySelector('button').getBoundingClientRect();return {x:r.x+r.width/2,y:r.y+r.height/2}})()", "returnByValue": True})["result"]["value"]
-        for expected in range(3):
-            if expected:
-                cdp.call("Input.dispatchMouseEvent", {"type": "mouseMoved", **position})
-                for event in ("mousePressed", "mouseReleased"):
-                    cdp.call("Input.dispatchMouseEvent", {"type": event, **position, "button": "left", "clickCount": 1})
-            actual = cdp.call("Runtime.evaluate", {"expression": "document.querySelector('output').value", "returnByValue": True})
-            if actual["result"].get("value") != str(expected):
-                raise RuntimeError(f"real click did not update counter: {actual}")
-            for _ in range(7):
-                started = time.monotonic()
-                request = cdp.send("Page.captureScreenshot", {"format": "png"})
-                deadline = started + 2
-                while request not in cdp.responses and time.monotonic() < deadline:
-                    cdp.pump()
-                response = cdp.responses.pop(request, None)
-                ended = time.monotonic()
-                if ended > deadline or not response or "error" in response:
-                    raise TimeoutError("fixture screenshot exceeded two seconds")
-                target = frames / f"frame-{len(samples):08d}.png"
-                target.write_bytes(base64.b64decode(response["result"]["data"]))
-                samples.append({"state": expected, "started": started, "completed": ended, "frame": target.name})
-                time.sleep(max(0, .2 - (time.monotonic() - started)))
-        shutil.copyfile(frames / "frame-00000000.png", work / "still.png")
-        write_json(work / "browser-samples.json", samples)
-    finally:
-        if cdp:
-            cdp.ws.close()
-            cdp.trace.close()
-        job.close()
-    shutil.rmtree(work / "browser-profile")
-
-
-def prepare(work, shell):
-    work.mkdir(parents=True)
-    capture_browser(work)
-    card = work / "source.html"
-    card.write_text('<meta charset="utf-8"><body style="background:#293548;color:white;font:48px system-ui;padding:90px"><h1>Existing movie segment</h1><p>Original 440 Hz reference tone</p><p>This segment keeps its own audio. No speech.</p>', encoding="utf-8")
-    browser = load_script("browser_tools")
-    browser.render_card(card, work / "source.png", browser=browser.find_browser(None), width=1600, height=900)
-    _run_ffmpeg(["-loop", "1", "-i", str(work / "source.png"), "-f", "lavfi", "-i", "sine=frequency=440:duration=2",
-                 "-t", "2", "-r", "30", "-c:v", "libx264", "-pix_fmt", "yuv420p", "-c:a", "aac", "-ac", "2", str(work / "source.mp4")], cwd=work)
-    scenes = {"resolution": {"width": 1600, "height": 900}, "fps": 30, "scenes": [
-        {"id": "title", "kind": "card", "title": "Native Windows proof", "body": shell, "duration": 1,
-         "narration": "This movie shows a local browser and a real terminal running on Windows."},
-        {"id": "still", "kind": "image", "src": "still.png", "duration": 4, "narration": "The counter starts at zero."},
-        {"id": "source", "kind": "movie", "src": "source.mp4", "height": 900},
-        {"id": "browser", "kind": "frames", "src": "browser", "rate": 5, "narration": "Two real clicks change the counter from zero to two."},
-        {"id": "take-one", "kind": "frames", "src": "TAKE_ONE", "rate": 5,
-         "narration": "The terminal shows three colored states. The command stays alive between takes."},
-        {"id": "take-two", "kind": "frames", "src": "TAKE_TWO", "rate": 5,
-         "narration": "The same command receives the exit key and finishes successfully."}]}
-    write_json(work / "scenes.json", scenes, bom=True)
-    write_json(work / "prepare.json", {"shell_selection": shell, "browser": browser.find_browser(None), "source_audio": "440 Hz sine, two seconds, no speech"})
-
-
-def tone(path):
-    with wave.open(str(path)) as f:
-        rate = f.getframerate()
-        values = array.array("h", f.readframes(f.getnframes()))
-    # Use the center second, avoiding AAC edge padding.
-    values = values[rate // 2:rate * 3 // 2]
-    crossings = sum(a <= 0 < b for a, b in zip(values, values[1:]))
-    return {"frequency": crossings * rate / len(values), "rms": math.sqrt(sum(v*v for v in values) / len(values))}
-
-
-def finish(work, one, two):
-    scenes = work / "scenes.json"
-    doc = json.loads(scenes.read_text(encoding="utf-8-sig"))
-    for scene, take in zip(doc["scenes"][-2:], [one, two]):
-        if not take.is_dir() or not list(take.glob("*.png")):
-            raise ValueError(f"exported frame directory required: {take}")
-        scene["src"] = str(take.resolve())
-    write_json(scenes, doc, bom=True)
-    voice, segments = work / "narration", work / "assembly work"
-    cut, movie, subtitles = work / "cut.mp4", work / "movie.mp4", work / "movie.srt"
-    evidence = work / "evidence"
-    evidence.mkdir(exist_ok=True)
-    steps = [("narrate", [str(scenes), str(voice), "--engine", "piper", "--verify", "on"]),
-             ("narrate", [str(scenes), str(work / "cached-model-repeat"), "--engine", "piper", "--verify", "on"]),
-             ("assemble", [str(scenes), str(cut), "--narration", str(voice), "--work", str(segments)]),
-             ("make-subtitles", [str(voice / "manifest.json"), str(subtitles), "--offsets-json", str(segments / "offsets.json")]),
-             ("burn-subtitles", [str(cut), str(subtitles), str(movie)]),
-             ("check-movie", [str(movie), "--out", str(evidence / "checker"), "--json"])]
-    records = []
-    for index, (name, args) in enumerate(steps):
-        start = time.time()
-        result = run_tool(name, args, cwd=evidence)
-        (evidence / f"{index}-{name}.stdout").write_bytes(result.stdout)
-        (evidence / f"{index}-{name}.stderr").write_bytes(result.stderr)
-        records.append({"tool": name, "argv": [shutil.which("uv"), "run", "--script", str(S / "scripts" / name), *args], "exit": result.returncode, "elapsed": time.time() - start})
-        write_json(evidence / "commands.json", records)
-        print(f"{name}: exit {result.returncode}", flush=True)
-        if result.returncode:
-            raise RuntimeError(f"{name} failed; inspect {evidence}")
-    verify_audio(work, doc, segments, movie, evidence)
-    write_json(evidence / "source-hashes.json", {str(p.relative_to(REPO)): hashlib.sha256(p.read_bytes()).hexdigest() for p in [S / "examples/film-terminal.py", *[S / "scripts" / n for n in ["narrate", "assemble", "make-subtitles", "burn-subtitles", "check-movie"]], Path(__file__)]})
-
-
-def verify_audio(work, doc, segments, movie, evidence):
-    narrator = load_script("narrate")
-    checks, clock = [], 0.
-    for scene in doc["scenes"]:
-        length = duration(segments / (scene["id"] + ".mp4"))
-        audio = evidence / (scene["id"] + "-rendered.wav")
-        _run_ffmpeg(["-ss", str(clock), "-i", str(movie), "-t", str(length), "-vn", "-ar", "16000", "-ac", "1", "-c:a", "pcm_s16le", str(audio)], cwd=work)
-        if scene["id"] == "source":
-            reference = evidence / "source-reference.wav"
-            _run_ffmpeg(["-i", str(work / "source.mp4"), "-vn", "-ar", "16000", "-ac", "1", "-c:a", "pcm_s16le", str(reference)], cwd=work)
-            actual, expected = tone(audio), tone(reference)
-            passed = abs(actual["frequency"] - expected["frequency"]) < 2 and .9 < actual["rms"] / expected["rms"] < 1.1
-            check = {"source_reference": expected, "rendered": actual, "passed": passed}
-        else:
-            heard = narrator.transcribe_local(audio)
-            if heard is None:
-                raise RuntimeError(f"rendered ASR unavailable for {scene['id']}")
-            drift, worst = narrator.structural_drift(scene["narration"], heard)
-            check = {"script": scene["narration"], "heard": heard, "drift": drift, "worst": worst, "passed": drift <= .15 and worst <= 3}
-        checks.append({"scene": scene["id"], "start": clock, "duration": length, **check})
-        write_json(evidence / "rendered-audio.json", checks)
-        if not check["passed"]:
-            raise RuntimeError(f"rendered audio differs for {scene['id']}: {check}")
-        clock += length
-
-
-def main():
-    parser = argparse.ArgumentParser(description=__doc__)
-    parser.add_argument("--work", type=Path, required=True)
-    parser.add_argument("--shell", choices=["powershell51", "powershell7", "gitbash"], required=True)
-    parser.add_argument("--phase", choices=["prepare", "finish"], required=True)
-    parser.add_argument("--take-one", type=Path)
-    parser.add_argument("--take-two", type=Path)
-    args = parser.parse_args()
-    if sys.platform != "win32":
-        parser.error("run this native Windows acceptance fixture on Windows")
-    if args.phase == "prepare":
-        prepare(args.work.resolve(), args.shell)
-    else:
-        if args.take_one is None or args.take_two is None:
-            parser.error("finish requires --take-one and --take-two exported frame directories")
-        finish(args.work.resolve(), args.take_one, args.take_two)
-
-
-if __name__ == "__main__":
-    main()

tests/proving-it-works-with-a-movie/test-assemble.sh

@@ -1,88 +0,0 @@
-#!/usr/bin/env bash
-# Regression tests for scripts/assemble and scripts/make-subtitles.
-#
-# The property that matters: a segment lasts max(narration, visuals), and
-# the scene offsets written for the subtitler match where scenes actually
-# start in the finished cut. Hand-computed offsets silently mistime every
-# cue after an inserted scene, which is why assemble emits them.
-#
-# Usage: tests/proving-it-works-with-a-movie/test-assemble.sh
-set -uo pipefail
-
-HERE="$(cd "$(dirname "$0")" && pwd)"
-SCRIPTS="$HERE/../../skills/proving-it-works-with-a-movie/scripts"
-WORK="$(mktemp -d)"
-trap 'rm -rf "$WORK"' EXIT
-pass=0; fail=0
-
-for tool in ffmpeg ffprobe uv; do
-  command -v "$tool" >/dev/null || { echo "SKIP: $tool not on PATH"; exit 0; }
-done
-
-ok() { echo "ok    $1"; pass=$((pass + 1)); }
-no() { echo "FAIL  $1"; fail=$((fail + 1)); }
-dur() { ffprobe -v error -show_entries format=duration -of csv=p=0 "$1"; }
-about() {  # about <actual> <expected> <tolerance>
-  awk -v a="$1" -v b="$2" -v t="$3" 'BEGIN{exit !(a-b<t && b-a<t)}'
-}
-
-mkdir -p "$WORK/shots"
-for i in 1 2 3 4; do
-  ffmpeg -nostdin -y -v error -f lavfi \
-    -i "color=c=0x${i}0${i}0${i}0:size=320x180:d=0.1" -frames:v 1 \
-    "$WORK/shots/s0$i.png"
-done
-# a 6-second narration stand-in, so the 4s of pictures must be padded to it
-mkdir -p "$WORK/narration"
-ffmpeg -nostdin -y -v error -f lavfi -i "sine=frequency=300:duration=6" \
-  "$WORK/narration/body.wav"
-python3 - "$WORK" <<'PY'
-import json, sys, subprocess
-w = sys.argv[1]
-d = float(subprocess.run(["ffprobe","-v","error","-show_entries","format=duration",
-    "-of","csv=p=0",f"{w}/narration/body.wav"],capture_output=True,text=True).stdout)
-json.dump([{"id":"body","text":"one two three four five six seven eight nine ten",
-            "wav":"body.wav","duration":d}], open(f"{w}/narration/manifest.json","w"))
-PY
-
-cat > "$WORK/scenes.yaml" <<'YAML'
-resolution: { width: 640, height: 360 }
-fps: 30
-scenes:
-  - id: opener
-    kind: image
-    src: shots/s01.png
-    duration: 2
-  - id: body
-    kind: frames
-    src: shots
-    rate: 1.0
-YAML
-
-out="$WORK/out.mp4"
-if "$SCRIPTS/assemble" "$WORK/scenes.yaml" "$out" >"$WORK/log" 2>&1; then
-  ok "assemble runs"
-else
-  no "assemble runs"; sed 's/^/      /' "$WORK/log"
-fi
-
-# opener 2s + body max(4s pictures, 6s narration) = 8s
-total="$(dur "$out")"
-if about "$total" 8 0.4; then ok "segment = max(narration, visuals)"
-else no "segment = max(narration, visuals): got ${total}s, wanted ~8"; fi
-
-offset="$(python3 -c "import json;print(json.load(open('$WORK/segments/offsets.json'))['body'])" 2>/dev/null)"
-if about "${offset:-0}" 2 0.3; then ok "offsets.json places the narrated scene"
-else no "offsets.json places the narrated scene: got ${offset:-none}, wanted ~2"; fi
-
-if "$SCRIPTS/make-subtitles" "$WORK/narration/manifest.json" "$WORK/out.srt" \
-     --offsets-json "$WORK/segments/offsets.json" >/dev/null 2>&1 \
-   && grep -q "00:00:0[2-9]" "$WORK/out.srt"; then
-  ok "cues start at the scene's real offset, not zero"
-else
-  no "cues start at the scene's real offset, not zero"
-fi
-
-echo
-echo "$pass passed, $fail failed"
-[ "$fail" -eq 0 ]

tests/proving-it-works-with-a-movie/test-check-movie.sh

@@ -1,105 +0,0 @@
-#!/usr/bin/env bash
-# Regression tests for scripts/check-movie.
-#
-# Synthesizes movies with known defects using ffmpeg's lavfi sources - no
-# fixtures committed, nothing downloaded - and asserts the checker's verdict
-# on each. The front-loaded case reproduces the real failure this skill
-# exists to prevent: a movie whose action finishes in the first seconds
-# while narration keeps talking over a frozen picture.
-#
-# Usage: tests/proving-it-works-with-a-movie/test-check-movie.sh
-set -uo pipefail
-
-HERE="$(cd "$(dirname "$0")" && pwd)"
-CHECKER="$HERE/../../skills/proving-it-works-with-a-movie/scripts/check-movie"
-WORK="$(mktemp -d)"
-trap 'rm -rf "$WORK"' EXIT
-
-pass=0
-fail=0
-
-for tool in ffmpeg ffprobe uv; do
-  command -v "$tool" >/dev/null || { echo "SKIP: $tool not on PATH"; exit 0; }
-done
-[ -x "$CHECKER" ] || { echo "FAIL: $CHECKER is not executable"; exit 1; }
-
-# --- fixtures -------------------------------------------------------------
-# action for 2s, then a frozen picture for 20s, narration (tone) throughout
-ffmpeg -nostdin -y -v error \
-  -f lavfi -i "testsrc2=size=320x240:rate=10:d=2" \
-  -f lavfi -i "color=c=navy:size=320x240:rate=10:d=20" \
-  -f lavfi -i "sine=frequency=300:duration=22" \
-  -filter_complex "[0:v][1:v]concat=n=2:v=1:a=0[v]" \
-  -map "[v]" -map 2:a -c:v libx264 -pix_fmt yuv420p -c:a aac -shortest \
-  "$WORK/front-loaded.mp4"
-
-# picture changing throughout, narration throughout
-ffmpeg -nostdin -y -v error \
-  -f lavfi -i "testsrc2=size=320x240:rate=10:d=22" \
-  -f lavfi -i "sine=frequency=300:duration=22" \
-  -c:v libx264 -pix_fmt yuv420p -c:a aac -shortest "$WORK/paced.mp4"
-
-# one static frame for the whole runtime, narration throughout
-ffmpeg -nostdin -y -v error \
-  -f lavfi -i "color=c=navy:size=320x240:rate=10:d=12" \
-  -f lavfi -i "sine=frequency=300:duration=12" \
-  -c:v libx264 -pix_fmt yuv420p -c:a aac -shortest "$WORK/still.mp4"
-
-# motion, but no audio track at all
-ffmpeg -nostdin -y -v error \
-  -f lavfi -i "testsrc2=size=320x240:rate=10:d=12" \
-  -c:v libx264 -pix_fmt yuv420p "$WORK/silent.mp4"
-
-# subtitles: one covering the whole runtime, one that gives up early
-cat > "$WORK/paced.srt" <<'SRT'
-1
-00:00:00,000 --> 00:00:07,000
-A narrated movie needs subtitles:
-plenty of people watch muted.
-
-2
-00:00:07,000 --> 00:00:14,000
-The checker treats their absence
-as a defect, not a nicety.
-
-3
-00:00:14,000 --> 00:00:21,500
-And it notices when they stop
-before the narration does.
-SRT
-head -8 "$WORK/paced.srt" > "$WORK/short.srt"
-cp "$WORK/paced.mp4" "$WORK/short.mp4"
-
-# --- assertions -----------------------------------------------------------
-check() {  # check <label> <expected-exit> <must-contain> <movie> [extra args...]
-  local label="$1" want="$2" needle="$3" movie="$4"; shift 4
-  local out rc
-  out="$("$CHECKER" "$movie" --out "$WORK/$(basename "$movie" .mp4)-check" "$@" 2>&1)"
-  rc=$?
-  if [ "$rc" -ne "$want" ]; then
-    echo "FAIL  $label: exit $rc, wanted $want"
-    echo "$out" | sed 's/^/      /'
-    fail=$((fail + 1)); return
-  fi
-  if ! printf '%s' "$out" | grep -qi -- "$needle"; then
-    echo "FAIL  $label: output missing '$needle'"
-    echo "$out" | sed 's/^/      /'
-    fail=$((fail + 1)); return
-  fi
-  echo "ok    $label"
-  pass=$((pass + 1))
-}
-
-check "front-loaded action is rejected"       1 "every visible change happens in the first" "$WORK/front-loaded.mp4"
-check "paced + subtitles is accepted"         0 "Mechanical checks pass"                    "$WORK/paced.mp4"
-check "narrated without subtitles rejected"   1 "no subtitles"                              "$WORK/paced.mp4" --subs "$WORK/nope.srt"
-check "subtitles that stop early rejected"    1 "subtitles stop at"                         "$WORK/short.mp4"
-check "subtitle check is opt-outable"         0 "Mechanical checks pass"                    "$WORK/paced.mp4" --subs "$WORK/nope.srt" --no-expect-subtitles
-check "a still with audio is rejected"        1 "never reaches a new state"                 "$WORK/still.mp4"
-check "missing narration is rejected"         1 "no audio stream"                           "$WORK/silent.mp4"
-check "silent movie passes when unnarrated"   0 "Mechanical checks pass"                    "$WORK/silent.mp4" --no-expect-audio
-check "a contact sheet is always written"     0 "contact-sheet.png"                         "$WORK/paced.mp4"
-
-echo
-echo "$pass passed, $fail failed"
-[ "$fail" -eq 0 ]

tests/proving-it-works-with-a-movie/test-narrate.sh

@@ -1,54 +0,0 @@
-#!/usr/bin/env bash
-# Regression tests for the narration gate's drift rule.
-#
-# The rule has to survive a real asymmetry: an ASR mangles unusual names, so
-# exact word-matching produces false alarms — and, worse, a *dropped* word
-# scores as more similar than two mispronounced ones. So the gate measures
-# missing/invented CONTENT, and these tests pin that distinction.
-#
-# Usage: tests/proving-it-works-with-a-movie/test-narrate.sh
-set -uo pipefail
-
-HERE="$(cd "$(dirname "$0")" && pwd)"
-NARRATE="$HERE/../../skills/proving-it-works-with-a-movie/scripts/narrate"
-WORK="$(mktemp -d)"
-trap 'rm -rf "$WORK"' EXIT
-pass=0; fail=0
-
-command -v uv >/dev/null || { echo "SKIP: uv not on PATH"; exit 0; }
-
-drift() {  # drift <label> <expect-exit> <script> <heard>
-  local label="$1" want="$2"
-  printf '%s' "$3" > "$WORK/script.txt"
-  printf '%s' "$4" > "$WORK/heard.txt"
-  local out rc
-  out="$("$NARRATE" --drift-check "$WORK/script.txt" "$WORK/heard.txt" 2>&1)"; rc=$?
-  if [ "$rc" -eq "$want" ]; then
-    echo "ok    $label  ($out)"; pass=$((pass + 1))
-  else
-    echo "FAIL  $label: exit $rc wanted $want ($out)"; fail=$((fail + 1))
-  fi
-}
-
-SCRIPT="This is smevals studio. Every eval on the shelf is a folder of tasks and graders."
-
-# the ASR mangles jargon on a perfectly good clip - must NOT fail
-drift "mispronounced jargon passes" 0 "$SCRIPT" \
-  "This is Mevil studio. Every Yvel on the shelf is a folder of tasks and graders."
-
-# identical - must pass
-drift "exact transcript passes" 0 "$SCRIPT" "$SCRIPT"
-
-# the voice skipped a whole clause - must fail
-drift "a dropped clause fails" 1 "$SCRIPT" "This is smevals studio."
-
-# a chat model prepended a preamble - must fail
-drift "an invented preamble fails" 1 "$SCRIPT" \
-  "Sure, here it is, happy to help with that. This is smevals studio. Every eval on the shelf is a folder of tasks and graders."
-
-# a silent/garbage clip - must fail
-drift "an empty clip fails" 1 "$SCRIPT" "you"
-
-echo
-echo "$pass passed, $fail failed"
-[ "$fail" -eq 0 ]