|
|
@@ -32,23 +32,23 @@ Every scenario fails on any pageerror and on the client's connection-loss/gap-re
|
|
|
|
|
|
### Expected outputs
|
|
|
|
|
|
-One committed golden per scenario: a normalized `ariaSnapshot()` of the conversation region (`ui.expected.md`) — uuid/cwd/workspace-basename/duration tokens normalized, captured poll-until-equal at the settled milestone — plus a few role/text anchor assertions that stay green under a semantics-preserving component rewrite while the golden churns reviewably. The aria tree is the mechanization of the client rule "assert what the user would see, never class names". World-state assertions ride root-context session events inline (which tool call produced which durable result, whether `turn/end` completed) instead of a second committed log golden: the persisted-log surface is pinned by the ACP/headless/TUI suites through the same loop and persistence, and re-pinning it here would double refresh cost against the tier discipline. `refresh` is the sole golden writer — a missing golden in replay mode fails with the healing command rather than self-bootstrapping.
|
|
|
+At least one committed golden per scenario, and one per DISTINCT end-state for the interactive scenarios (cancel/error/retry, waiting/answered, mid-steer/settled, panel-open, post-reload): a normalized `ariaSnapshot()` of the scenario's owning region — uuid/cwd/workspace-basename/duration tokens normalized, captured poll-until-equal at the settled milestone — plus a few role/text anchor assertions that stay green under a semantics-preserving component rewrite while the golden churns reviewably. The aria tree is the mechanization of the client rule "assert what the user would see, never class names". World-state assertions ride root-context session events inline (which tool call produced which durable result, whether `turn/end` completed) instead of a second committed log golden: the persisted-log surface is pinned by the ACP/headless/TUI suites through the same loop and persistence, and re-pinning it here would double refresh cost against the tier discipline. `refresh` is the sole golden writer — a missing golden in replay mode fails with the healing command rather than self-bootstrapping.
|
|
|
|
|
|
The typecheck plane split is structural: the three files that boot the host spine (`scaffold`, `replay-round-trip.e2e`, and `seeded-history.e2e`) are excluded from the client-registered `apps/web` project. Those files and their shared `support.ts` are included file-by-file in `tsconfig.host.json` — one program cannot hold both sides of the cordis `Context` merges.
|
|
|
|
|
|
### Modes and fixtures
|
|
|
|
|
|
-`DSH_SNAPSHOT` selects replay (default, keyless), record (with key), or refresh (keyless) as inline spec branches — the TUI shape, not a suite factory: at two scenarios the acp-snapshot factory machinery has no owner, and the genuinely shared parts are already exported (`scrubRequestHeaders`, `parseSessionLog`, `installLlmReplay`). Each spec splits into drive steps (type, send, `whenTurnSettled` — run in all modes, never waiting on model-content selectors, so record cannot hang on a live model answering differently) and assertion steps (replay/refresh only). Record = drive live through the real composer + harvest the in-memory `session.header`/`session.events` (the TUI `rawSessionLog` shape — no file decompression) + `scrubRequestHeaders` + `{{sessionId}}`/`{{cwd}}`/`{{rpcId}}` tokenization; a follow-up keyless refresh regenerates `ui.expected.md`. Every prompting scenario's fixture was recorded against this assembly through this flow. A drift guard ties each spec's drive prompt to the fixture's recorded `user/message`. A fixture-inventory guard holds each scenario directory closed (exact file set, every JSONL a scrub fixed-point with no run-local `rpcId`). Web fixtures scrub headers everywhere and pin no header class, following the TUI precedent over the strict [pinned-header](2026-07-06-pin-request-header-content-in-one-scenario.md) reading — see Deferred.
|
|
|
+`DSH_SNAPSHOT` selects replay (default, keyless), record (with key), or refresh (keyless) as inline spec branches — the TUI shape, not a suite factory: at two scenarios the acp-snapshot factory machinery has no owner, and the genuinely shared parts are already exported (`scrubRequestHeaders`, `parseSessionLog`, `installLlmReplay`). Each spec splits into drive steps (type, send, `whenTurnSettled` — run in all modes, never waiting on model-content selectors, so record cannot hang on a live model answering differently) and assertion steps (replay/refresh only). Record = drive live through the real composer + harvest the in-memory `session.header`/`session.events` (the TUI `rawSessionLog` shape — no file decompression) + `scrubRequestHeaders` + `{{sessionId}}`/`{{cwd}}`/`{{rpcId}}` tokenization; a follow-up keyless refresh regenerates the aria goldens. Every prompting scenario's fixture was recorded against this assembly through this flow. A drift guard ties each spec's drive prompt to the fixture's recorded `user/message`. A fixture-inventory guard holds each scenario directory closed (exact file set, every JSONL a scrub fixed-point with no run-local `rpcId`). Web fixtures scrub headers everywhere and pin no header class, following the TUI precedent over the strict [pinned-header](2026-07-06-pin-request-header-content-in-one-scenario.md) reading — see Deferred.
|
|
|
|
|
|
### Scenarios
|
|
|
|
|
|
1. **`replay-round-trip`** — new session, prompt through the real composer, replay streams reasoning + a `bash` tool call that really executes in the temp workspace + final text (paced 15ms). Asserts settled markdown, the aria golden, and inline world state (the bash call's durable result is exactly `WEB_E2E_OK\n`, completed `turn/end`, >10 chunk events).
|
|
|
2. **`seeded-history`** — a recorded session seeded cold; the sidebar lists it (group row → session row, collapsed by default), opening renders tool cards and text purely from the log through the implicit cold-resume attach inside `session.history` — zero model calls in replay, so no binding constraints; record mode drives the same turn live (real `read` tool against seeded workspace files) to produce the seed.
|
|
|
-3. **`live-interactions`** — one tool-free recorded turn serves three replay-only scenarios through override sidecars whose CONTENT is authored in the spec and minted as a per-run file in a spec-owned temp dir (the derived success entry for the retry append is re-derived from the fixture via `deriveReplayScript`, never copied into a committed sidecar). Cancel: a `{ patches }` `hang` with a `readyFile` marker — the marker's existence proves the stream is parked mid-turn before the test clicks Stop, making mid-stream cancellation deterministic by construction (`turn/end` reason `aborted`, composer re-enabled). AUTH error: a pre-chunk `throw` outside llm-retry's retryable set (`turn/end` reason `error`, zero `llm/retry` events, composer recovers). SERVER retry: `throw` at call 0 + the fixture's own success appended at 1, proving llm-retry end-to-end in the browser via the durable `llm/retry` record (`request/header` logs only on change, so attempt count is invisible there).
|
|
|
-4. **`question-composer`** — the shipped composition's resident `ask_user_question` takeover: a recorded turn blocks mid-step on the real userInteraction seam, the composer (`[data-question-key]`) renders in the browser, the test answers through it (the ONE sanctioned place a drive step reacts to model content: the turn cannot complete without the answer, in record and replay alike), and the tool result carries the chosen label. Golden: the composer's stable waiting state.
|
|
|
-5. **`steering`** — mid-turn steer while the question composer blocks the step (the deterministic mid-turn window; no timing dependence). The composer locks while running, so the steer POSTs `session.prompt` `mode:'steer'` from the page over the same same-origin `/api` wire the client uses (`TODO(web-steer-composer)`: drive a composer gesture once one exists); everything downstream is product — gateway → `Agent.steer` → step-boundary drain → durable `steering/message` → SSE → badged interjection bubble. Record-mode fixture honesty: the recording is rejected unless the live model's final reply obeys an instruction only the steering message carries.
|
|
|
-6. **`navigation-panes`** — one rich two-turn seed (turn 1: bash + two parallel reads in one assistant message; turn 2: a markdown-heavy reply) rendered cold through the seeded-history pattern (zero model calls), serving four surfaces: sidebar search (client-side title filter — asserted only after the durable title lands with the attach baseline, because a cold `SessionSummary` carries no title and search matches the `displayTitle` the user sees; negative query empties the tree, positive narrows, clear restores), the Trajectory tab (turn sections + the step group's tool mix plus the view-area aria golden), the Waterfall tab (span stats + one lane per span — the P-I fold counts a turn-0 prologue span because only assistant/steering nodes carry a turn number, pinned as-is), and the details column (the bash toolview row routes click to openDetails; open/closed is asserted on the frame's `data-details-collapsed` attribute because close collapses the grid column to width 0 without unmounting the subtree).
|
|
|
-7. **`lifecycle-chrome`** — one tiny recorded text turn drives three whole-page concerns. Workspace flow over the real wire: the empty-state hero's first send materializes a real Workspace + Session (the jsdom `workspace-flow.snapshot.ts` suite pins this state machine over the fixture client; this scenario pins it through HTTP RPC + SSE + the gateway), proven durably by the session header's cwd being the create-by-name target `<workspaceRoot>/workspace`, plus the hero waiting-state aria golden. Reload recovery: collapse the sidebar (persisted `dsh.layout.panels`), `page.reload`, and the surface comes back whole from persistence alone — layout collapsed, selection restored (`dsh.sessions.current`), the recorded turn re-rendered from `session.history` with zero model calls (the drained replay cursor makes any stray request fail loud at close). Dark mode: no product control flips the theme yet, so the scenario drives the ThemeService's entire DOM contract — the `body[data-ds-dark-theme]` attribute — and pins the shipped cascade: the alias token flips, a painted surface repaints, and removing the attribute restores the light sample exactly (`TODO(web-theme-gesture)` upgrades to a real settings control); per the scope ruling there is no theme/layout golden (aria is color-blind).
|
|
|
+3. **`live-interactions`** — one tool-free recorded turn serves three replay-only scenarios through override sidecars whose CONTENT is authored in the spec and minted as a per-run file in a spec-owned temp dir (the derived success entry for the retry append is re-derived from the fixture via `deriveReplayScript`, never copied into a committed sidecar). Cancel: a `{ patches }` `hang` with a `readyFile` marker — the marker's existence proves the stream is parked mid-turn before the test clicks Stop, making mid-stream cancellation deterministic by construction (`turn/end` reason `aborted`, composer re-enabled). AUTH error: a pre-chunk `throw` outside llm-retry's retryable set (`turn/end` reason `error`, zero `llm/retry` events, composer recovers). SERVER retry: `throw` at call 0 + the fixture's own success appended at 1, proving llm-retry end-to-end in the browser via the durable `llm/retry` record (`request/header` logs only on change, so attempt count is invisible there). Each scenario pins its terminal surface as a golden: `cancel.expected.md` (frozen `partial`, 已停止 marker), `error-auth.expected.md` (the prompt bubble alone — the committed artifact of the web-error-surface gap, the diff that flips when error rendering lands), `retry.expected.md` (indistinguishable from a clean completion — retries are deliberately invisible in the transcript).
|
|
|
+4. **`question-composer`** — the shipped composition's resident `ask_user_question` takeover: a recorded turn blocks mid-step on the real userInteraction seam, the composer (`[data-question-key]`) renders in the browser, the test answers through it (the ONE sanctioned place a drive step reacts to model content: the turn cannot complete without the answer, in record and replay alike), and the tool result carries the chosen label. Goldens: the composer's stable waiting state (`ui.expected.md`) and the answered transcript (`answered.expected.md` — the question resolved into its tool round trip plus the final reply, takeover gone).
|
|
|
+5. **`steering`** — mid-turn steer while the question composer blocks the step (the deterministic mid-turn window; no timing dependence). The composer locks while running, so the steer POSTs `session.prompt` `mode:'steer'` from the page over the same same-origin `/api` wire the client uses (`TODO(web-steer-composer)`: drive a composer gesture once one exists); everything downstream is product — gateway → `Agent.steer` → step-boundary drain → durable `steering/message` → SSE → badged interjection bubble. Record-mode fixture honesty: the recording is rejected unless the live model's final reply obeys an instruction only the steering message carries. Goldens pin the timing semantics visually: `mid-steer.expected.md` captures the accepted-but-invisible state (the loop drains steering only at the step boundary, so no interjection bubble exists while the question still blocks — if the client ever renders pending steers eagerly, this golden flips first) and `settled.expected.md` the badged bubble plus obeying reply.
|
|
|
+6. **`navigation-panes`** — one rich two-turn seed (turn 1: bash + two parallel reads in one assistant message; turn 2: a markdown-heavy reply) rendered cold through the seeded-history pattern (zero model calls), serving four surfaces: sidebar search (client-side title filter — asserted only after the durable title lands with the attach baseline, because a cold `SessionSummary` carries no title and search matches the `displayTitle` the user sees; negative query empties the tree, positive narrows, clear restores), the Trajectory tab (turn sections + the step group's tool mix plus the view-area aria golden), the Waterfall tab (span stats + one lane per span — the P-I fold counts a turn-0 prologue span because only assistant/steering nodes carry a turn number, pinned as-is), and the details column (the bash toolview row routes click to openDetails; open/closed is asserted on the frame's `data-details-collapsed` attribute because close collapses the grid column to width 0 without unmounting the subtree). Goldens: `trajectory.expected.md` and `waterfall.expected.md` (each tab's view area) plus `details-open.expected.md` (the open panel: tool-name header, Input args, Output result).
|
|
|
+7. **`lifecycle-chrome`** — one tiny recorded text turn drives three whole-page concerns. Workspace flow over the real wire: the empty-state hero's first send materializes a real Workspace + Session (the jsdom `workspace-flow.snapshot.ts` suite pins this state machine over the fixture client; this scenario pins it through HTTP RPC + SSE + the gateway), proven durably by the session header's cwd being the create-by-name target `<workspaceRoot>/workspace`, plus the hero waiting-state aria golden. Reload recovery: collapse the sidebar (persisted `dsh.layout.panels`), `page.reload`, and the surface comes back whole from persistence alone — layout collapsed, selection restored (`dsh.sessions.current`), the recorded turn re-rendered from `session.history` with zero model calls (the drained replay cursor makes any stray request fail loud at close), and `reloaded.expected.md` pins the rebuilt conversation region — rendering the same settled transcript from persistence alone IS the recovery claim. Dark mode: no product control flips the theme yet, so the scenario drives the ThemeService's entire DOM contract — the `body[data-ds-dark-theme]` attribute — and pins the shipped cascade: the alias token flips, a painted surface repaints, and removing the attribute restores the light sample exactly (`TODO(web-theme-gesture)` upgrades to a real settings control); per the scope ruling there is no theme/layout golden (aria is color-blind).
|
|
|
|
|
|
### CI stance
|
|
|
|
Tianyi Cui