Quellcode durchsuchen

chore(web-e2e): gate fixes — catalog, budgets, knip

Regenerate config-catalog for the llm-replay paceMs row; condense the
testing.md web-lane entry to pointer form and raise its ceiling 1020->1060
(the two-sentence tier entry for a genuinely new surface does not fit the
old ceiling after relocation-first trims); internalize two harness helpers
knip flagged (rawSessionLog/normalizeAria are module-internal).
Tianyi Cui vor 1 Monat
Ursprung
Commit
d08050ab69

+ 2 - 7
apps/web/tests/harness.ts

@@ -230,10 +230,8 @@ export async function launchWebHarness(options: LaunchOptions = {}): Promise<Web
  * Serialize a live session back to raw session-JSONL (header + events) — the
  * in-memory record-mode harvest, so the on-disk zstd default never matters.
  * Mirrors the TUI suite's rawSessionLog.
- * @param session - the live session to serialize.
- * @returns raw JSONL text ending in one newline.
  */
-export function rawSessionLog(session: Session): string {
+function rawSessionLog(session: Session): string {
   return [
     JSON.stringify({ type: 'session', ...session.header }),
     ...session.events.map(event => JSON.stringify(event)),
@@ -333,11 +331,8 @@ export async function seedSession(harness: WebHarness, fixtureText: string, id:
 /**
  * Normalize an aria snapshot: uuid, cwd, workspace-basename, and duration
  * volatility collapse to stable tokens.
- * @param snapshot - raw ariaSnapshot text.
- * @param workspaceCwd - the harness workspace (basename doubles as the header breadcrumb).
- * @returns tokenized snapshot text.
  */
-export function normalizeAria(snapshot: string, workspaceCwd: string): string {
+function normalizeAria(snapshot: string, workspaceCwd: string): string {
   // The header breadcrumb renders the workspace's basename, not the full
   // path, so both spellings must collapse to the token.
   const base = workspaceCwd.split('/').pop()!

+ 1 - 1
docs/config-catalog.md

@@ -643,7 +643,7 @@ export interface ReplayModelConfig {
 }
 ```
 
-Source: [`packages/support/llm-replay/src/index.ts:453`](../packages/support/llm-replay/src/index.ts)
+Source: [`packages/support/llm-replay/src/index.ts:454`](../packages/support/llm-replay/src/index.ts)
 
 ## `@deepseek-ai/dsh-llm-retry`
 

+ 2 - 2
docs/testing.md

@@ -8,7 +8,7 @@ How this repo tests, tier by tier, and the rules that keep a green suite meaning
 - **Coverage gate** (`pnpm run test:coverage`): the gating run, per-file 100% on `packages/*/*/src`. An uncovered line is often dead code the gate is correctly flagging for deletion, not a missing test to bolt on. Line coverage is necessary, never sufficient — it proves lines ran, not that the feature works as shipped.
 - **Real-API e2e** (`pnpm run test:e2e`): with-key tests against live provider APIs — the DeepSeek model plus provider-specific smokes that gate on their own keys (`EXA_API_KEY`, `PERPLEXITY_API_KEY`, …); each suite self-skips without its key so keyless CI stays green ([real-API e2e Agent Note](../.agents/notes/implemented/testing/2026-06-19-real-api-e2e-ci.md)).
 - **Snapshot** (`pnpm run test:snapshot`): transport-specific keyless expected outputs cover external presentation. ACP suites boot the real example subprocess, replay a recorded session, and diff normalized JSON-RPC plus the re-persisted log ([ACP snapshot Agent Note](../.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md)); the headless suite independently pins `stream-json` through its real one-shot subprocess. TUI completed journeys replay recorded primary/child JSONL through the real agent loop and tools before projecting ANSI into semantic terminal-state expected outputs; package-local snapshots retain transient renderer states, and a real PTY conversation covers the process boundary ([TUI snapshot Agent Note](../.agents/notes/implemented/testing/2026-07-18-tui-terminal-state-snapshots.md)). Use `pnpm run test:snapshot:record` when a model transcript must change and `pnpm run test:snapshot:refresh` when committed replay input remains correct; review every JSONL and expected-output diff. System-prompt/tool-schema content is pinned by one ACP scenario (`text-turn`) and tokenized in every other fixture, so a prompt or schema edit churns one committed line ([pinned-header Agent Note](../.agents/notes/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)).
-- **Web browser snapshot** (inside `pnpm run test:web`, gate-exempt like the rest of that lane): the web GUI's keyless assembled-transcript tier — a real chromium over the real in-process web assembly (`llm: false` + `dsh-llm-replay`), scenario fixtures and normalized conversation aria goldens under `apps/web/tests/snapshots/`. Its record/refresh commands diverge from `test:snapshot` (`DSH_SNAPSHOT=record pnpm run test:web` re-records against the live model; `DSH_SNAPSHOT=refresh` rewrites goldens keylessly); the [web e2e lane Agent Note](../.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md) owns the design, and CI browser provisioning is deferred there.
+- **Web browser snapshot** (gate-exempt `pnpm run test:web`): real chromium over the in-process web assembly replays recorded fixtures against conversation aria goldens (`apps/web/tests/snapshots/`); `DSH_SNAPSHOT=record`/`refresh` semantics and the deferred CI browser decision live in the [web e2e lane Agent Note](../.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md).
 
 ## The with-key policy: inference is cheap here
 
@@ -42,4 +42,4 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword
 
 ## When a snapshot test is required
 
-Every non-trivial model- or human-visible change adds or updates a keyless scenario in the same PR through a runnable example's owning snapshot suite. Package tests, e2e assertions, mock/test-only compositions, and PR rationale do not replace the assembled transcript; extend the harness when needed. ACP surfaces use `examples/<name>/tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the `stream-json` snapshot and replay fixtures. Completed interactive-terminal journeys use JSONL-driven scenarios under `examples/tui-agent/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. Browser-rendered web GUI journeys use `apps/web/tests/snapshots/` through the web e2e lane's harness. New capability seams, lifecycle shapes, or transcript surfaces name every coverage tier at plan time and verify the harness can express it before implementation.
+Every non-trivial model- or human-visible change adds or updates a keyless scenario in the same PR through a runnable example's owning snapshot suite. Package tests, e2e assertions, mock/test-only compositions, and PR rationale do not replace the assembled transcript; extend the harness when needed. ACP surfaces use `examples/<name>/tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the `stream-json` snapshot and replay fixtures. Completed interactive-terminal journeys use JSONL-driven scenarios under `examples/tui-agent/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. Browser-rendered web GUI journeys use `apps/web/tests/snapshots/`. New capability seams, lifecycle shapes, or transcript surfaces name every coverage tier at plan time and verify the harness can express it before implementation.

+ 26 - 1
packages/support/llm-replay/tests/llm-replay.spec.ts

@@ -476,6 +476,31 @@ describe('installLlmReplay (through the real LlmService)', () => {
     expect(() => { handle.assertConsumed() }).not.toThrow()
   })
 
+  it('paces a throw-entry prefix too (the recorded partial streams at the same cadence)', async () => {
+    writeFileSync(file, sessionJsonl([]), 'utf8')
+    const overrideFile = join(dir, 'replay.override.json')
+    const partial: StreamChunk[] = [{ type: 'block-start', index: 0, blockType: 'text' }]
+    writeFileSync(overrideFile, JSON.stringify([
+      { kind: 'throw', chunks: partial, message: 'boom', code: 'STREAM_CLOSED' },
+    ]), 'utf8')
+    const ctx = new Context()
+    await ctx.plugin(LlmService)
+    installLlmReplay(ctx, { file, overrideFile, paceMs: 10 })
+    const started = performance.now()
+    await expect(drain(ctx.llm.stream({ provider: 'm', model: 'm', messages: [] }))).rejects.toThrow('boom')
+    expect(performance.now() - started).toBeGreaterThanOrEqual(5)
+  })
+
+  it('assertConsumed names an underrunning identified session by its id', async () => {
+    writeLog(TEXT_CHUNKS, TEXT_CHUNKS)
+    const ctx = new Context()
+    await ctx.plugin(LlmService)
+    const handle = installLlmReplay(ctx, { file })
+    const sessionId = 'live-underrun' as NonNullable<GenerateOptions['sessionId']>
+    await drain(ctx.llm.stream({ provider: 'm', model: 'm', messages: [], sessionId }))
+    expect(() => { handle.assertConsumed() }).toThrow(/session live-underrun consumed 1\/2/)
+  })
+
   it('assertConsumed reports recorded scripts no live session ever bound', async () => {
     writeLog(TEXT_CHUNKS)
     const childFile = join(dir, 'session.1.jsonl')
@@ -690,7 +715,7 @@ describe('apply (the plugin entry)', () => {
     writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c))), 'utf8')
     const ctx = new Context()
     await ctx.plugin(LlmService)
-    apply(ctx, { file, providers: [{ id: 'm', models: [{ id: 'm' }] }] })
+    apply(ctx, { file, providers: [{ id: 'm', models: [{ id: 'm' }] }], paceMs: 1 })
     expect(ctx.llm.listProviders()).toEqual([{ id: 'm', name: 'm' }])
     expect(await drain(ctx.llm.stream({ provider: 'm', model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS)
   })

+ 1 - 1
scripts/doc-budgets.manifest.json

@@ -4,7 +4,7 @@
   "docs/architecture.md": 1800,
   "docs/cordis-primer.md": 600,
   "docs/defensive-patterns.md": 550,
-  "docs/testing.md": 1020,
+  "docs/testing.md": 1060,
   "examples/AGENTS.md": 310,
   "packages/AGENTS.md": 660,
   "packages/README.md": 760