Browse Source

test(session): pin deliberate historical corpus refusals

Keep supported generations and native V3 restoration mandatory. Enumerate only chronology-preserving migration refusals by source path, generation and exact typed reason; retain headerless harness protocol examples separately. Verify source bytes remain unchanged and reject stale, widened, or current-generation exceptions.
Tianyi Cui 1 month ago
parent
commit
c09ca68e6f

+ 2 - 2
.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md
-2026-08-31-released-session-format-migrations.md: 0acab910326317201329a3e9e19a012c5ce20f6c
-2026-08-31-released-session-format-migrations.zh.md: 3110d53b7e6cf993f03ecc9f591cb9fee30c6c45
+2026-08-31-released-session-format-migrations.md: ec450c4a39f58551f667ee2012f12c47fd270968
+2026-08-31-released-session-format-migrations.zh.md: 23c677b60ba9d080dd1fcfbb0bfb6a9dc124e72f

+ 2 - 0
.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md

@@ -74,6 +74,8 @@ A source inherited count can be unknown before EOF: V2 derives it from seed mark
 
 All structural changes compose in the one unreleased V2→V3 edge; feature or review order does not allocate extra Session format versions. V0, V1, and V2 generations remain byte-frozen, and migration publishes only the final V3 successor. The unreleased target can evolve until release, but an already-written V3 file does not rerun its incoming migration. Integration tests therefore require isolated disposable homes and unchanged historical inputs rather than rewriting committed generations.
 
+The [committed-corpus inventory](../../../../packages/test-support/llm-replay/tests/session-format-corpus-inventory.ts) identifies deliberately unsupported historical conversions by source path, generation, and exact refusal reason. Retaining those artifacts must not force chronology-changing migration or permit a blanket skip: every listed artifact must still raise the typed migration refusal, and unlisted artifacts must restore. Native current-generation fixtures cannot be classified as unsupported, because they do not traverse an incoming edge. Headerless test-harness protocol examples remain a separate explicit class. The corpus test checks source bytes after both successful and refused restoration; it does not rewrite historical evidence to satisfy the current reader.
+
 ### Physical codecs and packed runs
 
 Each released codec creates a row decoder with explicit `strict` or `recoverable` recovery. The decoder validates and emits one event or one codec-owned `SessionFormatEventRun` at a time through separate context methods. v0-to-v1 and v1-to-v2 implement both `transformEvent()` and `transformRun()`, so packed Assistant chunks can reach the folding edge without first becoming millions of ordinary events.

+ 2 - 0
.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md

@@ -74,6 +74,8 @@ V2→V3 负责[系统提示词结构转换](2026-09-02-system-prompt-as-surface-
 
 所有结构变更组合在唯一且尚未发布的 V2→V3 迁移边中;功能或评审顺序不分配额外 Session 格式版本。V0、V1、V2 代际保持字节冻结,迁移只发布最终 V3 后继代际。未发布的目标可以持续演化至发布,但已经写出的 V3 文件不会重新执行入边迁移。因此,集成测试必须使用隔离、可丢弃的 home 和未变更的历史输入,而非改写已提交代际。
 
+[已提交语料清单](../../../../packages/test-support/llm-replay/tests/session-format-corpus-inventory.ts) 按源路径、代际与精确拒绝原因标识有意不支持的历史转换。保留这些产物不能迫使迁移改变时序,也不能允许统一跳过:每个清单中的产物仍必须抛出类型化迁移拒绝,未列入的产物必须还原。原生当前代际 fixture 不经过入边,因此不能被归为不支持。没有版本 header 的测试框架协议示例保持为独立的显式类别。语料测试在还原成功和拒绝后都检查源字节;它不通过改写历史证据来满足当前 reader。
+
 ### Physical codec 与 packed run
 
 每个 released codec 会用显式 `strict` 或 `recoverable` 策略创建 row decoder。Decoder 每次通过不同的 context 方法校验并 emit 一个 event 或 codec-owned `SessionFormatEventRun`。v0-to-v1 与 v1-to-v2 都实现 `transformEvent()` 和 `transformRun()`,因此 packed Assistant chunk 可以直接到达 folding edge,无需先变成数百万个普通事件。

+ 2 - 2
packages/test-support/llm-replay/README.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write packages/test-support/llm-replay/README.md
-README.md: 2a9e202410ebfd98083f7d21b7d3531b2c4bd0a8
-README.zh.md: c93dff322f6a2708a921f900d39333149cc9d804
+README.md: 43ad5b28e55550658a83f5e517516cce24cae740
+README.zh.md: 0f7cfefea8b37ae7cd9150f023c898b3f1ae5501

+ 2 - 2
packages/test-support/llm-replay/README.md

@@ -97,14 +97,14 @@ This section explains the design of the replay plugin; the observable behavior i
 
 Replay treats the selected projected Session generation as the fixture. One parser completes projected envelopes, validates and migrates the whole artifact through `sessionFormatCatalog`, and returns the current header, inherited cut, and event list as one result. `deriveReplayScript` expands each `assistant/message` or `assistant/attempt` stream in log order, so each durable settlement becomes one `chunks` entry; a non-empty stream without a `finish` chunk is the fingerprint of a thrown `stream()` and must be expressed through an override sidecar. A `compaction/summary` carrying `llmStreamCall: true` and a complete `rawOutput` replays as one canonical successful stream at that event's position. Scripted strings may embed `{{fromRequest:<regex>}}`; at stream time each placeholder resolves against the live request's string leaves, taking the pattern's last match and its first capture group (or the whole match) in place.
 
-The committed-corpus test discovers every versioned `session*.jsonl` under `snapshots/`, `packages/`, and `scripts/snapshots/python-sdk-single-exe/`. Every artifact must restore to the current view through the real catalog. Any refusal fails with the artifact path, so the fixture owner must correct the invalid relationship before replay or comparison.
+The [committed-corpus test](tests/session-format-corpus.spec.ts) restores each versioned `session*.jsonl` under `snapshots/`, `packages/`, and `scripts/snapshots/python-sdk-single-exe/` through the real catalog without changing source bytes. Its [inventory](tests/session-format-corpus-inventory.ts) pins deliberate historical refusals by path, source generation, error type, and exact reason; a refusal that disappears or changes fails. Current-generation artifacts cannot receive an exception. Headerless snapshot-harness protocol examples have a separate explicit exemption. All other restoration errors fail with the artifact path; historical files remain unchanged, and native current fixtures require owner correction.
 
 ### Source map
 
 | File | Role |
 |---|---|
 | [`src/index.ts`](src/index.ts) | Types, fixture derivation, override validation, placeholder resolution, session binding, `installLlmReplay`, and the plugin export |
-| [`tests/session-format-corpus.spec.ts`](tests/session-format-corpus.spec.ts) | Complete committed-generation restoration burn-in; every refusal is a failure |
+| [`tests/session-format-corpus.spec.ts`](tests/session-format-corpus.spec.ts) | Committed-generation restoration and exact historical refusal checks |
 | — | No runtime invariant companion is published; this test-only adapter consumes a fixed replay script; its stream grammar is checked by the LLM companion and fixture derivation tests. |
 
 ### Binding and stream flow

+ 2 - 2
packages/test-support/llm-replay/README.zh.md

@@ -97,14 +97,14 @@ parent agent 委托给进程内 subagent 的场景会为每个 Session 记录一
 
 replay 把选定的投影 Session generation 视为 fixture。一个 parser 补全投影 envelope,通过 `sessionFormatCatalog` 校验并迁移完整产物,再以一个结果返回当前 header、继承 cut 与事件列表。`deriveReplayScript` 按日志顺序展开每个 `assistant/message` 或 `assistant/attempt` stream,因此每个持久 settlement 都成为一条 `chunks` entry;非空 stream 缺少 `finish` chunk 是 `stream()` 抛出异常的 fingerprint,必须通过 override sidecar 表达。携带 `llmStreamCall: true` 与完整 `rawOutput` 的 `compaction/summary` 会在该事件位置 replay 为一条规范成功 stream。脚本字符串可以内嵌 `{{fromRequest:<regex>}}`;stream 输出时每个 placeholder 针对 live request 的 string leaf 解析,取该 pattern 的最后一次 match,用其第一个 capture group(无 capture group 时用整个 match)原位替换。
 
-已提交语料测试会发现 `snapshots/`、`packages/` 与 `scripts/snapshots/python-sdk-single-exe/` 下每个带版本的 `session*.jsonl`。每个产物都必须通过真实 catalog 还原为当前视图。任何拒绝都会携带产物路径并使测试失败,因此 fixture owner 必须在 replay 或比较前修正无效关系。
+[已提交语料测试](tests/session-format-corpus.spec.ts) 通过真实 catalog 还原 `snapshots/`、`packages/` 与 `scripts/snapshots/python-sdk-single-exe/` 下每个带版本的 `session*.jsonl`,且不改变源字节。其[清单](tests/session-format-corpus-inventory.ts) 按路径、源代际、错误类型与精确原因固定有意拒绝的历史转换;拒绝消失或变化都会使测试失败。当前代际产物不能获得例外。没有版本 header 的快照框架协议示例具有独立的显式豁免。其他所有还原错误都携带产物路径并使测试失败;历史文件保持不变,原生当前 fixture 则由 owner 修正。
 
 ### 源码地图
 
 | 文件 | 职责 |
 |---|---|
 | [`src/index.ts`](src/index.ts) | 类型、fixture 派生、override 校验、占位符解析、会话绑定、`installLlmReplay` 与插件导出 |
-| [`tests/session-format-corpus.spec.ts`](tests/session-format-corpus.spec.ts) | 完整已提交 generation restore burn-in;任何拒绝都是失败 |
+| [`tests/session-format-corpus.spec.ts`](tests/session-format-corpus.spec.ts) | 已提交代际还原与精确历史拒绝检查 |
 | — | 不发布运行时不变式伴生入口;该仅测试适配器消费固定的回放脚本;其流语法由 LLM 伴生插件与 fixture 派生测试检验。 |
 
 ### 绑定与流式流程

+ 67 - 0
packages/test-support/llm-replay/tests/session-format-corpus-inventory.ts

@@ -0,0 +1,67 @@
+/** Exact historical conversion refusals; current-generation fixtures never qualify. */
+export const expectedUnsupported: Readonly<Partial<Record<string, { sourceVersion: number; reason: string }>>> = {
+  'packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-architecture-review/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 surface before first step cannot acquire a system head without changing chronology',
+  },
+  'packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-follow-up-builder/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 surface before first step cannot acquire a system head without changing chronology',
+  },
+  'packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-showcase/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 surface before first step cannot acquire a system head without changing chronology',
+  },
+  'packages/test-support/session-snapshot/tests/fixtures/suite/pin-turn/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 changed request prompt outside an open step cannot retain source chronology',
+  },
+  'packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.1.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 changed request prompt outside an open step cannot retain source chronology',
+  },
+  'packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 changed request prompt outside an open step cannot retain source chronology',
+  },
+  'packages/test-support/session-snapshot/tests/fixtures/suite/shared-pin/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 changed request prompt outside an open step cannot retain source chronology',
+  },
+  'snapshots/web/message-feedback-protocol/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 surface before first step cannot acquire a system head without changing chronology',
+  },
+  'snapshots/web/navigation-panes/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 surface before first step cannot acquire a system head without changing chronology',
+  },
+  'snapshots/web/pwsh-terminal/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 surface before first step cannot acquire a system head without changing chronology',
+  },
+  'snapshots/web/schedule-catalog/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 surface before first step cannot acquire a system head without changing chronology',
+  },
+  'snapshots/web/seeded-history/session.v2.jsonl': {
+    sourceVersion: 2,
+    reason: 'session snapshot line 3: format v2 surface before first step cannot acquire a system head without changing chronology',
+  },
+}
+
+/** Headerless snapshot-harness protocol examples, not released Session artifacts. */
+export const unversionedProtocolFixtures = new Set([
+  'packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/session.1.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-skip/session.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/suite/authored-error/session.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/suite/blocked-log/session.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/suite/no-model/session.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/suite/pin-turn/session.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.jsonl',
+  'packages/test-support/session-snapshot/tests/fixtures/suite/shared-pin/session.jsonl',
+])

+ 91 - 12
packages/test-support/llm-replay/tests/session-format-corpus.spec.ts

@@ -1,6 +1,9 @@
 import { readFileSync, readdirSync } from 'node:fs'
 import { join, relative, resolve } from 'node:path'
 import { describe, expect, it } from 'vitest'
+import { SESSION_FORMAT_VERSION } from '@deepseek-ai/dsh-session'
+import { SessionFormatUnsupportedMigrationError } from '@deepseek-ai/dsh-session-format-catalog'
+import { expectedUnsupported, unversionedProtocolFixtures } from './session-format-corpus-inventory.ts'
 import { parseSessionLog } from '../src/index.ts'
 
 const repoRoot = resolve(import.meta.dirname, '../../../..')
@@ -37,21 +40,97 @@ function filenameFormatVersion(path: string): number {
   return match[1] === undefined ? 0 : Number(match[1])
 }
 
+type Unsupported = { sourceVersion: number; reason: string }
+
+function assertRestoration(
+  key: string,
+  sourceVersion: number,
+  restore: () => unknown,
+  unsupported: Unsupported | undefined,
+): void {
+  if (unsupported === undefined) {
+    expect(restore, key + ': current-format restoration').not.toThrow()
+    return
+  }
+  expect(sourceVersion, key + ': current-generation fixtures cannot be unsupported').toBeLessThan(SESSION_FORMAT_VERSION)
+  expect(sourceVersion, key + ': inventoried source generation').toBe(unsupported.sourceVersion)
+  let refusal: unknown
+  try {
+    restore()
+  } catch (error) {
+    refusal = error
+  }
+  expect(refusal, key + ': expected unsupported migration').toBeInstanceOf(SessionFormatUnsupportedMigrationError)
+  expect((refusal as Error).message, key + ': exact refusal reason').toBe(unsupported.reason)
+}
+
+const fixtures = ['snapshots', 'packages', 'scripts/snapshots/python-sdk-single-exe']
+  .flatMap(root => committedSessionFixtures(join(repoRoot, root)))
+  .map(file => ({ file, key: relative(repoRoot, file).split('\\').join('/') }))
+  .sort((a, b) => a.key.localeCompare(b.key))
+
 describe('committed Session format corpus', () => {
-  it('restores every versioned fixture through the current format catalog', () => {
-    const files = ['snapshots', 'packages', 'scripts/snapshots/python-sdk-single-exe']
-      .flatMap(root => committedSessionFixtures(join(repoRoot, root)))
-      .sort()
-
-    for (const file of files) {
-      const source = readFileSync(file, 'utf8')
-      if (!declaresFormat(source)) continue
-      const key = relative(repoRoot, file).split('\\').join('/')
+  it('keeps every exception tied to an existing fixture', () => {
+    const keys = new Set(fixtures.map(({ key }) => key))
+    for (const key of [...Object.keys(expectedUnsupported), ...unversionedProtocolFixtures]) {
+      expect(keys.has(key), key).toBe(true)
+    }
+  })
+
+  it.each(fixtures)('$key', ({ file, key }) => {
+    const bytes = readFileSync(file)
+    const source = bytes.toString('utf8')
+    try {
+      if (unversionedProtocolFixtures.has(key)) {
+        expect(declaresFormat(source), key + ': protocol fixture must remain unversioned').toBe(false)
+        expect(filenameFormatVersion(file), key + ': protocol fixture filename').toBe(0)
+        expect(expectedUnsupported[key], key + ': protocol fixture is not an unsupported migration').toBeUndefined()
+        return
+      }
+      expect(declaresFormat(source), key + ': Session header must declare its format').toBe(true)
       const header = JSON.parse(source.split(/\r?\n/u).find(line => line.trim().length > 0) ?? '{}') as {
-        version?: unknown
+        version: number
       }
-      expect(header.version, `${key}: filename/header Session generation`).toBe(filenameFormatVersion(file))
-      expect(() => parseSessionLog(source), `${key}: current-format restoration`).not.toThrow()
+      expect(header.version, key + ': filename/header Session generation').toBe(filenameFormatVersion(file))
+      assertRestoration(key, header.version, () => parseSessionLog(source), expectedUnsupported[key])
+    } finally {
+      expect(readFileSync(file), key + ': source bytes remain unchanged').toEqual(bytes)
     }
   })
 })
+
+describe('corpus unsupported policy', () => {
+  const refusal = { sourceVersion: 2, reason: 'deliberate historical refusal' }
+  const refused = (): never => { throw new SessionFormatUnsupportedMigrationError(refusal.reason) }
+
+  it('rejects an unlisted migration refusal', () => {
+    expect(() => assertRestoration('unlisted', 2, refused, undefined)).toThrow('current-format restoration')
+  })
+
+  it('rejects an exception that starts restoring successfully', () => {
+    expect(() => assertRestoration('supported', 2, () => [], refusal)).toThrow('expected unsupported migration')
+  })
+
+  it('rejects a changed refusal reason', () => {
+    expect(() => assertRestoration('changed', 2, refused, { ...refusal, reason: 'different' })).toThrow('exact refusal reason')
+  })
+
+  it('rejects corruption masquerading as an unsupported migration', () => {
+    expect(() => assertRestoration('corrupt', 2, () => { throw new Error(refusal.reason) }, refusal))
+      .toThrow('expected unsupported migration')
+  })
+
+  it('rejects a mismatched source generation', () => {
+    expect(() => assertRestoration('wrong-version', 1, refused, refusal)).toThrow('inventoried source generation')
+  })
+
+  it('never exempts a current-generation fixture', () => {
+    expect(() => assertRestoration('current', SESSION_FORMAT_VERSION, refused, {
+      ...refusal, sourceVersion: SESSION_FORMAT_VERSION,
+    })).toThrow('current-generation fixtures cannot be unsupported')
+  })
+
+  it('accepts only the exact typed historical refusal', () => {
+    expect(() => assertRestoration('historical', 2, refused, refusal)).not.toThrow()
+  })
+})