Просмотр исходного кода

test(snapshot): retire vanished generation roles

Tianyi Cui 3 недель назад
Родитель
Сommit
eb56627f36

+ 2 - 2
.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md
-2026-08-31-released-session-format-migrations.md: 5fcc3d7d99abfb8e1c0b96e2056a1a13b95984c9
-2026-08-31-released-session-format-migrations.zh.md: fe5bb82bce5c838ef30675df1a35dec96ca73bb3
+2026-08-31-released-session-format-migrations.md: 9ea008cf9edf06378ef39d4f14701247bbce0c93
+2026-08-31-released-session-format-migrations.zh.md: 378eebd8c8de5ceb3aa75b0d90ae3ac0c4adfa16

+ 3 - 3
.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md

@@ -16,7 +16,7 @@ Migration must retain the exact source path, bytes, and inode, including a torn
 
 Each edge freezes strict source and target semantics, while its target physical codec remains vocabulary-neutral so ordinary event growth can stay within one format version. The catalog restores the final generation through the installed peer `@deepseek-ai/dsh-session` and its current `KNOWN_SESSION_EVENT_TYPES`, preventing a frozen historical edge from becoming the current vocabulary owner.
 
-Every event-body operation crosses the persistence coordinator's per-Session serialization chain and completes provider-owned ensure-current work before current values escape. JSONL fuses highest-generation resolution, classification or migration, and current decoding over one physical snapshot; fallback backends retain separate `ensureCurrent` and current-read hooks. The six public reads are prepare, load, inspect, borrowSession, readFrom, and readRaw; cold append adoption uses the same body path. Header-only list and listSnapshots never migrate: they rescan each Session directory and return one descriptor for its numerically highest canonical generation, so a future, unsupported, or malformed highest file remains visible instead of silently falling back.
+Every event-body operation crosses the persistence coordinator's per-Session serialization chain and completes provider-owned ensure-current work before current values escape. JSONL fuses highest-generation resolution, classification or migration, and current decoding over one physical snapshot; fallback backends retain separate `ensureCurrent` and current-read hooks. The six public reads are prepare, load, inspect, borrowSession, readFrom, and readRaw; cold append adoption uses the same body path. Header-only list and listSnapshots never migrate: they rescan each Session directory and return one descriptor for its numerically highest canonical generation, so a future, unsupported, or malformed highest file remains visible instead of silently falling back. When the backend can recover the Session id from its location without trusting the header, the descriptor carries that `storageId`; explicit identity reservations therefore reject unreadable artifacts that already occupy the id.
 
 For `prepare`, `inspect`, and `borrowSession`, cancellation belongs to the observing call rather than shared preparation or migration. A cancelled observer stops waiting, while already-started work may finish for another inspector or later resume; durable publication is never rolled back to satisfy observer cancellation. Detached `readFrom` and `readRaw` operations instead pass cancellation into their serialized backend read.
 
@@ -24,7 +24,7 @@ The configured JSONL encoding owns one full suffix, `.jsonl` or `.jsonl.zstd`. M
 
 Canonical filenames encode the physical format generation: v0 is `session.jsonl` or `session.jsonl.zstd`; every positive generation is lowercase `session.vN.jsonl` or `session.vN.jsonl.zstd`. Publication never renames, replaces, or deletes a committed generation path. If the target already exists, it is accepted only as a regular current-format file with exactly the expected bytes; any other target refuses. Lower generations remain for operator inspection or explicit copying, but normal runtime operations select the numerically highest canonical name and never use retained predecessors as automatic fallback, restore, or downgrade support.
 
-The current-format fast path classifies the header from one stable source snapshot, invokes no historical converter or generation write, and passes that snapshot to current decoding without another file read. A validated current selection is cached for later opens in the same backend instance under the one-writer assumption, while listing deliberately rescans. Multiple edges leave the original generation unchanged and publish only the final target; intermediate versions exist only in memory. Same-process operations are serialized, and a source fingerprint recheck restarts the complete attempt when content changes. Cross-process writer fencing remains outside this guarantee.
+The current-format fast path classifies the header from one stable source snapshot, invokes no historical converter or generation write, and passes that snapshot to current decoding without another file read. A validated current selection is cached for later opens in the same backend instance under the one-writer assumption, while listing deliberately rescans; if the selected file disappears, the backend invalidates the cache and resolves the directory again. Multiple edges leave the original generation unchanged and publish only the final target; intermediate versions exist only in memory. Same-process operations are serialized, and a source fingerprint recheck restarts the complete attempt when content changes. Cross-process writer fencing remains outside this guarantee.
 
 The first edge, `@deepseek-ai/dsh-session-format-v0-to-v1`, is intentionally identity-shaped: aside from the version and bounded historical normalizations already accepted by v0, it preserves logical headers, events, sequence numbers, references, timestamps, payloads, and the configured compression choice. The exact `session.jsonl[.zstd]` source remains byte- and inode-identical, while the current writer encodes the new `session.v1.jsonl[.zstd]` successor. This exercises the complete publication lifecycle before a cardinality-changing format needs it.
 
@@ -40,7 +40,7 @@ This note supersedes the continue-only persistence rule and the deferred-chain s
 
 ## Verification
 
-Release verification ran the committed Session-format corpus gate over 152 versioned persisted-or-projected `session*.jsonl` fixtures under `snapshots/`, `packages/`, and `scripts/snapshots/python-sdk-single-exe/`. Fixture-only omitted envelopes and request-header tokens are materialized before the real static catalog; 150 fixtures reached the current v1 view through current restoration or historical migration. Released-v0 replay inputs remain suffixless, while fresh v1 writer outputs use `session.v1.jsonl` for a parent and `session.<ordinal>.v1.jsonl` for children; older role generations remain beside the selected highest file. The two exact alpha refusals were `snapshots/session/agent-instructions/session.jsonl`, whose projected compaction checkpoint has no matching start, and `snapshots/web/schedule-catalog/session.jsonl`, whose title source contradicts its citations. The continuing gate discovers the corpus dynamically and fails any refusal outside that closed manifest; separate assembled JSONL tests own exact physical-byte migration.
+Release verification ran the committed Session-format corpus gate over 152 versioned persisted-or-projected `session*.jsonl` fixtures under `snapshots/`, `packages/`, and `scripts/snapshots/python-sdk-single-exe/`. Fixture-only omitted envelopes and request-header tokens are materialized before the real static catalog; 150 fixtures reached the current v1 view through current restoration or historical migration. Released-v0 replay inputs remain suffixless, while fresh v1 writer outputs use `session.v1.jsonl` for a parent and `session.<ordinal>.v1.jsonl` for children. Record and refresh preserve older generations for every role still produced and remove all generations of a child role that the new run no longer produces. The two exact alpha refusals were `snapshots/session/agent-instructions/session.jsonl`, whose projected compaction checkpoint has no matching start, and `snapshots/web/schedule-catalog/session.jsonl`, whose title source contradicts its citations. The continuing gate discovers the corpus dynamically and fails any refusal outside that closed manifest; separate assembled JSONL tests own exact physical-byte migration.
 
 Current-head performance used three independent runs, each with 100 warmups and 600 alternating samples per case; a pooled 1,800-sample marginal estimator compared the immutable resolver with the same-commit dispatch-disabled baseline. Hot median/p95 deltas were raw small `-1.864%/-1.109%`, raw 100-turn `-0.711%/-0.445%`, Zstandard small `-0.880%/-5.025%`, and Zstandard 100-turn `-0.301%/-2.090%`, all within the five-percent regression ceiling. Cold enabled-path median/p95 costs were raw small `220.125/294.708 µs`, raw 100-turn `580.625/730.834 µs`, Zstandard small `248.042/960.083 µs`, and Zstandard 100-turn `636.291/1421.208 µs`. Repeated hot body reads performed zero directory scans; two listing calls performed two scans.
 

+ 3 - 3
.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md

@@ -16,7 +16,7 @@ Session 格式 v0 已随 alpha 版本发布,因此结构化 writer 变更不
 
 每条迁移边都会冻结严格的源与目标语义,其目标物理 codec 则保持词汇中立,使普通事件增长可以留在同一格式版本内。目录通过已安装的 peer `@deepseek-ai/dsh-session` 及其当前 `KNOWN_SESSION_EVENT_TYPES` 还原最终代,避免冻结的历史迁移边反过来成为当前词汇 owner。
 
-每个事件正文操作都经过持久化协调器的逐 Session 串行链,并在当前值离开前完成 provider 拥有的 ensure-current 工作。JSONL 会基于同一个物理快照融合最高 generation 解析、分类或迁移与当前格式解码;fallback 后端保留分离的 `ensureCurrent` 与当前读取钩子。六个公开读取是 prepare、load、inspect、borrowSession、readFrom 与 readRaw;冷 append 接管使用同一正文路径。仅 header 的 list 与 listSnapshots 从不迁移:它们重新扫描每个 Session 目录,并为数值最高的规范 generation 返回一个 descriptor,因此未来、不支持或 malformed 的最高文件会保持可见,而不会静默 fallback。
+每个事件正文操作都经过持久化协调器的逐 Session 串行链,并在当前值离开前完成 provider 拥有的 ensure-current 工作。JSONL 会基于同一个物理快照融合最高 generation 解析、分类或迁移与当前格式解码;fallback 后端保留分离的 `ensureCurrent` 与当前读取钩子。六个公开读取是 prepare、load、inspect、borrowSession、readFrom 与 readRaw;冷 append 接管使用同一正文路径。仅 header 的 list 与 listSnapshots 从不迁移:它们重新扫描每个 Session 目录,并为数值最高的规范 generation 返回一个 descriptor,因此未来、不支持或 malformed 的最高文件会保持可见,而不会静默 fallback。当后端无需信任 header 就能从 location 恢复 Session id 时,descriptor 会携带该 `storageId`;因此显式 identity 预留会拒绝已经占用该 id 的不可读产物。
 
 对于 `prepare`、`inspect` 与 `borrowSession`,取消属于观察调用,而不属于共享准备或迁移。被取消的观察者会停止等待,而已经开始的工作可以为另一检查者或后续恢复继续完成;持久发布绝不会为了满足观察者取消而回滚。分离的 `readFrom` 与 `readRaw` 操作则会把取消传入其串行化后端读取。
 
@@ -24,7 +24,7 @@ Session 格式 v0 已随 alpha 版本发布,因此结构化 writer 变更不
 
 规范文件名编码物理格式 generation:v0 是 `session.jsonl` 或 `session.jsonl.zstd`;每个正 generation 都是小写 `session.vN.jsonl` 或 `session.vN.jsonl.zstd`。发布绝不重命名、替换或删除已提交 generation 路径。目标已经存在时,只有它是普通当前格式文件且字节与预期完全相同时才接受;其他目标都会拒绝。低 generation 为 operator 检查或显式复制而保留,但普通 runtime 操作选择数值最高的规范名称,绝不把保留的前任当作自动 fallback、restore 或 downgrade 支持。
 
-当前格式快速路径从一个稳定源快照分类 header,不调用历史 converter,不写 generation,并把该快照交给当前格式解码,而不再次读取文件。后端会在单 writer 假设下缓存已校验的当前选择,供同一实例后续打开使用,而列表会有意重新扫描。多条迁移边保持原 generation 不变,并只发布最终目标;中间版本只存在于内存。同一进程内的操作会串行化,源 fingerprint 重新检查会在内容变化时重启完整尝试。跨进程 writer 隔离不在此保证内。
+当前格式快速路径从一个稳定源快照分类 header,不调用历史 converter,不写 generation,并把该快照交给当前格式解码,而不再次读取文件。后端会在单 writer 假设下缓存已校验的当前选择,供同一实例后续打开使用,而列表会有意重新扫描;如果选定文件消失,后端会让缓存失效并重新解析目录。多条迁移边保持原 generation 不变,并只发布最终目标;中间版本只存在于内存。同一进程内的操作会串行化,源 fingerprint 重新检查会在内容变化时重启完整尝试。跨进程 writer 隔离不在此保证内。
 
 第一条迁移边 `@deepseek-ai/dsh-session-format-v0-to-v1` 有意保持恒等形态:除版本和 v0 已接纳的有限历史归一化外,它保留逻辑 header、事件、序号、引用、时间戳、payload 与已配置的压缩选择。精确的 `session.jsonl[.zstd]` 源保持字节与 inode 相同,当前 writer 则编码新的 `session.v1.jsonl[.zstd]` 后继。这样可在出现改变基数的格式前先验证完整发布生命周期。
 
@@ -40,7 +40,7 @@ JSONL 发布在 POSIX 上使用硬链接创建与目录同步,在 Windows 上
 
 ## 验证
 
-发布验证针对 `snapshots/`、`packages/` 与 `scripts/snapshots/python-sdk-single-exe/` 下 152 个带版本、来自持久化或投影的 `session*.jsonl` fixture 运行了已提交 Session 格式语料门禁。fixture 专用的缺失信封与 request-header token 会先被实体化,再进入真实静态 catalog;其中 150 个通过当前格式 restore 或历史迁移得到当前 v1 视图。Released-v0 replay 输入保持无后缀,而新鲜 v1 writer 输出对 parent 使用 `session.v1.jsonl`、对 child 使用 `session.<ordinal>.v1.jsonl`;较旧角色 generation 保留在选定最高文件旁边。两个精确 alpha 拒绝分别是 `snapshots/session/agent-instructions/session.jsonl`(投影出的 compaction checkpoint 没有匹配 start)与 `snapshots/web/schedule-catalog/session.jsonl`(title 来源与其 citation 矛盾)。持续运行的门禁会动态发现语料,并拒绝封闭 manifest 之外的任何失败;独立组装式 JSONL 测试负责精确物理字节迁移。
+发布验证针对 `snapshots/`、`packages/` 与 `scripts/snapshots/python-sdk-single-exe/` 下 152 个带版本、来自持久化或投影的 `session*.jsonl` fixture 运行了已提交 Session 格式语料门禁。fixture 专用的缺失信封与 request-header token 会先被实体化,再进入真实静态 catalog;其中 150 个通过当前格式 restore 或历史迁移得到当前 v1 视图。Released-v0 replay 输入保持无后缀,而新鲜 v1 writer 输出对 parent 使用 `session.v1.jsonl`、对 child 使用 `session.<ordinal>.v1.jsonl`。Record 与 refresh 会为仍由新运行产生的每个 role 保留旧 generation,并删除新运行不再产生的 child role 的全部 generation。两个精确 alpha 拒绝分别是 `snapshots/session/agent-instructions/session.jsonl`(投影出的 compaction checkpoint 没有匹配 start)与 `snapshots/web/schedule-catalog/session.jsonl`(title 来源与其 citation 矛盾)。持续运行的门禁会动态发现语料,并拒绝封闭 manifest 之外的任何失败;独立组装式 JSONL 测试负责精确物理字节迁移。
 
 当前 head 性能证据来自 3 次独立运行,每个 case 含 100 次 warmup 与 600 个 alternating sample;pooled 1,800-sample marginal estimator 把不可变 resolver 与同一 commit 下禁用 dispatch 的 baseline 比较。Hot median/p95 delta 分别为 raw small `-1.864%/-1.109%`、raw 100-turn `-0.711%/-0.445%`、Zstandard small `-0.880%/-5.025%`、Zstandard 100-turn `-0.301%/-2.090%`,全部满足 5% regression ceiling。Cold enabled-path median/p95 cost 分别为 raw small `220.125/294.708 µs`、raw 100-turn `580.625/730.834 µs`、Zstandard small `248.042/960.083 µs`、Zstandard 100-turn `636.291/1421.208 µs`。重复 hot body read 执行 0 次目录扫描;两次 listing 调用执行 2 次扫描。
 

+ 9 - 0
apps/web/tests/scaffold-generation.spec.ts

@@ -166,6 +166,15 @@ describe('Web snapshot generation filenames', () => {
       .resolves.toBe(join(root, 'session.jsonl'))
   })
 
+  it('selects a committed sibling when the requested older generation is absent', async () => {
+    const root = await mkdtemp(join(tmpdir(), 'dsh-web-fixture-generations-'))
+    roots.push(root)
+    await writeFile(join(root, 'session.v1.jsonl'), '')
+
+    await expect(selectedSessionFixture(join(root, 'session.jsonl')))
+      .resolves.toBe(join(root, 'session.v1.jsonl'))
+  })
+
   it('records beside an older generation and preserves the parent or child role', () => {
     const fixtures = join('/', 'fixtures')
     expect(recordedSessionFixturePath(join(fixtures, 'session.jsonl'), 1))

+ 6 - 7
apps/web/tests/scaffold.ts

@@ -47,7 +47,6 @@ import {
   normalizeSessionSnapshots,
   scrubRequestHeaders,
   scrubSessionSnapshot,
-  sessionFixtureFiles,
   sessionFixtureName,
   stabilizeFixtureMessageIds,
   type NormalizeContext,
@@ -153,12 +152,12 @@ async function ownsReplayFixture(replayFixture: string | undefined): Promise<boo
 export async function selectedSessionFixture(path: string): Promise<string> {
   const requested = parseSessionFixtureName(basename(path))
   if (requested === undefined) return path
-  // An override-only scenario deliberately has no projected parent log. Keep
-  // the absent source path so the replay adapter can use its replacement
-  // script without asking the fixture-role inventory to invent a parent.
-  if (!existsSync(path)) return path
-  const selected = sessionFixtureFiles(await readdir(dirname(path)))
-    .find(candidate => candidate.index === requested.index)
+  const selected = (await readdir(dirname(path)))
+    .map(parseSessionFixtureName)
+    .filter((candidate): candidate is NonNullable<typeof candidate> => candidate?.index === requested.index)
+    .sort((left, right) => right.version - left.version)[0]
+  // An override-only scenario deliberately has no projected parent role. Keep
+  // the absent source only when no committed generation exists for that role.
   return selected === undefined ? path : join(dirname(path), selected.name)
 }
 

+ 8 - 1
packages/test-support/session-snapshot/src/suite.ts

@@ -17,7 +17,7 @@
  * @module @deepseek-ai/dsh-session-snapshot/suite
  */
 
-import { readFile, readdir, writeFile } from 'node:fs/promises'
+import { readFile, readdir, rm, writeFile } from 'node:fs/promises'
 import { existsSync } from 'node:fs'
 import { join } from 'node:path'
 import { isSurfaceEligibleType } from '@deepseek-ai/dsh-session/surface'
@@ -28,6 +28,7 @@ import { redactSessionSnapshotIds } from './identity.ts'
 import { captureExpectedWorkspaceSnapshot } from './workspace.ts'
 import {
   assertSessionFixtureVersion,
+  parseSessionFixtureName,
   sessionFixtureName,
   sessionFixtureNames,
   sessionHeaderVersion,
@@ -1275,6 +1276,12 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
           const outputFixtures = redactSessionSnapshotIds(stabilizeFixtureMessageIds(freshFixtures, existingFixtures))
           await Promise.all(outputFixtures.map((fixture, index) =>
             writeFile(join(dir, outputFixtureFiles[index] as string), fixture)))
+          const retiredFixtures = (await readdir(dir))
+            .map(parseSessionFixtureName)
+            .filter((fixture): fixture is NonNullable<typeof fixture> => (
+              fixture !== undefined && fixture.index >= outputFixtureFiles.length
+            ))
+          await Promise.all(retiredFixtures.map(fixture => rm(join(dir, fixture.name))))
           fixtureFiles = outputFixtureFiles
           if (scenario.pinsHeader === true) {
             const primary = result.sessionLogs[0] as HarvestedLog

+ 3 - 3
packages/test-support/session-snapshot/tests/suite.spec.ts

@@ -1,4 +1,4 @@
-import { cpSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
+import { cpSync, existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'
 import { rm } from 'node:fs/promises'
 import { tmpdir } from 'node:os'
 import { join } from 'node:path'
@@ -214,12 +214,12 @@ describe('defineAcpSnapshotSuite: refresh write-back', () => {
 })
 
 describe('defineAcpSnapshotSuite: record inventory write-back', () => {
-  it('creates a missing primary fixture without deleting a retired child generation', () => {
+  it('creates a missing primary fixture and removes generations for a retired child role', () => {
     const fixture = readFileSync(join(recordDir, 'rec-pin', 'session.v1.jsonl'), 'utf8')
     expect(fixture).toContain('"type":"session"')
     expect(fixture).toContain('"cwd":"{{cwd}}"')
     if (!BOOTSTRAP) {
-      expect(readFileSync(join(recordDir, 'rec-child', 'session.2.jsonl'), 'utf8')).toBe(retiredChildFixture)
+      expect(existsSync(join(recordDir, 'rec-child', 'session.2.jsonl'))).toBe(false)
     }
     expect(readFileSync(join(recordDir, 'rec-child', 'tool-schemas.1.expected.json'), 'utf8'))
       .toContain('"name": "t1"')