瀏覽代碼

fix(ci): serial-windows notices timeout and generator store-scan cost

render() loaded the workspace manifests once per external dependency
name through workspaceLinkedManifest, an O(names x manifests) file
read on the cold path; on the loaded self-hosted Windows host with
coverage instrumentation the freshness spec crossed Vitest's default
5000ms budget and failed the serial-windows standby gate four times in
a week. Load the manifests once in render() and thread the map through
the collectors instead.

The serial-windows lane also ran the coverage inventory at the strictest
budget of any lane: add DSH_COVERAGE_TEST_TIMEOUT_MS=90000 to match the
pull-request windows-coverage lane, pinned by ci-workflow.spec.ts.
Chinesezjc 1 周之前
父節點
當前提交
a074e6131f

+ 6 - 0
.agents/notes/implemented/process/2026-08-31-serial-windows-notices-timeout-budget.i18n.yaml

@@ -0,0 +1,6 @@
+# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
+# side as of the last confirmed-consistent state. Both languages carry equal authority;
+# after editing either side, bring the other along and re-record with:
+#   pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-31-serial-windows-notices-timeout-budget.md
+2026-08-31-serial-windows-notices-timeout-budget.md: 184004122bf8050fa00038b7c57ae680748463f8
+2026-08-31-serial-windows-notices-timeout-budget.zh.md: 4fc00ed25cc24ef76e43a0099a362b8be0e59e3e

+ 30 - 0
.agents/notes/implemented/process/2026-08-31-serial-windows-notices-timeout-budget.md

@@ -0,0 +1,30 @@
+# Agent Note: serial-windows notices timeout budget and generator store-scan cost
+
+Status: implemented
+
+English | [中文](2026-08-31-serial-windows-notices-timeout-budget.zh.md)
+
+## Problem
+
+The `serial / windows (self-hosted standby)` master lane failed its `test:coverage` gate four times in a week (runs 33333033178, 33311481884, 33352293522, 33353113100), always on the same case: `scripts/gen-third-party-notices.spec.ts > THIRD_PARTY_NOTICES.md > matches what the generator produces from the current manifests`, with `Error: Test timed out in 5000ms`. Measured test wall times on the shared Windows host were 4149–8853 ms against Vitest's default 5000 ms per-test budget. All other 26 cases in the file finished in 0–3 ms, and the passing run two hours later (33360033028) had the same code green.
+
+The lane runs the complete unsharded Windows gate inventory serially with `DSH_COVERAGE_MAX_WORKERS=1`, so `render()` regenerates `THIRD_PARTY_NOTICES.md` from the workspace manifests and the installed pnpm store on a host shared by 32 runners. The cold path is dominated by `workspaceLinkedManifest`, which re-ran `loadWorkspaceManifests()` — a glob plus reads and JSON-parses of every workspace `package.json` — once per cache-missing external dependency name: 130 names × 270 manifests ≈ 35k file operations, on top of a `.pnpm` store scan per name. Under v8 coverage instrumentation and shared-host I/O contention that crossed the 5 s default.
+
+The lane also had no `DSH_COVERAGE_TEST_TIMEOUT_MS`, unlike the pull-request `windows-coverage` lane ([ci.yml](../../../../.github/workflows/ci.yml)) which grants 90000 ms, so the serial reference ran the same coverage inventory at the strictest budget of any lane.
+
+## Decision
+
+Two changes:
+
+1. [scripts/gen-third-party-notices.ts](../../../../scripts/gen-third-party-notices.ts) loads the workspace manifests once in `render()` and threads the map through `collectNpmDeps` → `installedMetadata` → `installedManifest` → `workspaceLinkedManifest` instead of reloading it per external dependency name. The cold `render()` wall time on the same checkout fell from ~893 ms to ~86 ms with byte-identical output (verified by diffing the rendered documents before and after the change).
+
+2. [ci-master.yml](../../../../.github/workflows/ci-master.yml) `serial-windows` step "Run complete unsharded Windows gate inventory serially" gains `DSH_COVERAGE_TEST_TIMEOUT_MS: '90000'`, matching the pull-request `windows-coverage` lane budget. This extends the per-test, expect.poll, and hook budget mechanism defined by [the Windows lane hook and Lefthook budget note](../testing/2026-08-29-windows-lane-hook-and-lefthook-budget.md) to a second lane; that note records which lanes set the env. `scripts/ci-workflow.spec.ts` pins this env with a `toMatchObject` assertion; removing the env turns the spec red (negative control exercised).
+
+## Alternatives considered
+
+- **Raise the lane budget only** - rejected as the sole fix: it would mask the O(names × manifests) reload for every lane that runs the generator, including the pre-commit hook and the standalone `--check` path.
+- **Module-level cache for `loadWorkspaceManifests()`** - rejected in favor of explicit threading, which keeps the single-load contract visible at the call site and avoids a second hidden cache next to `workspaceLinkedManifestCache`.
+
+## Consequences
+
+The generator resolves installed metadata from one manifest load per `render()` call, and clears the name-keyed linked-manifest cache at the start of each call so the cache cannot outlive the map it was resolved from. The serial-windows lane runs the coverage inventory at the same 90000 ms per-test budget as the pull-request coverage lane. `THIRD_PARTY_NOTICES.md` bytes are unchanged; the freshness spec still compares `render()` against the committed document.

+ 30 - 0
.agents/notes/implemented/process/2026-08-31-serial-windows-notices-timeout-budget.zh.md

@@ -0,0 +1,30 @@
+# Agent Note:serial-windows 的 notices 超时预算与 generator store 扫描成本
+
+Status: implemented
+
+[English](2026-08-31-serial-windows-notices-timeout-budget.md) | 中文
+
+## Problem
+
+`serial / windows (self-hosted standby)` master lane 一周内四次失败在 `test:coverage` gate(run 33333033178、33311481884、33352293522、33353113100),失败用例每次都相同:`scripts/gen-third-party-notices.spec.ts > THIRD_PARTY_NOTICES.md > matches what the generator produces from the current manifests`,报 `Error: Test timed out in 5000ms`。共享 Windows 主机上该用例实测 4149–8853 ms,超出 Vitest 默认的 5000 ms 单测预算。文件其余 26 个用例全部 0–3 ms 通过,两小时后的 passing run(33360033028)用同一份代码全绿。
+
+该 lane 以 `DSH_COVERAGE_MAX_WORKERS=1` 串行跑完整的无分片 Windows gate 清单,`render()` 要从 workspace manifest 和已安装的 pnpm store 全量重生成 `THIRD_PARTY_NOTICES.md`,而主机被 32 个 runner 共享。冷路径的代价集中在 `workspaceLinkedManifest`:它对每个未缓存的外部依赖名重跑一遍 `loadWorkspaceManifests()`——glob 并读取、解析全部 workspace `package.json`——即 130 名 × 270 manifest ≈ 3.5 万次文件操作,另加每个名字一次 `.pnpm` store 扫描。叠加 v8 覆盖率插桩与共享主机 I/O 争抢后越过 5 秒默认值。
+
+该 lane 还没有 `DSH_COVERAGE_TEST_TIMEOUT_MS`,而 pull-request 的 `windows-coverage` lane([ci.yml](../../../../.github/workflows/ci.yml))给的是 90000 ms——于是这条 serial 参考 lane 用全仓库最紧的预算跑同一份 coverage 清单。
+
+## Decision
+
+两处改动:
+
+1. [scripts/gen-third-party-notices.ts](../../../../scripts/gen-third-party-notices.ts) 在 `render()` 里只加载一次 workspace manifest,把 map 沿 `collectNpmDeps` → `installedMetadata` → `installedManifest` → `workspaceLinkedManifest` 显式传递,不再按外部依赖名逐个重载。同一 checkout 下冷 `render()` 墙钟从约 893 ms 降到约 86 ms,输出逐字节一致(改动前后渲染结果 diff 验证)。
+
+2. [ci-master.yml](../../../../.github/workflows/ci-master.yml) `serial-windows` 的 "Run complete unsharded Windows gate inventory serially" 步骤增加 `DSH_COVERAGE_TEST_TIMEOUT_MS: '90000'`,与 pull-request `windows-coverage` lane 对齐。这是把 [Windows 覆盖率 lane 的 hook 预算与 Lefthook 套件预算 note](../testing/2026-08-29-windows-lane-hook-and-lefthook-budget.zh.md) 定义的 per-test、expect.poll 与 hook 预算机制扩展到第二个 lane;该 note 记录了哪些 lane 设置此 env。`scripts/ci-workflow.spec.ts` 用 `toMatchObject` 断言钉住该 env;删掉 env 会让 spec 变红(已做负例验证)。
+
+## Alternatives considered
+
+- **只放宽 lane 预算** - 否决作为唯一修复:会掩盖所有运行 generator 的 lane 上的 O(名×manifest) 重载成本,包括 pre-commit hook 与独立 `--check` 路径。
+- **给 `loadWorkspaceManifests()` 加模块级缓存** - 否决,改用显式传递:把「单次加载」契约留在调用点可见,避免在 `workspaceLinkedManifestCache` 之外再加一层隐藏缓存。
+
+## Consequences
+
+generator 每次 `render()` 调用只加载一次 manifest 来解析已安装元数据,并在调用开头清空按名字作键的 linked-manifest 缓存,使缓存不会活过它解析自的那份 map。serial-windows lane 与 pull-request coverage lane 一样按 90000 ms 单测预算跑 coverage 清单。`THIRD_PARTY_NOTICES.md` 字节不变;新鲜度 spec 仍把 `render()` 与已提交文档逐字节比较。

+ 2 - 2
.agents/notes/implemented/testing/2026-08-29-windows-lane-hook-and-lefthook-budget.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-08-29-windows-lane-hook-and-lefthook-budget.md
-2026-08-29-windows-lane-hook-and-lefthook-budget.md: 6886e3ad4958d20a88a66df6a9e02f5a60a36a6a
-2026-08-29-windows-lane-hook-and-lefthook-budget.zh.md: 56c1e625d92f01e24f2268deb5d03040278da6af
+2026-08-29-windows-lane-hook-and-lefthook-budget.md: 40ebba25e459abd6d9ae755f831e703aa826ae39
+2026-08-29-windows-lane-hook-and-lefthook-budget.zh.md: c14be5da098920dbb7360c01bbb94f0bc22f700b

+ 1 - 1
.agents/notes/implemented/testing/2026-08-29-windows-lane-hook-and-lefthook-budget.md

@@ -24,7 +24,7 @@ A `git` or `node` spawn spike on the shared-volume runners no longer decides eit
 
 Both budgets widen what counts as an acceptable duration, so a real slowdown into tens of seconds now passes where the previous ceilings would have caught it. That detection is traded away deliberately: those ceilings were firing on host contention rather than on regressions.
 
-The hook change applies wherever `DSH_COVERAGE_TEST_TIMEOUT_MS` is set, which today is the Windows coverage lane alone. Lanes that leave it unset keep every Vitest default, including the 10 s hook budget.
+The hook change applies wherever `DSH_COVERAGE_TEST_TIMEOUT_MS` is set: the Windows coverage lane in [ci.yml](../../../../.github/workflows/ci.yml) and the `serial-windows` master standby in [ci-master.yml](../../../../.github/workflows/ci-master.yml) ([the serial-windows notices timeout note](../process/2026-08-31-serial-windows-notices-timeout-budget.md) records the second lane's adoption). Lanes that leave it unset keep every Vitest default, including the 10 s hook budget.
 
 ## Alternatives considered
 

+ 1 - 1
.agents/notes/implemented/testing/2026-08-29-windows-lane-hook-and-lefthook-budget.zh.md

@@ -24,7 +24,7 @@ Lefthook 套件取 `{ timeout: 90_000 }`,与 [`.github/workflows/ci.yml`](../.
 
 两份预算都放宽了「多长算可接受」,因此一个退化到几十秒的真实变慢现在会通过,而此前的上限会拦住它。这项检测能力是有意换掉的:那些上限触发的是宿主机争抢,不是回归。
 
-hook 的改动在所有设置了 `DSH_COVERAGE_TEST_TIMEOUT_MS` 的地方生效,目前仅 Windows 覆盖率 lane 一处。不设置它的 lane 保持全部 Vitest 默认值,包括 10 秒的 hook 预算。
+hook 的改动在所有设置了 `DSH_COVERAGE_TEST_TIMEOUT_MS` 的地方生效:[ci.yml](../../../../.github/workflows/ci.yml) 的 Windows 覆盖率 lane,以及 [ci-master.yml](../../../../.github/workflows/ci-master.yml) 的 `serial-windows` master standby([serial-windows notices 超时 note](../process/2026-08-31-serial-windows-notices-timeout-budget.zh.md) 记录了第二个 lane 的采用)。不设置它的 lane 保持全部 Vitest 默认值,包括 10 秒的 hook 预算。
 
 ## 备选方案
 

+ 1 - 0
.github/workflows/ci-master.yml

@@ -208,6 +208,7 @@ jobs:
         shell: pwsh
         env:
           DSH_COVERAGE_MAX_WORKERS: '1'
+          DSH_COVERAGE_TEST_TIMEOUT_MS: '90000'
           DSH_GATE_CONCURRENCY: '1'
           DSH_PUBLINT_CONCURRENCY: '1'
         run: pnpm run check:ci:windows-complete

+ 8 - 0
scripts/ci-workflow.spec.ts

@@ -201,6 +201,14 @@ describe('CI workflow', () => {
     expect(serialInstall!.run.split('\n').map(line => line.trim())).toContain('} else {')
     expect(serialInstall!.run.split('\n').map(line => line.trim())).toContain('pnpm install --frozen-lockfile')
     expect(serialInstall!.run).not.toContain('$cloneFlag')
+    // The unsharded reference runs the whole coverage inventory at the same
+    // per-test budget the PR coverage lane grants; the default 5000ms times
+    // out load-sensitive store scans (e.g. gen-third-party-notices).
+    const serialGate = serialSteps.find((step): step is Record<string, unknown> & { env?: Record<string, unknown> } => (
+      isRecord(step) && step.name === 'Run complete unsharded Windows gate inventory serially'
+    ))
+    expect(serialGate).toBeDefined()
+    expect(serialGate!.env).toMatchObject({ DSH_COVERAGE_TEST_TIMEOUT_MS: '90000' })
 
     // Aggregate: Wine and the required split native jobs are needed;
     // windows-coverage is temporarily non-blocking while Windows ACP

+ 19 - 14
scripts/gen-third-party-notices.ts

@@ -279,11 +279,13 @@ const workspaceLinkedManifestCache = new Map<string, VirtualManifest | undefined
  * Resolve the package version selected for a declaring workspace instead of an
  * unrelated historical version that still occupies the shared virtual store.
  * @param name - external package identity.
+ * @param manifests - workspace manifests already loaded by the caller, so one
+ *   load serves every dependency instead of a full re-read per name.
  * @returns the first current workspace link for that package, when installed.
  */
-function workspaceLinkedManifest(name: string): VirtualManifest | undefined {
+function workspaceLinkedManifest(name: string, manifests: Map<string, Manifest>): VirtualManifest | undefined {
   if (workspaceLinkedManifestCache.has(name)) return workspaceLinkedManifestCache.get(name)
-  for (const [path, manifest] of loadWorkspaceManifests().manifests) {
+  for (const [path, manifest] of manifests) {
     if (!ALL_KINDS.some(kind => name in (manifest[kind] ?? {}))) continue
     const linked = resolve(root, dirname(path), 'node_modules', name, 'package.json')
     if (!existsSync(linked)) continue
@@ -296,8 +298,8 @@ function workspaceLinkedManifest(name: string): VirtualManifest | undefined {
 }
 
 /** Resolve one installed external package manifest from either pnpm store. */
-function installedManifest(name: string, expectedVersion?: string): VirtualManifest | undefined {
-  const linked = workspaceLinkedManifest(name)
+function installedManifest(name: string, manifests: Map<string, Manifest>, expectedVersion?: string): VirtualManifest | undefined {
+  const linked = workspaceLinkedManifest(name, manifests)
   if (linked !== undefined && (expectedVersion === undefined || linked.version === expectedVersion)) return linked
   let manifest: (Manifest & { license?: string; repository?: string | { url?: string }; homepage?: string }) | undefined
   // Workspace-local link farms can expose a dependency that is not linked at
@@ -320,9 +322,9 @@ function installedManifest(name: string, expectedVersion?: string): VirtualManif
 }
 
 /** License and repository URL for an installed external package, from the pnpm store. */
-function installedMetadata(name: string): { license: string; repo: string } {
+function installedMetadata(name: string, manifests: Map<string, Manifest>): { license: string; repo: string } {
   const override = OVERRIDES[name]
-  const manifest = installedManifest(name)
+  const manifest = installedManifest(name, manifests)
   const license = override?.license ?? manifest?.license
   const rawRepo = typeof manifest?.repository === 'string' ? manifest.repository : manifest?.repository?.url ?? manifest?.homepage
   const repo = override?.repo ?? normalizeRepo(rawRepo)
@@ -332,8 +334,8 @@ function installedMetadata(name: string): { license: string; repo: string } {
   return { license, repo }
 }
 
-function collectClaudeDistribution(): ClaudeDistribution {
-  const manifest = installedManifest(CLAUDE_AGENT_SDK_PACKAGE)
+function collectClaudeDistribution(manifests: Map<string, Manifest>): ClaudeDistribution {
+  const manifest = installedManifest(CLAUDE_AGENT_SDK_PACKAGE, manifests)
   if (manifest === undefined) {
     throw new Error(
       `gen-third-party-notices: cannot resolve ${CLAUDE_AGENT_SDK_PACKAGE}; run \`pnpm install\`.`,
@@ -342,7 +344,7 @@ function collectClaudeDistribution(): ClaudeDistribution {
   const distribution = claudeDistributionFromManifest(manifest)
   let installedPayloads = 0
   for (const payload of distribution.payloads) {
-    const installed = installedManifest(payload.name, payload.version)
+    const installed = installedManifest(payload.name, manifests, payload.version)
     if (installed === undefined) continue
     installedPayloads += 1
     if (
@@ -383,12 +385,11 @@ function normalizeRepo(raw: string | undefined): string | undefined {
  * by tooling, test infrastructure, the website, or the demo leaves — whatever
  * the declaring section is called — is development-only.
  */
-function collectNpmDeps(): ExternalDep[] {
-  const { manifests, names } = loadWorkspaceManifests()
+function collectNpmDeps(manifests: Map<string, Manifest>, names: Set<string>): ExternalDep[] {
   return [...tierExternalDeps(manifests, names)]
     .filter(([name]) => !FIRST_PARTY.has(name))
     .sort(([a], [b]) => a.localeCompare(b))
-    .map(([name, runtime]) => ({ name, ...installedMetadata(name), runtime }))
+    .map(([name, runtime]) => ({ name, ...installedMetadata(name, manifests), runtime }))
 }
 
 /**
@@ -692,7 +693,11 @@ ${rows.join('\n')}
  */
 export function render(): string {
   verifyBuildTimePins()
-  const npm = collectNpmDeps()
+  // The linked-manifest cache is keyed by name only, so it must not outlive
+  // the manifests map it was resolved from; render() owns that single load.
+  workspaceLinkedManifestCache.clear()
+  const { manifests, names } = loadWorkspaceManifests()
+  const npm = collectNpmDeps(manifests, names)
   const runtimeDeps = npm.filter(dep => dep.runtime)
   const devDeps = npm.filter(dep => !dep.runtime)
   const vendored = collectVendored()
@@ -701,7 +706,7 @@ export function render(): string {
   const claudeDistribution = runtimeDeps.some(
     dep => dep.name === CLAUDE_AGENT_SDK_PACKAGE,
   )
-    ? collectClaudeDistribution()
+    ? collectClaudeDistribution(manifests)
     : undefined
   const nonPermissiveDev = devDeps.filter(dep => !isPermissive(dep.license))
   // A copyleft license reaching a shipped surface is a distribution decision,