Преглед изворни кода

fix(ci): isolate npm caches and synchronize ACP snapshot completion

Tianyi Cui пре 1 месец
родитељ
комит
e61d9fc0c7

+ 2 - 2
.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.md
-2026-09-06-pr-ci-runner-temporary-storage.md: 6badc6b5ece458e401ab107d7d1b5cbd42e88c97
-2026-09-06-pr-ci-runner-temporary-storage.zh.md: b5d173085845ee5f52f8b0367d352868a4ae4445
+2026-09-06-pr-ci-runner-temporary-storage.md: aabd208fb1c6ebe3d1e55611a054395170c63e82
+2026-09-06-pr-ci-runner-temporary-storage.zh.md: f83deb9341478568ba09a1666c4d22115c1712ba

+ 9 - 1
.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.md

@@ -12,8 +12,14 @@ The Linux failover pool runs multiple runner instances on one VM. PR coverage an
 
 The static, coverage, and consumer jobs in [PR CI](../../../../.github/workflows/ci.yml) export `TMPDIR=runner.temp` through `GITHUB_ENV` in their first step before any setup or test process starts. Node, Vite, tsx, and temporary test consumers inherit the runner-owned location. Each runner owns its directory and GitHub Actions clears its removable contents at job start and completion; fixtures still allocate unique children and retain their own cleanup.
 
+The three workers also set `npm_config_cache` to `runner.temp/npm-cache`. The [release workflows](../../../../.github/workflows/release.yml) apply the same cache location in their existing temporary-storage setup, including [vendor rehearsals](../../../../.github/workflows/release-vendor.yml). npm otherwise caches registry responses under the shared home directory regardless of `TMPDIR`; a temporary consumer alone does not isolate those writes. The persistent pnpm store is unchanged.
+
 The [release rehearsal decision](../process/2026-09-06-release-rehearsal-selfhosted.md) applies the same lifetime rule to release consumers. The [failover runbook](../process/2026-07-26-ci-failover-runbook.md) continues to own runner selection and shared-host capacity. This change does not retarget jobs, reduce concurrency, retry tests, change assertions, or modify master-only CI.
 
+## Recorded ACP completion order
+
+The [ACP diagnostic scenario](../../../../snapshots/session/subagent-acp-diagnostic/cordis.snapshot.yml) holds its scripted background response until `job_output` owns the completion wait. Without that synchronization, a fast child can publish a legitimate job notice between the recorded parent steps. A scenario-local wrapper releases the child after the jobs service registers the completion waiter; the mock watches an exclusive marker in the private test workspace and closes the watcher after release. The fixture restores the wrapped method on disposal. The recorded Session bytes and production job-notice behavior stay unchanged.
+
 ## Alternatives considered
 
 **Delete shared temporary files from a PR job.** Another runner may still own those files. Repository jobs must not reclaim a shared directory by pathname or age.
@@ -24,6 +30,8 @@ The [release rehearsal decision](../process/2026-09-06-release-rehearsal-selfhos
 
 ## Consequences
 
-Temporary output follows the job lifetime instead of accumulating in unmanaged host storage. This does not reclaim existing shared temporary files, guarantee filesystem capacity, or clean files the runner account cannot remove. Operators still own historical residue, disk provisioning, and jobs outside this PR workflow.
+Output honoring these temporary-directory and cache settings follows the job lifetime instead of accumulating in unmanaged host storage. This does not reclaim existing shared temporary files, guarantee filesystem capacity, or clean files the runner account cannot remove. Operators still own historical residue, disk provisioning, and jobs outside this PR workflow.
+
+Linux bwrap and Landlock workspace-write profiles grant literal `/tmp` and the workspace, not an inherited `TMPDIR` outside it; confined fixtures must place temporary writes in those granted paths. The [snapshot spill helper](../../../../packages/test-support/session-snapshot/src/harness.ts) also uses literal `/tmp/dsh-acp-snap-*` on POSIX for stable path lengths, so that output remains outside runner cleanup. This workflow change neither widens sandbox grants nor rewrites fixed-path fixtures.
 
 The parsed-workflow cases in [ci-workflow.spec.ts](../../../../scripts/ci-workflow.spec.ts) require the assignment on all three workers and reject step-level overrides. They fail against the unmodified workflow. Independent-process smoke checks and repeated PR runs validate the actual tooling; the YAML assertions alone do not prove host capacity.

+ 9 - 1
.agents/notes/implemented/testing/2026-09-06-pr-ci-runner-temporary-storage.zh.md

@@ -12,8 +12,14 @@ Linux 故障切换池在同一台虚拟机上运行多个 runner 实例。PR 覆
 
 [PR CI](../../../../.github/workflows/ci.yml) 的静态检查、覆盖率和消费者作业在任何准备或测试进程启动前,在首个步骤通过 `GITHUB_ENV` 导出 `TMPDIR=runner.temp`。Node、Vite、tsx 和临时测试消费者继承 runner 管理的位置。每个 runner 管理自己的目录,GitHub Actions 在作业开始和完成时清除其中可删除的内容;测试夹具仍分配唯一子目录,并保留自身清理逻辑。
 
+这三个 worker 还将 `npm_config_cache` 设为 `runner.temp/npm-cache`。[发布工作流](../../../../.github/workflows/release.yml) 在既有临时存储准备步骤中采用相同缓存位置,[vendor 演练](../../../../.github/workflows/release-vendor.yml) 也如此。否则,无论 `TMPDIR` 如何设置,npm 都会在共享 home 目录中缓存注册表响应;仅使用临时消费者目录不能隔离这些写入。持久化 pnpm store 保持不变。
+
 [发布演练决策](../process/2026-09-06-release-rehearsal-selfhosted.zh.md) 对发布消费者采用相同的生命周期规则。[故障切换运行手册](../process/2026-07-26-ci-failover-runbook.zh.md) 继续负责 runner 选择和共享主机容量。本变更不调整作业目标、不降低并发、不重试测试、不修改断言,也不修改仅在 master 上执行的 CI。
 
+## ACP 完成顺序的录制
+
+[ACP 诊断场景](../../../../snapshots/session/subagent-acp-diagnostic/cordis.snapshot.yml) 暂停脚本化的后台响应,直到 `job_output` 开始等待完成。没有这种同步,快速子进程可能在录制的父步骤之间发布合法的作业通知。场景本地 wrapper 在 jobs 服务注册完成等待器后释放子进程;mock 在测试私有 workspace 中监听独占创建的标记,并在释放后关闭 watcher。夹具在销毁时恢复被包装的方法。录制的 Session 字节和生产作业通知行为保持不变。
+
 ## 考虑过的替代方案
 
 **由 PR 作业删除共享临时文件。** 其他 runner 可能仍在使用这些文件。仓库作业不得按路径或文件年龄回收共享目录。
@@ -24,6 +30,8 @@ Linux 故障切换池在同一台虚拟机上运行多个 runner 实例。PR 覆
 
 ## 影响
 
-临时输出随作业生命周期清理,不再累积于无人管理的主机存储。本方案不回收既有共享临时文件、不保证文件系统容量,也不清理 runner 账号无权删除的文件。历史残留、磁盘配置以及本 PR 工作流以外的作业仍由运维人员负责。
+遵循临时目录和缓存配置的输出随作业生命周期清理,而不累积于无人管理的主机存储。本方案不回收既有共享临时文件、不保证文件系统容量,也不清理 runner 账号无权删除的文件。历史残留、磁盘配置以及本 PR 工作流以外的作业仍由运维人员负责。
+
+Linux bwrap 和 Landlock 的 workspace-write profile 允许写入字面路径 `/tmp` 和 workspace,而不允许写入其外部继承的 `TMPDIR`;受限测试夹具必须将临时写入放在这些已授权路径中。[快照 spill helper](../../../../packages/test-support/session-snapshot/src/harness.ts) 在 POSIX 上也使用字面路径 `/tmp/dsh-acp-snap-*` 以保持路径长度稳定,因此这些输出仍不受 runner 清理管理。本工作流变更既不扩大沙箱授权,也不重写固定路径夹具。
 
 [ci-workflow.spec.ts](../../../../scripts/ci-workflow.spec.ts) 的 YAML 解析用例要求三个 worker 都包含该赋值,并拒绝步骤级别的覆盖。它们在未修改的工作流上失败。独立进程 smoke 检查和重复 PR 运行验证实际工具链;仅有 YAML 断言不能证明主机容量充足。

+ 9 - 3
.github/workflows/ci.yml

@@ -52,7 +52,9 @@ jobs:
     steps:
       # Runner cleanup owns tool caches and fixtures, including cancelled runs.
       - name: Use runner-owned temporary storage
-        run: echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
+        run: |
+          echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
+          echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
 
       # Fetch complete history so the archive gate can read the trusted PR base from a reused shallow checkout.
       - uses: actions/checkout@v6
@@ -115,7 +117,9 @@ jobs:
     steps:
       # Runner cleanup owns tool caches and fixtures, including cancelled runs.
       - name: Use runner-owned temporary storage
-        run: echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
+        run: |
+          echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
+          echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
 
       - uses: actions/checkout@v6
         with:
@@ -230,7 +234,9 @@ jobs:
     steps:
       # Runner cleanup owns tool caches and fixtures, including cancelled runs.
       - name: Use runner-owned temporary storage
-        run: echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
+        run: |
+          echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
+          echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
 
       - uses: actions/checkout@v6
         with:

+ 1 - 0
.github/workflows/release-vendor.yml

@@ -53,6 +53,7 @@ jobs:
         run: |
           echo "NODE_COMPILE_CACHE=${{ runner.temp }}/node-compile-cache" >> "$GITHUB_ENV"
           echo "npm_config_devdir=${{ runner.temp }}/node-gyp" >> "$GITHUB_ENV"
+          echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
           echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
 
       - uses: pnpm/action-setup@v4

+ 2 - 0
.github/workflows/release.yml

@@ -50,6 +50,7 @@ jobs:
         run: |
           echo "NODE_COMPILE_CACHE=${{ runner.temp }}/node-compile-cache" >> "$GITHUB_ENV"
           echo "npm_config_devdir=${{ runner.temp }}/node-gyp" >> "$GITHUB_ENV"
+          echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
           echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
 
       - uses: pnpm/action-setup@v4
@@ -112,6 +113,7 @@ jobs:
         run: |
           echo "NODE_COMPILE_CACHE=${{ runner.temp }}/node-compile-cache" >> "$GITHUB_ENV"
           echo "npm_config_devdir=${{ runner.temp }}/node-gyp" >> "$GITHUB_ENV"
+          echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"
           echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"
 
       - uses: pnpm/action-setup@v4

+ 17 - 1
packages/subagent/subagent-acp/tests/mock-acp-server.ts

@@ -34,6 +34,8 @@
  *                        handler is in flight (it has streamed its chunk). A test
  *                        polls for this file to cancel on a CONDITION rather than
  *                        an arbitrary timeout (subprocess cold-start is variable).
+ * - `MOCK_PROMPT_HOLD` — if set, prompt waits while this file exists; removing
+ *                        it releases the response without a timing assumption.
  * - `MOCK_MISSING_SESSION_ID` — if `1`, return a malformed empty `session/new`
  *                        response to exercise startup rollback.
  * - `MOCK_FLUSH_ON_EOF` — if set, on stdin EOF the agent takes an async beat
@@ -59,7 +61,8 @@
  */
 
 import { randomUUID } from 'node:crypto'
-import { existsSync, writeFileSync } from 'node:fs'
+import { existsSync, watch, writeFileSync } from 'node:fs'
+import { dirname } from 'node:path'
 import { Readable, Writable } from 'node:stream'
 import {
   agent as createAcpAgentApp,
@@ -141,6 +144,19 @@ function makeAgent() {
     },
     async prompt(params: PromptRequest, conn: AgentContext): Promise<PromptResponse> {
       if (CRASH_ON_PROMPT) process.exit(1)
+      const hold = process.env.MOCK_PROMPT_HOLD
+      if (hold !== undefined) {
+        const released = Promise.withResolvers<undefined>()
+        const check = (): void => { if (!existsSync(hold)) released.resolve(undefined) }
+        const watcher = watch(dirname(hold), check)
+        watcher.on('error', released.reject)
+        try {
+          check()
+          await released.promise
+        } finally {
+          watcher.close()
+        }
+      }
       if (WANT_PERMISSION) {
         // Ask the client to approve before answering; honor its decision. Under
         // MOCK_NO_ALLOW the only options are reject-shaped, so an `allow`-policy

+ 6 - 1
scripts/ci-workflow.spec.ts

@@ -43,11 +43,16 @@ describe('CI workflow', () => {
       if (!Array.isArray(job.steps)) throw new TypeError(`${jobName} must define steps`)
       expect(job.steps[0]).toEqual({
         name: 'Use runner-owned temporary storage',
-        run: 'echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"',
+        run: [
+          'echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"',
+          'echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"',
+          '',
+        ].join('\n'),
       })
       for (const step of job.steps) {
         if (isRecord(step) && isRecord(step.env)) {
           expect(step.env.TMPDIR).toBeUndefined()
+          expect(step.env.npm_config_cache).toBeUndefined()
         }
       }
     },

+ 33 - 0
scripts/tests/ci-release-selfhosted.spec.ts

@@ -9,6 +9,7 @@ const root = resolve(import.meta.dirname, '../..')
 const repository = 'deepseek-harness/deepseek-harness'
 const selfhosted = ['self-hosted', 'linux', 'x64', 'vm-backup']
 const hosted = 'ubuntu-24.04'
+const npmCacheExport = 'echo "npm_config_cache=${{ runner.temp }}/npm-cache" >> "$GITHUB_ENV"'
 
 interface Step {
   name?: string
@@ -38,6 +39,14 @@ function evaluate(expression: string, context: Record<string, string | boolean>)
   return runInNewContext(source, { fromJSON: JSON.parse }, { timeout: 1000 }) as unknown
 }
 
+function assertEarlyNpmCacheExport(steps: Step[]): void {
+  const cacheIndex = steps.findIndex(step => step.run?.split('\n').includes(npmCacheExport))
+  const pnpmIndex = steps.findIndex(step => step.uses?.startsWith('pnpm/') || /\bpnpm\b/.test(step.run ?? ''))
+  expect(cacheIndex).toBeGreaterThanOrEqual(0)
+  expect(cacheIndex).toBeLessThan(pnpmIndex)
+  expect(steps[cacheIndex]?.if).toBeUndefined()
+}
+
 const trustedPr = {
   'vars.DSH_CI_FAILOVER_LINUX': 'selfhosted',
   'github.repository': repository,
@@ -108,6 +117,30 @@ for (const [file, jobIds] of [['release.yml', ['dependencies', 'pack']], ['relea
             .toBe('${{ runner.temp }}/setup-pnpm-${{ github.run_id }}-${{ github.run_attempt }}-${{ github.job }}')
           expect(job.steps.find(step => step.name === 'Install (immutable)')?.run).toBe('pnpm install --frozen-lockfile')
         })
+        it('exports a runner-private npm cache before package-manager setup', () => {
+          assertEarlyNpmCacheExport(job.steps)
+        })
+        it.each([
+          ['missing', ''],
+          ['shared home', 'echo "npm_config_cache=$HOME/.npm" >> "$GITHUB_ENV"'],
+          ['step-local', 'export npm_config_cache="${{ runner.temp }}/npm-cache"'],
+        ])('rejects a %s npm cache export', (_name, replacement) => {
+          const steps = job.steps.map(step => step.run === undefined
+            ? step
+            : { ...step, run: step.run.replace(npmCacheExport, replacement) })
+          expect(() => { assertEarlyNpmCacheExport(steps) }).toThrow()
+        })
+        it('rejects a conditional npm cache export', () => {
+          const steps = job.steps.map(step => step.run?.includes(npmCacheExport) ? { ...step, if: 'false' } : step)
+          expect(() => { assertEarlyNpmCacheExport(steps) }).toThrow()
+        })
+        it('rejects an npm cache export after package-manager setup', () => {
+          const steps = job.steps.map(step => step.run === undefined
+            ? step
+            : { ...step, run: step.run.replace(npmCacheExport, '') })
+          steps.push({ run: npmCacheExport })
+          expect(() => { assertEarlyNpmCacheExport(steps) }).toThrow()
+        })
         it('uses the persistent store without remote cache reads or writes on self-hosted', () => {
           expect(job.steps.find(step => step.name === 'Configure pnpm store path')?.run).toContain('store_root="$HOME/.local/share/pnpm/store"')
           const caches = job.steps.filter(step => step.uses?.startsWith('actions/cache'))

+ 3 - 0
snapshots/session/subagent-acp-diagnostic/cordis.snapshot.yml

@@ -5,6 +5,8 @@
   disabled: true
 
 - insert:
+    - id: acp-diagnostic-release-on-job-output
+      name: './release-on-job-output.mjs'
     - id: llm-replay
       name: '@deepseek-ai/dsh-llm-replay'
       config:
@@ -25,6 +27,7 @@
         env:
           MOCK_PERMISSION: '1'
           MOCK_TOOL_KIND: execute
+          MOCK_PROMPT_HOLD: .dsh/acp-diagnostic-prompt-hold
     - id: tool-subagent-acp-diagnostic
       name: '@deepseek-ai/dsh-tool-subagent'
       config:

+ 3 - 0
snapshots/session/subagent-acp-diagnostic/cordis.yml

@@ -2,6 +2,8 @@
 # execute permission returns `cancelled` and exercises diagnostic-bearing
 # remote-abort parity.
 - insert:
+    - id: acp-diagnostic-release-on-job-output
+      name: './release-on-job-output.mjs'
     - id: subagent-acp-diagnostic
       name: '@deepseek-ai/dsh-subagent-acp'
       config:
@@ -13,6 +15,7 @@
         env:
           MOCK_PERMISSION: '1'
           MOCK_TOOL_KIND: execute
+          MOCK_PROMPT_HOLD: .dsh/acp-diagnostic-prompt-hold
     - id: tool-subagent-acp-diagnostic
       name: '@deepseek-ai/dsh-tool-subagent'
       config:

+ 31 - 0
snapshots/session/subagent-acp-diagnostic/release-on-job-output.mjs

@@ -0,0 +1,31 @@
+/** Hold the scripted ACP background response until job_output owns its wait. */
+import { rmSync, writeFileSync } from 'node:fs'
+
+export const name = 'acp-diagnostic-release-on-job-output'
+export const inject = ['tools', 'jobs']
+
+const HOLD = '.dsh/acp-diagnostic-prompt-hold'
+
+/** Install the scenario-local response barrier without changing tool output. */
+export function apply(ctx) {
+  ctx.on('tools/execute', (exec, next) => {
+    if (exec.name === 'subagent_acp' && exec.arguments.run_in_background === true) {
+      writeFileSync(HOLD, '', { flag: 'wx' })
+    }
+    return next()
+  })
+  ctx.effect(() => {
+    const jobs = ctx.jobs
+    const wait = jobs.wait
+    jobs.wait = function (...args) {
+      // The registry registers its waiter synchronously, unlike the tool middleware.
+      const pending = wait.apply(this, args)
+      rmSync(HOLD)
+      return pending
+    }
+    return () => {
+      jobs.wait = wait
+      rmSync(HOLD, { force: true })
+    }
+  })
+}