Browse Source

test(runtime): isolate busy-meter decisions from worker startup

Tianyi Cui 3 tuần trước cách đây
mục cha
commit
da1df501b8

+ 2 - 2
.agents/notes/implemented/testing/2026-09-08-ci-completion-observations.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-09-08-ci-completion-observations.md
-2026-09-08-ci-completion-observations.md: 51f31f40ebf0bdf1cd79297423200ae1f1a37c57
-2026-09-08-ci-completion-observations.zh.md: bcf4208c503a78d4b6a06055874cab4af196c0ef
+2026-09-08-ci-completion-observations.md: 76f73b4c31a2e61738b91daeb0e65a4b25422047
+2026-09-08-ci-completion-observations.zh.md: aa260528e0989899785338e6b7fedcd6581edb52

+ 2 - 0
.agents/notes/implemented/testing/2026-09-08-ci-completion-observations.md

@@ -34,6 +34,8 @@ The [whole-queue steering test](../../../../apps/web/tests/steering.e2e.ts) wait
 
 The [workspace-management test](../../../../apps/web/tests/workspace-management.e2e.ts) waits for restored composer focus before the next directory-dialog gesture, because workspace listing can render before Session restoration changes focus. Archive actions stay scoped to the seeded row rather than a captured fallback title that cold loading can replace; durable archive assertions retain the exact seed id.
 
+The [worker budget tests](../../../../packages/code-runtime/code-runtime-worker-thread/tests/budget.spec.ts) retain real worker execution and binding transport while controlling only host timers and event-loop-utilization samples. They prove that idle binding time can exceed the compute allowance, active time still expires with a binding pending, and the independent wall ceiling still expires. Real measured hot-loop tests remain; a small total active-time allowance cannot isolate idle accounting from worker bootstrap and host scheduling.
+
 ### Built-client import classification
 
 The [master Windows run](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34204779455/job/101996934534) also rejects the intentional CSS import exposed by `ui-dockkit`. The [Node import sweep](../../../../packages/experimental/webworker-runtime/tests/compile/transform-corpus-check.ts) admits that exact bundle only when Node reports `ERR_UNKNOWN_FILE_EXTENSION` for its `dockkit.module.css`. Other errors at the same entry still fail, and an entry that imports successfully reports a stale exemption. This preserves the import sweep without requiring a browser-only component library to load its stylesheet under bare Node.

+ 2 - 0
.agents/notes/implemented/testing/2026-09-08-ci-completion-observations.zh.md

@@ -34,6 +34,8 @@ Status: implemented
 
 [工作区管理测试](../../../../apps/web/tests/workspace-management.e2e.ts)在下一次目录对话框操作前等待恢复后的输入框焦点,因为工作区列表可能早于 Session 恢复引发的焦点变化完成渲染。归档操作始终限定在种子行内,不依赖冷加载可能替换的已捕获备用标题;持久化归档断言保留精确种子 id。
 
+[worker 预算测试](../../../../packages/code-runtime/code-runtime-worker-thread/tests/budget.spec.ts)保留真实 worker 执行和绑定传输,只控制宿主定时器与事件循环利用率样本。测试证明空闲绑定时间可以超过计算额度、绑定待完成时活动时间仍会触发到期,以及独立壁钟上限仍会触发到期。真实计量的热循环测试仍保留;较小的总活动时间额度无法将空闲计费与 worker 启动及宿主调度隔离。
+
 ### 已构建客户端的导入分类
 
 [master Windows 运行](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34204779455/job/101996934534)还拒绝了 `ui-dockkit` 有意暴露的 CSS 导入。[Node 导入检查](../../../../packages/experimental/webworker-runtime/tests/compile/transform-corpus-check.ts)仅在 Node 针对其 `dockkit.module.css` 报告 `ERR_UNKNOWN_FILE_EXTENSION` 时,才允许这个精确的 bundle。相同入口的其他错误仍然失败,成功导入则报告豁免已过期。这保留了导入检查,同时不要求仅面向浏览器的组件库在裸 Node 中加载样式表。

+ 2 - 2
packages/code-runtime/code-runtime-worker-thread/README.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-worker-thread/README.md
-README.md: c26f349762ab2dac9956099ab637cde9d6d93771
-README.zh.md: 39885e64de0b98cfefe20f8e1b5cd362d4404eea
+README.md: 1350a71f3cd6e94f185dfb830365ca9ddd423e8a
+README.zh.md: db74da4639293a857bca15dc53cea0b1d5e4e157

+ 2 - 0
packages/code-runtime/code-runtime-worker-thread/README.md

@@ -86,6 +86,8 @@ Model code can reach `parentPort` and forge traffic, so every inbound message is
 
 Two independent budgets exist because the peer is hostile: `computeMs` meters the worker's measured busy time (`eventLoopUtilization()` polling every 25 ms), so a hot loop expires it whether or not a decoy dispatch is in flight, while a program idling on a slow binding accrues nothing; `maxWallMs` backstops what busy time cannot see, such as a promise nobody resolves. Both funnel into `worker.terminate()`. `maxWallMs` is range-checked at load against `MAX_TIMER_DELAY_MS` because `setTimeout` clamps a longer delay to 1 ms.
 
+[Budget tests](tests/budget.spec.ts) control host clocks and measured ELU input while keeping worker execution and binding messages real. [Runtime tests](tests/runtime.spec.ts) separately exercise hot-loop containment with actual measurements.
+
 ### Output ledger
 
 `maxOutputBytes` accounts the JSON serialization of the outer `logs` array plus the completion value or failure-message payload; fixed `CodeRunResult` field names and envelope syntax are outside that ledger. At or below the cap the exact value returns; a lossy completion is `invalid-output`, and a combined overflow is `output-limit` rather than a substituted inspected string. The failure retains a fitting captured prefix of the logs.

+ 2 - 0
packages/code-runtime/code-runtime-worker-thread/README.zh.md

@@ -86,6 +86,8 @@ kind: "package-reference"
 
 存在两个独立预算,因为对端不可信:`computeMs` 计量 worker 的实测忙碌时间(每 25 ms 轮询一次 `eventLoopUtilization()`),因此热循环无论是否有诱饵 dispatch 在途都会到期,而等待慢绑定的程序不累计;`maxWallMs` 为忙碌时间无法观测的情况兜底,例如永远不会 resolve 的 promise。二者最终都会调用 `worker.terminate()`。`maxWallMs` 在加载时对照 `MAX_TIMER_DELAY_MS` 做范围校验,因为 `setTimeout` 会把更长的延迟限制为 1 ms。
 
+[预算测试](tests/budget.spec.ts)控制宿主时钟与实测 ELU 输入,同时保留真实 worker 执行和绑定消息。[运行时测试](tests/runtime.spec.ts)独立使用真实测量验证热循环约束。
+
 ### 输出账本
 
 `maxOutputBytes` 统计外层 `logs` 数组加完成值或失败消息载荷的 JSON 序列化;固定的 `CodeRunResult` 字段名与信封语法不计入这份账本。未超过上限时返回精确值;有损完成值属于 `invalid-output`,组合溢出属于 `output-limit`,不会用 inspected string 代替。失败会保留日志中能容纳的已捕获前缀。

+ 95 - 0
packages/code-runtime/code-runtime-worker-thread/tests/budget.spec.ts

@@ -0,0 +1,95 @@
+/** Host budget decisions use controlled clocks and ELU samples; worker execution and binding transport stay real. */
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+import { Context } from '@deepseek-ai/cordis'
+import { WorkerThreadCodeRuntime } from '@deepseek-ai/dsh-code-runtime-worker-thread'
+import type { CodeRunResult } from '@deepseek-ai/dsh-code-runtime'
+
+const meter = vi.hoisted(() => ({ sample: vi.fn() }))
+
+vi.mock('node:worker_threads', async (importOriginal) => {
+  const original = await importOriginal<typeof import('node:worker_threads')>()
+  return {
+    ...original,
+    Worker: class extends original.Worker {
+      constructor(...args: ConstructorParameters<typeof original.Worker>) {
+        super(...args)
+        this.performance.eventLoopUtilization = meter.sample
+      }
+    },
+  }
+})
+
+describe('worker budgets with controlled ELU samples and real binding transport', () => {
+  let ctx: Context
+  let controller: AbortController
+  let run: Promise<CodeRunResult> | undefined
+  let release: (() => void) | undefined
+
+  beforeEach(() => {
+    ctx = new Context()
+    controller = new AbortController()
+    run = undefined
+    release = undefined
+    meter.sample.mockReset().mockReturnValue({ active: 10, idle: 0, utilization: 1 })
+    // Worker bootstrap and scheduling contribute to ELU active time; only the
+    // measured input and host deadlines are controlled, not worker execution.
+    vi.useFakeTimers({ toFake: ['setTimeout', 'clearTimeout', 'setInterval', 'clearInterval'] })
+  })
+
+  afterEach(async () => {
+    const owned = { ctx, controller, run, release }
+    try {
+      owned.controller.abort('test cleanup')
+      owned.release?.()
+    } finally {
+      vi.useRealTimers()
+    }
+    try {
+      await owned.run
+    } finally {
+      await owned.ctx.fiber.dispose()
+    }
+  })
+
+  async function pendingBinding(): Promise<void> {
+    await ctx.plugin(WorkerThreadCodeRuntime, { computeMs: 1_000, maxWallMs: 30_000 })
+    let entered!: () => void
+    const ready = new Promise<void>((resolve) => { entered = resolve })
+    const binding = new Promise<string>((resolve) => { release = () => { resolve('slow-done') } })
+    run = ctx.codeRuntime.run({
+      program: 'return await tools.slow({})',
+      bindings: [{ global: 'tools', functions: { slow: () => { entered(); return binding } } }],
+      signal: controller.signal,
+    })
+    await Promise.race([
+      ready,
+      run.then((result) => { throw new Error('Worker settled before binding entry: ' + JSON.stringify(result)) }),
+    ])
+  }
+
+  it('does not charge a binding wait longer than the compute budget', async () => {
+    await pendingBinding()
+    const settled = vi.fn()
+    void run!.then(settled, settled)
+    meter.sample.mockReturnValue({ active: 10, idle: 1_500, utilization: 10 / 1_510 })
+    await vi.advanceTimersByTimeAsync(1_500)
+    expect(meter.sample).toHaveBeenCalled()
+    expect(settled).not.toHaveBeenCalled()
+    release!()
+    expect(await run).toEqual({ logs: [], value: 'slow-done' })
+  })
+
+  it('expires active time even while a binding is pending', async () => {
+    await pendingBinding()
+    meter.sample.mockReturnValue({ active: 1_001, idle: 1_500, utilization: 1_001 / 2_501 })
+    await vi.advanceTimersByTimeAsync(25)
+    expect(await run).toEqual({ logs: [], error: { kind: 'timeout', message: 'compute budget exhausted (1000ms busy)' } })
+  })
+
+  it('expires the wall ceiling while active time remains below the compute budget', async () => {
+    await pendingBinding()
+    meter.sample.mockReturnValue({ active: 10, idle: 30_000, utilization: 10 / 30_010 })
+    await vi.advanceTimersByTimeAsync(30_000)
+    expect(await run).toEqual({ logs: [], error: { kind: 'timeout', message: 'wall-clock ceiling reached (30000ms)' } })
+  })
+})