Parcourir la source

docs(testing): state the concurrent execution model where tests are written

The skill carries the reliability rules, but nothing an agent loads by default
said that specs run concurrently at all. The testing policy described tiers and
evidence without ever stating how a spec is executed, and neither subtree
AGENTS.md mentioned it — including scripts/, where the two suites that recently
failed on unrelated branches live.

docs/testing.md gains the execution model as the one home for the fact: forked
workers, concurrent coverage partitions beside other gates, and self-hosted
runners sharing a host and volume, with the rule that a spec passing only when
run alone is a defect in the spec. It links the skill for the detailed rules.

packages/AGENTS.md and scripts/AGENTS.md carry the short actionable form and
link that section, so the rule is present in the context loaded while a test in
either subtree is being written.

Both ceilings are raised for the added words and the targets in docs/AGENTS.md
move with them: docs/testing.md 1150 to 1300 (now 1237) and packages/AGENTS.md
675 to 750 (now 712), each keeping the 5% headroom the standard requires.
Yichen Jiang il y a 1 semaine
Parent
commit
cc5173f4cf

+ 1 - 1
docs/AGENTS.md

@@ -54,7 +54,7 @@ When the gate goes red:
 2. **Condense** content that belongs here but can be shorter.
 3. **Raise** the ceiling only when the words need the space; justify the manifest diff in the PR. A too-low ceiling is a budget bug.
 
-Ceilings are guardrails, not reduction targets. At or below target, retain at least 5% headroom; above target, freeze the ceiling until relocation or condensation brings the document under target. Lower a ceiling only when the document still has room. Targets: root `AGENTS.md` ≤ 1,950; `architecture.md` ≤ 2,400; subtree `AGENTS.md` ≤ 600, except `packages/AGENTS.md` ≤ 675 and this file ≤ 1,320; `packages/README.md` ≤ 994; plus `cordis-primer.md` 600, `defensive-patterns.md` 550, `testing.md` 1,150, `examples/AGENTS.md` 310. Review governs unbudgeted tiers.
+Ceilings are guardrails, not reduction targets. At or below target, retain at least 5% headroom; above target, freeze the ceiling until relocation or condensation brings the document under target. Lower a ceiling only when the document still has room. Targets: root `AGENTS.md` ≤ 1,950; `architecture.md` ≤ 2,400; subtree `AGENTS.md` ≤ 600, except `packages/AGENTS.md` ≤ 750 and this file ≤ 1,320; `packages/README.md` ≤ 994; plus `cordis-primer.md` 600, `defensive-patterns.md` 550, `testing.md` 1,300, `examples/AGENTS.md` 310. Review governs unbudgeted tiers.
 
 ## The slop checklist
 

+ 2 - 2
docs/testing.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write docs/testing.md
-testing.md: c21c67bba85387e35e80d16217c2673653281672
-testing.zh.md: 008bee58918510c8afb695a7e0b880828d698227
+testing.md: 25514702e7aa8649c10ec213f5a6bf5c6e9e9a09
+testing.zh.md: 5338e4bc392e1e1ff16c32f708970be917e6efa7

+ 4 - 0
docs/testing.md

@@ -15,6 +15,10 @@ How this repo tests, tier by tier, and the rules that keep a green suite meaning
 
 Session fixtures keep headers and payloads but omit body sequence/time envelopes. Replay synthesizes them. Fixtures use canonical packed rows; [the migrator](../scripts/migrate-packed-session-fixtures.ts) rewrites old layouts.
 
+## How specs execute
+
+Forked workers run several spec files at once, the coverage gate splits into concurrent partitions beside the other gates in its job, and the self-hosted runners share one host and one volume. Only the process is isolated: ports, predictable paths, external namespaces, and inherited children are not. Own each acquired resource through its teardown, and read a spec that passes only when it runs alone as a defect in the spec rather than an unstable runner. [dsh-ci-test-reliability](../.agents/skills/dsh-ci-test-reliability/SKILL.md) owns the allocation, restoration, synchronization, timeout-budget, platform, and teardown rules; its [flake diagnosis workflow](../.agents/skills/dsh-ci-test-reliability/references/ci-flake-diagnosis.md) classifies an existing probabilistic failure.
+
 ## The with-key policy: inference is cheap here
 
 We are DeepSeek — do not ration real-API tests. A no-key test proves plumbing; only a with-key run proves the agent works against a real model. Cover file-writing prompts, multi-turn conversations, tool use, and mid-stream cancellation. Highest-value are **smoke tests** that boot a shipped `dsh` profile, send one prompt, and check the world — they catch the "green unit tests, broken product" class that mocks cannot ([postmortem 0001](postmortem/0001-acp-default-export-drops-inject.md)). Self-skip keeps secretless CI and keyless contributors unblocked; it is not a cost signal. Profile-level integration tests live under `apps/cli/tests/profiles/`; package-specific compositions stay with their package tests.

+ 4 - 0
docs/testing.zh.md

@@ -15,6 +15,10 @@
 
 会话 fixture 保留 header 与 payload,但省略正文序号/时间 envelope。回放会合成这些字段;运行时持久化不变。fixture 使用规范打包行;[迁移器](../scripts/migrate-packed-session-fixtures.ts)会改写旧布局。
 
+## spec 如何被执行
+
+fork 出的 worker 会同时运行多个 spec 文件,coverage gate 会拆成并发的 partition,与同一个 job 中的其它 gate 并排运行,而自托管 runner 共用同一台宿主机和同一个卷。被隔离的只有进程:端口、可预测路径、外部命名空间和继承而来的子进程都不隔离。为每个占用的资源负责到它的 teardown,并把「只有单独运行时才通过」的 spec 读作该 spec 的缺陷,而不是 runner 不稳定。[dsh-ci-test-reliability](../.agents/skills/dsh-ci-test-reliability/SKILL.md) 负责资源分配、状态恢复、同步、超时预算、平台差异与 teardown 规则;它的 [flake 诊断流程](../.agents/skills/dsh-ci-test-reliability/references/ci-flake-diagnosis.md)用于归类已经存在的概率性失败。
+
 ## 带密钥策略:推理(inference)在这里很便宜
 
 我们是 DeepSeek,不要吝惜真实 API 测试。无密钥测试只能证明底层通路;只有带密钥运行才能证明 agent(智能体)能对接真实模型正常工作。覆盖文件写入提示词、包含多个轮次的对话、工具使用和流中取消。价值最高的是**冒烟测试**:启动已交付的 `dsh` profile、发送一条提示词,并检查外部世界;它们能捕获「单元测试全绿、产品却坏了」这一类 mock 无法发现的问题([事故复盘 0001](postmortem/0001-acp-default-export-drops-inject.zh.md))。自动跳过让无密钥 CI 和无密钥贡献者不受阻塞;它不是成本信号。Profile 级集成测试位于 `apps/cli/tests/profiles/`;包专属组合留在对应包的测试目录中。

+ 1 - 0
packages/AGENTS.md

@@ -15,6 +15,7 @@ These package-specific rules supplement the repo-wide [conventions](../AGENTS.md
 - **Publish state only at its commit point.** Emit each notification and update derived state only after the operation succeeds; derive caches, prompts, UI echoes, replay, and query views from one authoritative source.
 - **Apply bounds to the complete result.** Enforce byte, token, item, and time limits where the complete emitted or retained value, including wrappers and metadata, is known; test tiny and exact limits, oversized single chunks, and multibyte byte limits.
 - **Registry contributions prove disposal** through the HMR-safety test required by [testing policy](../docs/testing.md): dispose the fiber and observe removal.
+- **Specs run concurrently** in forked workers beside other gate processes. Own each acquired port, path, and child process through teardown; a spec that passes only when run alone is a defect in the spec ([execution model](../docs/testing.md#how-specs-execute)).
 - **Every package owns `./invariant`.** Register the manifest name; check an event/data relation or give empty installers package-specific `No runtime invariant:` reasons. Generated companions, unexplained empties, and ignored reporters fail [`verify-package-invariants`](../.agents/notes/implemented/architecture/2026-07-19-package-invariant-runtime-contracts.md).
 
 [Naming rules](../docs/cookbook/adding-a-package.md#name-the-role-that-exists):

+ 2 - 0
scripts/AGENTS.md

@@ -1,3 +1,5 @@
 # AGENTS.md — Repository scripts
 
 Gate scripts invoke pnpm shell-free, normalize repository-relative glob paths to `/` at ingestion, and keep platform adaptation in the gate that needs it instead of a shared platform layer. Source-ownership gates use syntax-aware discovery, guard against an empty or narrowed corpus, and test every admitted/excluded form that changes their detection boundary.
+
+Script specs run in forked workers beside the rest of the suite and beside the other gate processes in their job, so own every port, temporary path, and child process a spec acquires. A spec that passes only when it runs alone is a defect in the spec; [the testing policy](../docs/testing.md#how-specs-execute) states the execution model and [dsh-ci-test-reliability](../.agents/skills/dsh-ci-test-reliability/SKILL.md) owns the rules.

+ 2 - 2
scripts/doc-budgets.manifest.json

@@ -4,7 +4,7 @@
   "docs/architecture.md": 2400,
   "docs/cordis-primer.md": 600,
   "docs/defensive-patterns.md": 550,
-  "docs/testing.md": 1150,
-  "packages/AGENTS.md": 675,
+  "docs/testing.md": 1300,
+  "packages/AGENTS.md": 750,
   "packages/README.md": 994
 }