Browse Source

Merge origin/master into worktree/ci-under-minute

Tianyi Cui 1 month ago
parent
commit
c0831f0e99

+ 1 - 1
.agents/notes/implemented/process/2026-07-06-parallel-pre-push-gates.md

@@ -20,7 +20,7 @@ The build gate makes the hook self-contained from a clean worktree. `publint`, `
 
 [scripts/publint-all.ts](../../../../scripts/publint-all.ts) discovers the package list from `packages/<group>/<pkg>` and calls publint's supported API against an in-memory view of each manifest's declared publication files plus npm's mandatory metadata files. That keeps unpublished workspace files invisible to publint without a package-manager subprocess per package. A worker pool sized from `availableParallelism()` bounds parallel file loading and linting; `DSH_PUBLINT_CONCURRENCY` can cap or raise it, and results print in deterministic package order.
 
-The aggregate package scripts remain the source of truth for ad hoc local runs. The scheduler is a parallel execution plan over their member gates, not a replacement vocabulary.
+The per-gate package scripts remain the vocabulary for ad hoc local runs. `hygiene` stays an aggregate `&&` chain the scheduler mirrors, while `doc-sync` has since moved its member list into the scheduler itself ([doc-sync through the gate scheduler](2026-07-21-doc-sync-through-gate-scheduler.md)).
 
 ## Alternatives considered
 

+ 6 - 0
.agents/notes/implemented/process/2026-07-21-doc-sync-through-gate-scheduler.i18n.yaml

@@ -0,0 +1,6 @@
+# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
+# side as of the last confirmed-consistent state. Both languages carry equal authority;
+# after editing either side, bring the other along and re-record with:
+#   pnpm run verify-translation-pairing --write
+2026-07-21-doc-sync-through-gate-scheduler.md: b79df2dd7d3515cb0434ac672f7f87c3271d900b
+2026-07-21-doc-sync-through-gate-scheduler.zh.md: 9395244e3c7700166ad87c49219073210c66bc7e

+ 25 - 0
.agents/notes/implemented/process/2026-07-21-doc-sync-through-gate-scheduler.md

@@ -0,0 +1,25 @@
+# Agent Note: doc-sync through the gate scheduler
+
+Status: implemented
+
+English | [中文](2026-07-21-doc-sync-through-gate-scheduler.zh.md)
+
+## Problem
+
+`pnpm run doc-sync` was a `&&` chain of 24 `pnpm run` subcommands. Each link paid a full pnpm wrapper start (workspace resolution, script lookup, tsx boot) before its script ran; measured on a development host, the 24 script bodies together finish in about 34 seconds while the chained form takes around 3 minutes, and the wrapper stall reproduces on local disk, so every developer and CI lane pays it, not just network-filesystem checkouts. The chain also ran serially even though the member gates are read-only and independent, and it silently drifted from [scripts/run-gates.ts](../../../../scripts/run-gates.ts): `verify-cordis-api` joined the chain when the runtime API catalog landed but was never added to `docSyncLeafGates`, so CI and pre-push never enforced that catalog's freshness.
+
+## Decision
+
+`doc-sync` in `package.json` now delegates to the existing bounded scheduler — `tsx scripts/run-gates.ts doc-sync` — the same way `check:pre-push` and the `check:ci:*` scripts already do ([parallel pre-push gates](2026-07-06-parallel-pre-push-gates.md), [parallel GitHub CI gates](2026-07-06-parallel-github-ci-gates.md)). The new `doc-sync` mode expands to exactly `docSyncLeafGates()`, making the leaf list in `run-gates.ts` the single source of truth for the member set; the chain that could drift from it is gone. Like `pre-push`, the mode caps default concurrency at four workers because several doc gates each build a full `ts.Program`; `DSH_GATE_CONCURRENCY` still overrides.
+
+The drift this consolidation surfaced is fixed in the same change: `docSyncLeafGates` gains the missing `verify-cordis-api` leaf, so CI and pre-push now gate the generated runtime API catalog alongside the other generated docs.
+
+## Alternatives considered
+
+- **Keep the `&&` chain and only fix the missing leaf** — repairs today's drift but keeps two member lists that will drift again, and keeps the 24 serial pnpm wrapper starts.
+- **A dedicated `scripts/doc-sync.ts` importing each verify module in one process** — saves even the per-gate tsx boot, but requires refactoring all 24 scripts from run-at-import to callable entry points and loses the scheduler's per-gate timing, isolation, and failure grouping; the wrapper start the scheduler already avoids is the dominant cost.
+- **Shell loop over `tsx scripts/*.ts`** — avoids pnpm wrapper starts cheaply but adds a second execution vocabulary next to the scheduler CI already uses, with none of its scheduling or reporting.
+
+## Consequences
+
+One `pnpm run doc-sync` now costs one pnpm wrapper start plus the slowest dependency chain of member gates instead of 24 wrapper starts plus the sum of all members. Adding a doc gate is one edit in `docSyncLeafGates` (plus the package script itself for ad hoc runs); `package.json` keeps the per-gate `verify-*` scripts as the vocabulary for running one gate by hand. The scheduler prints per-gate timing, so a slow doc-sync points at the gate that dominates. `pnpm run doc-sync` output is now interleaved scheduler output rather than sequential per-command output; anything parsing that output must key on the `run-gates:` summary lines.

+ 25 - 0
.agents/notes/implemented/process/2026-07-21-doc-sync-through-gate-scheduler.zh.md

@@ -0,0 +1,25 @@
+# Agent Note: doc-sync 走门禁调度器
+
+Status: implemented
+
+[English](2026-07-21-doc-sync-through-gate-scheduler.md) | 中文
+
+## 问题
+
+`pnpm run doc-sync` 原本是把 24 个 `pnpm run` 子命令用 `&&` 串起来的链。每一环都要先付一次完整的 pnpm 包装层启动(workspace 解析、脚本查找、tsx 启动)才轮到脚本本体;在开发机上实测,24 个脚本本体合计约 34 秒即可跑完,而链式形态耗时约 3 分钟,且包装层的停顿在本地磁盘上同样复现,因此每位开发者和每条 CI 车道都在付这笔开销,并非只有网络文件系统上的检出受影响。这条链还是串行执行的,尽管各成员门禁只读且相互独立;它也在悄悄偏离 [scripts/run-gates.ts](../../../../scripts/run-gates.ts):运行时 API 目录落地时 `verify-cordis-api` 加入了链,却从未加进 `docSyncLeafGates`,导致 CI 和 pre-push 从未把关该目录的新鲜度。
+
+## 决策
+
+`package.json` 中的 `doc-sync` 现在委托给既有的有界调度器——`tsx scripts/run-gates.ts doc-sync`——与 `check:pre-push` 和各 `check:ci:*` 脚本的做法一致([并行 pre-push 门禁](2026-07-06-parallel-pre-push-gates.md)、[并行 GitHub CI 门禁](2026-07-06-parallel-github-ci-gates.md))。新增的 `doc-sync` 模式恰好展开为 `docSyncLeafGates()`,使 `run-gates.ts` 里的叶子列表成为成员集合的唯一真源;那条可能与之漂移的链不复存在。与 `pre-push` 一样,该模式把默认并发上限设为四个 worker,因为多个文档门禁各自要构建完整的 `ts.Program`;`DSH_GATE_CONCURRENCY` 仍可覆盖。
+
+这次整合暴露出的漂移在同一变更中修复:`docSyncLeafGates` 补上缺失的 `verify-cordis-api` 叶子,CI 和 pre-push 从此与其他生成文档一起把关生成的运行时 API 目录。
+
+## 考虑过的替代方案
+
+- **保留 `&&` 链,只补缺失的叶子**——能修好今天的漂移,但保留了两份还会再漂移的成员列表,也保留了 24 次串行的 pnpm 包装层启动。
+- **专门的 `scripts/doc-sync.ts` 在单进程内 import 各校验模块**——连每个门禁的 tsx 启动也能省掉,但需要把全部 24 个脚本从 import 即执行改造成可调用入口,还会失去调度器的按门禁计时、隔离和失败分组;而调度器已经避免的包装层启动才是开销的大头。
+- **用 shell 循环跑 `tsx scripts/*.ts`**——以低成本避开 pnpm 包装层启动,却在 CI 已经使用的调度器旁边增加了第二套执行词汇,且没有它的任何调度与报告能力。
+
+## 结果
+
+一次 `pnpm run doc-sync` 的成本从 24 次包装层启动加全部成员之和,变为一次包装层启动加成员门禁中最慢的依赖链。新增文档门禁只需在 `docSyncLeafGates` 改一处(外加 package script 本身以便手工单独运行);`package.json` 保留各 `verify-*` 脚本作为手工运行单个门禁的词汇。调度器输出按门禁计时,doc-sync 变慢时能直接指向占大头的门禁。`pnpm run doc-sync` 的输出从逐命令顺序输出变为调度器的交错输出;解析该输出的工具必须以 `run-gates:` 摘要行为准。

+ 1 - 1
AGENTS.md

@@ -57,7 +57,7 @@ pnpm run lint
 pnpm run duplication    # cross-file TypeScript clone detection
 pnpm run build          # tsc emits lib/types, tsdown bundles runtime
 pnpm run hygiene        # knip + publint + workspace constraints + NodeNext consumer check
-pnpm run doc-sync       # all documentation gates; see the doc-sync script in package.json
+pnpm run doc-sync       # all documentation gates; see the doc-sync leaf list in scripts/run-gates.ts
 pnpm run website:build  # VitePress build (doubles as the site's dead-link check)
 pnpm run demo:headless "task" # one-shot agent (needs DEEPSEEK_API_KEY)
 pnpm run demo:tui       # full-screen TUI coding agent (needs DEEPSEEK_API_KEY)

+ 2 - 2
docs/development.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write
-development.md: 3327d094a31ad9af62a23c9562cdfa03218961a5
-development.zh.md: cb1cb86f9f3a34c455844467a20dfdfd08e15098
+development.md: f0db7fbcb4a9df98e83d6c1edd5610e5cc4dd517
+development.zh.md: 62e16479a49d5548e1fbd773dabca5bd741a24fe

+ 1 - 1
docs/development.md

@@ -90,7 +90,7 @@ pnpm run verify-md-wrap  # fail on hard-wrapped prose paragraphs in docs/README
 pnpm run verify-mermaid  # fail if a ```mermaid diagram has invalid Mermaid syntax
 pnpm run verify-type-equiv  # fail if a ```ts type-equiv doc block drifts from its source type
 pnpm run verify-doc-budgets  # fail if a budgeted standing doc exceeds its word ceiling
-pnpm run doc-sync       # all Markdown/doc gates; see the doc-sync script in package.json for the full list
+pnpm run doc-sync       # all Markdown/doc gates, scheduled concurrently; the doc-sync leaf list in scripts/run-gates.ts is the full list
 pnpm run gen-module-graph     # regenerate docs/module-graph.md from package peerDeps
 pnpm run verify-module-graph  # fail if docs/module-graph.md is stale
 pnpm run build          # emit lib/types intermediates, then bundle lib/index.* runtime files

+ 1 - 1
docs/development.zh.md

@@ -90,7 +90,7 @@ pnpm run verify-md-wrap  # fail on hard-wrapped prose paragraphs in docs/README
 pnpm run verify-mermaid  # fail if a ```mermaid diagram has invalid Mermaid syntax
 pnpm run verify-type-equiv  # fail if a ```ts type-equiv doc block drifts from its source type
 pnpm run verify-doc-budgets  # fail if a budgeted standing doc exceeds its word ceiling
-pnpm run doc-sync       # all Markdown/doc gates; see the doc-sync script in package.json for the full list
+pnpm run doc-sync       # all Markdown/doc gates, scheduled concurrently; the doc-sync leaf list in scripts/run-gates.ts is the full list
 pnpm run gen-module-graph     # regenerate docs/module-graph.md from package peerDeps
 pnpm run verify-module-graph  # fail if docs/module-graph.md is stale
 pnpm run build          # emit lib/types intermediates, then bundle lib/index.* runtime files

+ 1 - 1
package.json

@@ -78,7 +78,7 @@
     "verify-scoped-events": "tsx scripts/gen-scoped-events.ts --check",
     "verify-module-graph": "tsx scripts/gen-module-graph.ts --check",
     "constraints": "tsx scripts/check-workspace-constraints.ts",
-    "doc-sync": "pnpm run doc-typecheck && pnpm run verify-cordis-catalog && pnpm run verify-cordis-api && pnpm run verify-export-jsdoc && pnpm run verify-tool-catalog && pnpm run verify-config-catalog && pnpm run verify-persistence-catalog && pnpm run verify-doc-graphs && pnpm run verify-scoped-events && pnpm run verify-md-wrap && pnpm run verify-md-links && pnpm run verify-doc-refs && pnpm run verify-package-paths && pnpm run verify-package-readme-model-experience && pnpm run verify-mermaid && pnpm run verify-agent-note-classification && pnpm run verify-agent-note-format && pnpm run verify-type-equiv && pnpm run verify-translation-prompt && pnpm run verify-translation-pairing && pnpm run verify-doc-budgets && pnpm run verify-package-readme-limitations && pnpm run docs:check",
+    "doc-sync": "tsx scripts/run-gates.ts doc-sync",
     "hygiene": "pnpm run knip && pnpm run publint && pnpm run constraints && pnpm run verify-package-invariants && pnpm run verify-built-package-invariants && pnpm run verify-cordis-config && pnpm run verify-node-next-types && pnpm run verify-runtime-closure",
     "demo:headless": "node --expose-internals --import tsx packages/examples/cli-demo/src/bin.ts --config examples/headless-agent/cordis.yml",
     "demo:tui": "node --expose-internals --import tsx packages/examples/tui-demo/src/bin.ts examples/tui-agent/cordis.yml",

+ 12 - 4
scripts/run-gates.ts

@@ -20,6 +20,7 @@ type Mode =
   | 'ci-artifacts'
   | 'node-compat'
   | 'pre-push'
+  | 'doc-sync'
 type GateStatus = 'pending' | 'running' | 'passed' | 'failed' | 'skipped'
 
 interface Gate {
@@ -89,21 +90,25 @@ function parseMode(raw: string | undefined): Mode {
     case 'ci-artifacts':
     case 'node-compat':
     case 'pre-push':
+    case 'doc-sync':
       return raw
     default:
       throw new Error(
-        `run-gates: expected mode ci-primary | ci-static | ci-lint | ci-coverage | ci-snapshot | ci-artifacts | node-compat | pre-push, got ${JSON.stringify(raw)}.`,
+        `run-gates: expected mode ci-primary | ci-static | ci-lint | ci-coverage | ci-snapshot | ci-artifacts | node-compat | pre-push | doc-sync, got ${JSON.stringify(raw)}.`,
       )
   }
 }
 
 function defaultConcurrency(selectedMode: Mode, total: number): ConcurrencyDefault {
   const available = availableParallelism()
-  const modeLimit = selectedMode === 'pre-push' ? Math.min(4, available) : available
+  // Local modes cap workers: several doc gates each build a full ts.Program,
+  // so an uncapped default on a large host trades wall clock for memory blowups.
+  const localCap = selectedMode === 'pre-push' || selectedMode === 'doc-sync'
+  const modeLimit = localCap ? Math.min(4, available) : available
   return {
     workers: Math.min(total, modeLimit),
-    source: selectedMode === 'pre-push'
-      ? `${available} available CPU(s), pre-push cap 4`
+    source: localCap
+      ? `${available} available CPU(s), ${selectedMode} cap 4`
       : `${available} available CPU(s)`,
   }
 }
@@ -199,6 +204,8 @@ function gatesForMode(selected: Mode): Gate[] {
         }),
         pnpmScript('module-graph', 'verify-module-graph', { label: 'module graph' }),
       ]
+    case 'doc-sync':
+      return docSyncLeafGates()
   }
 }
 
@@ -363,6 +370,7 @@ function docSyncLeafGates(options: {
   return [
     pnpmScript('doc-typecheck', 'doc-typecheck', docTypecheckOptions),
     pnpmScript('cordis-catalog', 'verify-cordis-catalog', { label: 'cordis catalog' }),
+    pnpmScript('cordis-api', 'verify-cordis-api', { label: 'cordis api' }),
     pnpmScript('export-jsdoc', 'verify-export-jsdoc', { label: 'export jsdoc' }),
     pnpmScript('tool-catalog', 'verify-tool-catalog', { label: 'tool catalog' }),
     pnpmScript('config-catalog', 'verify-config-catalog', { label: 'config catalog' }),