Explorar el Código

perf(app-boot): avoid fallback locks for complete profiles

Resolve the installation fallback generation before locking and return immediately when every required symlink or packaged proxy is complete. Parallel SDK rollouts sharing an initialized DSH_HOME therefore do not queue on profiles/node_modules.lock.

Missing or stale entries still acquire the cross-process writer lock, recheck the generation, and repair under exclusive ownership. Tests hold the lock to prove the steady-state bypass and verify that a partial repair retains already-correct siblings.
Tianyi Cui hace 1 mes
padre
commit
ab4e65ba82

+ 2 - 2
.agents/notes/implemented/architecture/2026-08-23-python-sdk-dsh-profile-runtime.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-23-python-sdk-dsh-profile-runtime.md
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-23-python-sdk-dsh-profile-runtime.md
-2026-08-23-python-sdk-dsh-profile-runtime.md: 19c870b3b0e10b25480bacc85b9db29b01d2577d
-2026-08-23-python-sdk-dsh-profile-runtime.zh.md: 404798ff4ae1fcafbaa8403c7187297adf374a19
+2026-08-23-python-sdk-dsh-profile-runtime.md: e3df2d01e3aef7e6eadaa011d51c9ab87456d35f
+2026-08-23-python-sdk-dsh-profile-runtime.zh.md: 3d50cac36d469172e91be450cea658f3a4830ccf

+ 1 - 1
.agents/notes/implemented/architecture/2026-08-23-python-sdk-dsh-profile-runtime.md

@@ -32,7 +32,7 @@ The runtime wheel installs a `dsh` console command. Ordinary profile and SDK exe
 
 
 The zero-code deployment manifest is `dsh-python-runtime-closure`. It packages `node_modules/@deepseek-ai/dsh/lib/bin.js` and profile, bundle, preset, native-addon, and shared-library assets into `deepseek-harness-sdk-runtime-<platform>-<arch>`. The wheel distribution names, Python import modules, JSON-RPC messages, and wire-stable `serverInfo.name = deepseek-harness-sdk-runtime` remain unchanged.
 The zero-code deployment manifest is `dsh-python-runtime-closure`. It packages `node_modules/@deepseek-ai/dsh/lib/bin.js` and profile, bundle, preset, native-addon, and shared-library assets into `deepseek-harness-sdk-runtime-<platform>-<arch>`. The wheel distribution names, Python import modules, JSON-RPC messages, and wire-stable `serverInfo.name = deepseek-harness-sdk-runtime` remain unchanged.
 
 
-Plain Node profiles use symlinks in `$DSH_HOME/profiles/node_modules` to share installation packages with external plugins. An operating-system symlink cannot traverse pkg's `/snapshot` filesystem, so the packaged CLI writes small real ESM proxy packages instead. Each proxy resolves the source package's explicit ESM export map directly under Node import conditions, exposes targets that exist in the installation, and re-exports their virtual module URLs. Export rows without an ESM runtime target and executable-only or declaration-only packages produce no unusable proxy entry; malformed export maps fail startup. One cross-process writer lock serializes fallback healing, preventing partial proxy visibility and allowing either carrier to replace the other carrier's managed entry. Loader rows and external plugin peers therefore resolve through the normal profile parent walk while retaining one Cordis and one instance of each bundled module.
+Plain Node profiles use symlinks in `$DSH_HOME/profiles/node_modules` to share installation packages with external plugins. An operating-system symlink cannot traverse pkg's `/snapshot` filesystem, so the packaged CLI writes small real ESM proxy packages instead. Each proxy resolves the source package's explicit ESM export map directly under Node import conditions, exposes targets that exist in the installation, and re-exports their virtual module URLs. Export rows without an ESM runtime target and executable-only or declaration-only packages produce no unusable proxy entry; malformed export maps fail startup. A complete matching generation returns without acquiring the cross-process writer lock. A missing or stale entry acquires the lock, rechecks the generation, and repairs it without exposing partial proxies; either carrier can replace the other carrier's managed entry. Loader rows and external plugin peers therefore resolve through the normal profile parent walk while retaining one Cordis and one instance of each bundled module.
 
 
 The published target set is Linux x64, Linux arm64, and macOS arm64. Installed-wheel black-box CI owns artifact provenance, default and patched profiles, external bundle installation, native tools, MCP, direct JSON-RPC, snapshots, and trusted real-provider turns on every target.
 The published target set is Linux x64, Linux arm64, and macOS arm64. Installed-wheel black-box CI owns artifact provenance, default and patched profiles, external bundle installation, native tools, MCP, direct JSON-RPC, snapshots, and trusted real-provider turns on every target.
 
 

+ 1 - 1
.agents/notes/implemented/architecture/2026-08-23-python-sdk-dsh-profile-runtime.zh.md

@@ -32,7 +32,7 @@ Python SDK 分发一个私有 Node 应用,直接启动完整外部 `cordis.yml
 
 
 零代码部署 manifest 是 `dsh-python-runtime-closure`。它把 `node_modules/@deepseek-ai/dsh/lib/bin.js` 以及 profile、bundle、preset、原生 addon 与共享库资源打包进 `deepseek-harness-sdk-runtime-<platform>-<arch>`。Wheel distribution 名称、Python import 模块、JSON-RPC 消息和协议稳定的 `serverInfo.name = deepseek-harness-sdk-runtime` 保持不变。
 零代码部署 manifest 是 `dsh-python-runtime-closure`。它把 `node_modules/@deepseek-ai/dsh/lib/bin.js` 以及 profile、bundle、preset、原生 addon 与共享库资源打包进 `deepseek-harness-sdk-runtime-<platform>-<arch>`。Wheel distribution 名称、Python import 模块、JSON-RPC 消息和协议稳定的 `serverInfo.name = deepseek-harness-sdk-runtime` 保持不变。
 
 
-普通 Node profile 在 `$DSH_HOME/profiles/node_modules` 中使用符号链接,让外部插件共享安装包。操作系统符号链接无法进入 pkg 的 `/snapshot` 文件系统,因此打包 CLI 改为写入小型真实 ESM 代理包。每个代理直接按 Node import 条件解析源包的显式 ESM exports map,公开安装中实际存在的目标,并重新导出其虚拟模块 URL。没有 ESM 运行时目标的 export 项以及仅含可执行入口或类型声明入口的包不会产生不可用的代理条目;格式错误的 exports map 会导致启动失败。一把跨进程写入锁会串行执行后备修复,避免暴露未完整写入的代理,并允许任一载体替换另一载体留下的受管条目。Loader 配置项和外部插件 peer 因而可以通过普通 profile 逐级向上查找解析,同时保留一个 Cordis 和每个内置模块的单一实例。
+普通 Node profile 在 `$DSH_HOME/profiles/node_modules` 中使用符号链接,让外部插件共享安装包。操作系统符号链接无法进入 pkg 的 `/snapshot` 文件系统,因此打包 CLI 改为写入小型真实 ESM 代理包。每个代理直接按 Node import 条件解析源包的显式 ESM exports map,公开安装中实际存在的目标,并重新导出其虚拟模块 URL。没有 ESM 运行时目标的 export 项以及仅含可执行入口或类型声明入口的包不会产生不可用的代理条目;格式错误的 exports map 会导致启动失败。完整且匹配的 generation 不会获取跨进程写入锁。缺失或过期的配置项会获取该锁、重新检查 generation,并在不暴露半成品代理的前提下修复;任一载体都可以替换另一载体留下的受管配置项。Loader 配置项和外部插件 peer 因而可以通过普通 profile 逐级向上查找解析,同时保留一个 Cordis 和每个内置模块的单一实例。
 
 
 已发布目标集合是 Linux x64、Linux arm64 与 macOS arm64。Installed-wheel 黑盒 CI 在每个目标上负责产物来源、默认及 patched profile、外部 bundle 安装、原生工具、MCP、直接 JSON-RPC、快照,以及可信真实提供方轮次。
 已发布目标集合是 Linux x64、Linux arm64 与 macOS arm64。Installed-wheel 黑盒 CI 在每个目标上负责产物来源、默认及 patched profile、外部 bundle 安装、原生工具、MCP、直接 JSON-RPC、快照,以及可信真实提供方轮次。
 
 

+ 2 - 2
packages/boot/app-boot/README.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write packages/boot/app-boot/README.md
 #   pnpm run verify-translation-pairing --write packages/boot/app-boot/README.md
-README.md: 1fa00eefae367e2a5a44966d2f2debff9f95c074
-README.zh.md: ef57478b8a3de723e4fa8ce88f87d563eb5a81e5
+README.md: 0adcb0ac20516b1eea97792a11471a383a3ab429
+README.zh.md: 1da1725b1e0681f83f2c82b82cb9bbaac8ea90cb

La diferencia del archivo ha sido suprimido porque es demasiado grande
+ 0 - 0
packages/boot/app-boot/README.md


La diferencia del archivo ha sido suprimido porque es demasiado grande
+ 0 - 0
packages/boot/app-boot/README.zh.md


+ 76 - 36
packages/boot/app-boot/src/profile.ts

@@ -407,6 +407,69 @@ function ensureModuleProxy(
   }
   }
 }
 }
 
 
+type ModuleFallbackEntry =
+  | { kind: 'symlink'; packageName: string; packageDir: string }
+  | { kind: 'proxy'; packageName: string; version: string; targets: Record<string, string> }
+
+/** Resolve the installation generation that every profile must find through the fallback directory. */
+function resolveModuleFallbackEntries(installAnchor: string): ModuleFallbackEntry[] {
+  const appManifest = JSON.parse(readFileSync(installAnchor, 'utf8')) as ProfileManifest
+  const links = new Map<string, string>()
+  /* v8 ignore next -- a real app manifest always declares its name */
+  if (appManifest.name !== undefined) links.set(appManifest.name, dirname(installAnchor))
+  // BFS over the resolvable dependency graph; the visited set is the link
+  // map itself (first resolution wins, matching Node's own nearest-wins).
+  const queue: { anchor: string; manifest: ProfileManifest }[] = [{ anchor: installAnchor, manifest: appManifest }]
+  for (let next = queue.shift(); next !== undefined; next = queue.shift()) {
+    // Peer dependencies participate: Service Definition packages (dsh-subprocess,
+    // dsh-compaction, ...) are peers of their implementations, never plain
+    // dependencies, yet out-of-tree plugins import them directly.
+    /* v8 ignore next -- a real app manifest always declares dependencies */
+    for (const dep of [...Object.keys(next.manifest.dependencies ?? {}), ...Object.keys(next.manifest.peerDependencies ?? {})]) {
+      if (links.has(dep)) continue
+      const dir = packageDirFromAnchor(next.anchor, dep)
+      // A declared-but-uninstalled dependency cannot be a loader-visible
+      // plugin; skip it rather than fail the whole boot.
+      if (dir === undefined) continue
+      links.set(dep, dir)
+      const manifestPath = join(dir, 'package.json')
+      queue.push({ anchor: manifestPath, manifest: JSON.parse(readFileSync(manifestPath, 'utf8')) as ProfileManifest })
+    }
+  }
+  if (!isPackagedExecutable()) {
+    return [...links].map(([packageName, packageDir]) => ({ kind: 'symlink', packageName, packageDir }))
+  }
+  return [...links].flatMap(([packageName, packageDir]) => {
+    const source = packageProxySource(packageName, packageDir)
+    return Object.keys(source.targets).length === 0
+      ? []
+      : [{ kind: 'proxy' as const, packageName, version: source.version, targets: source.targets }]
+  })
+}
+
+/** Return whether one existing fallback entry already matches its resolved installation generation. */
+function moduleFallbackEntryCurrent(modulesDir: string, entry: ModuleFallbackEntry): boolean {
+  const link = join(modulesDir, entry.packageName)
+  try {
+    const stat = lstatSync(link)
+    if (entry.kind === 'symlink') {
+      return stat.isSymbolicLink() && readlinkSync(link) === entry.packageDir
+    }
+    if (!stat.isDirectory()) return false
+    const existing = readModuleProxyRecord(link)
+    return existing?.version === entry.version
+      && JSON.stringify(existing.dsh?.moduleFallback?.targets) === JSON.stringify(entry.targets)
+      && Object.keys(entry.targets).every((_, index) => existsSync(join(link, `entry-${index}.js`)))
+  } catch {
+    return false
+  }
+}
+
+/** Return whether every required fallback entry is already ready for this installation. */
+function moduleFallbackCurrent(modulesDir: string, entries: readonly ModuleFallbackEntry[]): boolean {
+  return entries.every(entry => moduleFallbackEntryCurrent(modulesDir, entry))
+}
+
 /**
 /**
  * Maintain the flat module fallback `$DSH_HOME/profiles/node_modules`: one
  * Maintain the flat module fallback `$DSH_HOME/profiles/node_modules`: one
  * entry per package in the dsh app's resolvable dependency CLOSURE (BFS
  * entry per package in the dsh app's resolvable dependency CLOSURE (BFS
@@ -415,8 +478,9 @@ function ensureModuleProxy(
  * exports under ESM import conditions and writes small proxy packages because
  * exports under ESM import conditions and writes small proxy packages because
  * the host filesystem cannot follow a symlink into pkg's virtual `/snapshot`
  * the host filesystem cannot follow a symlink into pkg's virtual `/snapshot`
  * tree; the proxy re-exports the virtual URL, preserving the executable's
  * tree; the proxy re-exports the virtual URL, preserving the executable's
- * single module instance. One cross-process writer lock prevents partial
- * proxies and serializes carrier transitions. Node's
+ * single module instance. A complete matching generation returns without a
+ * writer lock; actual repairs acquire and recheck one cross-process lock so
+ * partial proxies and carrier transitions remain serialized. Node's
  * parent-directory walk from any profile finds this
  * parent-directory walk from any profile finds this
  * directory after the profile's own `node_modules`, so every in-box plugin
  * directory after the profile's own `node_modules`, so every in-box plugin
  * resolves without pnpm ever managing it — the exact "bundles come from the
  * resolves without pnpm ever managing it — the exact "bundles come from the
@@ -431,53 +495,29 @@ function ensureModuleProxy(
  * reused because resolution cannot discover it.
  * reused because resolution cannot discover it.
  * @param installAnchor - absolute path of the dsh app's package.json.
  * @param installAnchor - absolute path of the dsh app's package.json.
  * @param home - the Harness home; defaults to {@link resolveDshHome}.
  * @param home - the Harness home; defaults to {@link resolveDshHome}.
- * @returns settlement after the locked fallback generation is complete.
+ * @returns settlement after current-state validation or a locked repair.
  */
  */
 export async function healProfilesModuleFallback(installAnchor: string, home: string = resolveDshHome()): Promise<void> {
 export async function healProfilesModuleFallback(installAnchor: string, home: string = resolveDshHome()): Promise<void> {
   const profilesDir = join(home, PROFILES_DIR)
   const profilesDir = join(home, PROFILES_DIR)
   const modulesDir = join(profilesDir, 'node_modules')
   const modulesDir = join(profilesDir, 'node_modules')
   mkdirSync(modulesDir, { recursive: true })
   mkdirSync(modulesDir, { recursive: true })
+  const entries = resolveModuleFallbackEntries(installAnchor)
+  if (moduleFallbackCurrent(modulesDir, entries)) return
   await withFileLock(modulesDir, () => {
   await withFileLock(modulesDir, () => {
-    healProfilesModuleFallbackLocked(installAnchor, modulesDir)
+    if (!moduleFallbackCurrent(modulesDir, entries)) healProfilesModuleFallbackLocked(entries, modulesDir)
     return Promise.resolve()
     return Promise.resolve()
   })
   })
 }
 }
 
 
 /** Heal one module-fallback generation while the cross-process writer lock is held. */
 /** Heal one module-fallback generation while the cross-process writer lock is held. */
-function healProfilesModuleFallbackLocked(installAnchor: string, modulesDir: string): void {
-  const appManifest = JSON.parse(readFileSync(installAnchor, 'utf8')) as ProfileManifest
-  const links = new Map<string, string>()
-  /* v8 ignore next -- a real app manifest always declares its name */
-  if (appManifest.name !== undefined) links.set(appManifest.name, dirname(installAnchor))
-  // BFS over the resolvable dependency graph; the visited set is the link
-  // map itself (first resolution wins, matching Node's own nearest-wins).
-  const queue: { anchor: string; manifest: ProfileManifest }[] = [{ anchor: installAnchor, manifest: appManifest }]
-  for (let next = queue.shift(); next !== undefined; next = queue.shift()) {
-    // Peer dependencies participate: Service Definition packages (dsh-subprocess,
-    // dsh-compaction, ...) are peers of their implementations, never plain
-    // dependencies, yet out-of-tree plugins import them directly.
-    /* v8 ignore next -- a real app manifest always declares dependencies */
-    for (const dep of [...Object.keys(next.manifest.dependencies ?? {}), ...Object.keys(next.manifest.peerDependencies ?? {})]) {
-      if (links.has(dep)) continue
-      const dir = packageDirFromAnchor(next.anchor, dep)
-      // A declared-but-uninstalled dependency cannot be a loader-visible
-      // plugin; skip it rather than fail the whole boot.
-      if (dir === undefined) continue
-      links.set(dep, dir)
-      const manifestPath = join(dir, 'package.json')
-      queue.push({ anchor: manifestPath, manifest: JSON.parse(readFileSync(manifestPath, 'utf8')) as ProfileManifest })
-    }
-  }
-  for (const [packageName, target] of links) {
-    const link = join(modulesDir, packageName)
+function healProfilesModuleFallbackLocked(entries: readonly ModuleFallbackEntry[], modulesDir: string): void {
+  for (const entry of entries) {
+    const link = join(modulesDir, entry.packageName)
     mkdirSync(dirname(link), { recursive: true })
     mkdirSync(dirname(link), { recursive: true })
-    if (isPackagedExecutable()) {
-      const source = packageProxySource(packageName, target)
-      if (Object.keys(source.targets).length > 0) {
-        ensureModuleProxy(link, packageName, source.version, source.targets)
-      }
+    if (entry.kind === 'proxy') {
+      ensureModuleProxy(link, entry.packageName, entry.version, entry.targets)
     } else {
     } else {
-      ensureSymlink(link, target)
+      ensureSymlink(link, entry.packageDir)
     }
     }
   }
   }
 }
 }

+ 40 - 1
packages/boot/app-boot/tests/profile.spec.ts

@@ -4,7 +4,7 @@
  * empty-root composition, and the installation module-fallback healing.
  * empty-root composition, and the installation module-fallback healing.
  */
  */
 
 
-import { existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readlinkSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
+import { existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readlinkSync, rmSync, symlinkSync, unlinkSync, writeFileSync } from 'node:fs'
 import { tmpdir } from 'node:os'
 import { tmpdir } from 'node:os'
 import { join } from 'node:path'
 import { join } from 'node:path'
 import { withFileLock } from '@deepseek-ai/dsh-atomic-write'
 import { withFileLock } from '@deepseek-ai/dsh-atomic-write'
@@ -321,6 +321,20 @@ describe('healProfilesModuleFallback', () => {
     expect(readlinkSync(join(fallback, 'dsh-app'))).toContain('app')
     expect(readlinkSync(join(fallback, 'dsh-app'))).toContain('app')
   })
   })
 
 
+  it('retains current links while repairing a missing sibling', async () => {
+    const anchor = stageInstallation({ 'bundle-a': { patch: '[]\n' } })
+    const home = tmp()
+    const fallback = join(home, 'profiles', 'node_modules')
+    await healProfilesModuleFallback(anchor, home)
+    const appTarget = readlinkSync(join(fallback, 'dsh-app'))
+    unlinkSync(join(fallback, 'bundle-a'))
+
+    await healProfilesModuleFallback(anchor, home)
+
+    expect(readlinkSync(join(fallback, 'dsh-app'))).toBe(appTarget)
+    expect(lstatSync(join(fallback, 'bundle-a')).isSymbolicLink()).toBe(true)
+  })
+
   it('serializes concurrent healers and retains the identical link', async () => {
   it('serializes concurrent healers and retains the identical link', async () => {
     const anchor = stageInstallation({})
     const anchor = stageInstallation({})
     const home = tmp()
     const home = tmp()
@@ -332,6 +346,31 @@ describe('healProfilesModuleFallback', () => {
     expect(lstatSync(join(fallback, 'dsh-app')).isSymbolicLink()).toBe(true)
     expect(lstatSync(join(fallback, 'dsh-app')).isSymbolicLink()).toBe(true)
   })
   })
 
 
+  it('does not acquire the writer lock for a complete generation', async () => {
+    const anchor = stageInstallation({})
+    const home = tmp()
+    const modules = join(home, 'profiles', 'node_modules')
+    await healProfilesModuleFallback(anchor, home)
+    let releaseLock: (() => void) | undefined
+    let reportLock: (() => void) | undefined
+    const lockHeld = new Promise<void>((resolve) => { reportLock = resolve })
+    const release = new Promise<void>((resolve) => { releaseLock = resolve })
+    const holder = withFileLock(modules, async () => {
+      reportLock?.()
+      await release
+    })
+    await lockHeld
+
+    const healer = healProfilesModuleFallback(anchor, home)
+    const outcome = await Promise.race([
+      healer.then(() => 'complete' as const),
+      new Promise<'blocked'>(resolve => setTimeout(() => { resolve('blocked') }, 100)),
+    ])
+    releaseLock?.()
+    await Promise.all([holder, healer])
+    expect(outcome).toBe('complete')
+  })
+
   it('waits for the module-fallback writer lock before publishing entries', async () => {
   it('waits for the module-fallback writer lock before publishing entries', async () => {
     const anchor = stageInstallation({})
     const anchor = stageInstallation({})
     const home = tmp()
     const home = tmp()

Algunos archivos no se mostraron porque demasiados archivos cambiaron en este cambio