Merge branch 'codex/canonical-tool-output' into codex/code-mode-typed-results

.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write
-2026-07-20-canonical-tool-output-contract.md: feca70f8fed284f64c5a5560e3fddd3e2aa9a752
-2026-07-20-canonical-tool-output-contract.zh.md: 0fca743c1cc050236787ce25553684bd393ba861
+2026-07-20-canonical-tool-output-contract.md: 0cdce1111d976058eab939e0f7e51d6310071ec8
+2026-07-20-canonical-tool-output-contract.zh.md: e62842ddbb1ff753288beaaaa81f33e1edba2303

.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.md

@@ -58,7 +58,7 @@ The first-party tools preserve their existing Native text while returning domain
 | `structured_output` | `{ recorded: true }` |
 | `run_code` | `{ logs: string[], result?: JsonValue }` |
 
-Provider and executor acquisition limits remain real limits on the canonical value. Formatting-only limits belong in `render`; `glob` and `grep`, for example, keep every acquired item in `value` while their Native projection retains and best-effort spills the configured first page. Filesystem mutations derive replayable diff metadata from `args` and the canonical before/after value rather than returning UI state from the body.
+Provider and executor acquisition limits remain real limits on the canonical value. Formatting-only limits belong in `render`; `glob` and `grep`, for example, keep every acquired item in `value` while their Native projection retains and best-effort spills the configured first page. Generic spill prepends and delegates its post-execute listener so an ordinary tool-owned asynchronous projection completes before generic byte bounding regardless of plugin load order. Filesystem mutations derive replayable diff metadata from `args` and the canonical before/after value rather than returning UI state from the body.
 
 MCP bridges preserve protocol blocks through `McpResult<{...}> = { content: JsonValue[]; structuredContent? }`. An advertised `outputSchema` is enforced when it belongs to the supported raw subset; unsupported schemas fall back to `JsonValue` rather than pretending to validate them. Native rendering still uses the existing MCP-to-`ContentBlock` projection, and MCP `isError` becomes a failed tool result.
 

.agents/notes/implemented/architecture/2026-07-20-canonical-tool-output-contract.zh.md

@@ -58,7 +58,7 @@ type ToolExecutionResult =
 | `structured_output` | `{ recorded: true }` |
 | `run_code` | `{ logs: string[], result?: JsonValue }` |
 
-提供方和执行器的采集上限仍会实际限制规范值。仅用于格式化的限制归 `render` 所有；例如，`glob` 和 `grep` 会在 `value` 中保留所有已采集项，而其 Native 投影仍只保留配置指定的第一页，并尽力将完整展示内容写入落盘文件。文件系统变更工具根据 `args` 和规范的变更前／后值推导可回放的 diff 元数据，不再由工具主体返回 UI 状态。
+提供方和执行器的采集上限仍会实际限制规范值。仅用于格式化的限制归 `render` 所有；例如，`glob` 和 `grep` 会在 `value` 中保留所有已采集项，而其 Native 投影仍只保留配置指定的第一页，并尽力将完整展示内容写入落盘文件。通用落盘机制会前置注册其 post-execute 监听器，并让该监听器先向后委托，因此无论插件加载顺序如何，普通工具自有的异步投影都会在通用字节数上限处理之前完成。文件系统变更工具根据 `args` 和规范的变更前／后值推导可回放的 diff 元数据，不再由工具主体返回 UI 状态。
 
 MCP 桥接层通过 `McpResult<{...}> = { content: JsonValue[]; structuredContent? }` 保留协议内容块。当公布的 `outputSchema` 属于受支持的原始子集时，系统会强制校验；不受支持的 schema 则回退为 `JsonValue`，而不会假装已完成校验。Native 渲染仍使用现有的 MCP 到 `ContentBlock` 投影，MCP `isError` 则会变为失败的工具结果。
 

docs/config-catalog.md

@@ -1047,7 +1047,7 @@ export interface Config {
 }
 ```
 
-Source: [`packages/spill/spill-policy/src/index.ts:50`](../packages/spill/spill-policy/src/index.ts)
+Source: [`packages/spill/spill-policy/src/index.ts:51`](../packages/spill/spill-policy/src/index.ts)
 
 ## `@deepseek-ai/dsh-subagent-acp`
 

docs/core-data-structures/filesystem.md

@@ -205,7 +205,7 @@ interface FsPolicyExec {
 
 ## Read outcome (consumer / read rendering)
 
-A text read is bounded by line window, byte cap, and backend limits. The outcome the model-facing `read` tool renders is purely presentational; there is no `full`/`partial` view — authorization is freshness-based (the tool emits `fs/observed` with the stat's version directly), so any windowed read can authorize a later write/edit when the file is unchanged. Read windowing and this outcome shape live in `dsh-tool-fs` (the executor that owns the read), not in the policy plugin.
+A text read is bounded by line window, byte cap, and backend limits. After the byte cap is reached, scanning continues without retaining more lines so `totalLines` remains exact. The outcome the model-facing `read` tool renders is purely presentational; there is no `full`/`partial` view — authorization is freshness-based (the tool emits `fs/observed` with the stat's version directly), so any windowed read can authorize a later write/edit when the file is unchanged. Read windowing and this outcome shape live in `dsh-tool-fs` (the executor that owns the read), not in the policy plugin.
 
 ```ts type-equiv
 /** Outcome of a bounded text read — what {@link formatReadOutput} renders. */
@@ -214,9 +214,9 @@ interface FileReadOutcome {
   offset: number
   /** Returned lines, already numbered. */
   lines: FileTextLine[]
-  /** Total line count in the file, unless `truncatedByBytes` stopped scanning early. */
+  /** Exact total line count in the file. */
   totalLines: number
-  /** Whether selected output hit the byte cap before EOF or the requested limit. */
+  /** Whether selected output hit the byte cap. */
   truncatedByBytes?: true
 }
 ```

packages/core/tools/README.md

@@ -89,7 +89,7 @@ ctx.tools.register(defineTool({
 
 The unified schema DSL uses `ParameterSchemaSpec` for the implicit open parameter object and `ValueSchemaSpec` for any JSON-value root. It supports `string`, `number`, `integer`, `boolean`, `null`, `array`, `object`, author-only `json`, and exact-one `oneOf`; scalar `enum`/`const` values are type-correct. Every explicit DSL object declares `additionalProperties: true | false`, while the implicit parameter root and raw JSON Schema keep the standard open default.
 
-A `defineTool` definition validates model arguments before execution and turns missing required values, wrong primitives, invalid enum members, and nested violations into `ToolArgsError` (`INVALID_ARGS`) for the normal error-result path. It also infers the body return and pure output projectors from `output.schema`; the registry snapshots and validates the returned lossless JSON before presentation. Extra parameter keys are allowed, defaults are not applied, and object or array fields without `properties` or `items` receive only a type check. Raw-registered tools own input validation but still declare and receive registry-enforced output.
+A `defineTool` definition validates model arguments before execution and turns missing required values, wrong primitives, invalid enum members, and nested violations into `ToolArgsError` (`INVALID_ARGS`) for the normal error-result path. It also infers the body return and pure output projectors from `output.schema`; the registry snapshots and validates the returned lossless JSON before presentation. The implicit parameter root is open; an explicit object accepts extra keys only with `additionalProperties: true`, and a closed object with no declared properties accepts only `{}`. Raw JSON Schema objects remain open unless they explicitly set `additionalProperties: false`. Defaults are not applied; open objects without `properties` and arrays without `items` receive only a container type check. Raw-registered tools own input validation but still declare and receive registry-enforced output.
 
 See `defineTool`, `validateArgs`, `ToolArgsError`, `ValueSchemaSpec`, `ParameterSchemaSpec`, `InferValue`, `InferArgs`, `valueSchemaSpecToJsonSchema`, and `parameterSchemaSpecToJsonSchema` in the public API for details.
 

packages/fs/tool-fs/src/read-render.ts

@@ -37,9 +37,9 @@ export interface FileTextLine {
 export interface WindowResult {
   /** Returned lines, already numbered. */
   lines: FileTextLine[]
-  /** Total line count in the file, unless `truncatedByBytes` stopped scanning early. */
+  /** Exact total line count in the file. */
   totalLines: number
-  /** Whether selected output hit the byte cap before EOF or the requested limit. */
+  /** Whether selected output hit the byte cap. */
   truncatedByBytes: boolean
 }
 
@@ -49,9 +49,9 @@ export interface FileReadOutcome {
   offset: number
   /** Returned lines, already numbered. */
   lines: FileTextLine[]
-  /** Total line count in the file, unless `truncatedByBytes` stopped scanning early. */
+  /** Exact total line count in the file. */
   totalLines: number
-  /** Whether selected output hit the byte cap before EOF or the requested limit. */
+  /** Whether selected output hit the byte cap. */
   truncatedByBytes?: true
 }
 
@@ -60,11 +60,10 @@ interface WindowAccumulator {
   totalLines: number
   outputBytes: number
   truncatedByBytes: boolean
-  done: boolean
 }
 
 function newAccumulator(): WindowAccumulator {
-  return { lines: [], totalLines: 0, outputBytes: 0, truncatedByBytes: false, done: false }
+  return { lines: [], totalLines: 0, outputBytes: 0, truncatedByBytes: false }
 }
 
 function truncateLine(line: string, maxLineLength: number): string {
@@ -77,13 +76,12 @@ function lineByteSize(line: string, currentLineCount: number): number {
 
 function consumeLine(acc: WindowAccumulator, rawLine: string, request: ReadWindow): void {
   acc.totalLines += 1
-  if (acc.totalLines < request.offset || acc.lines.length >= request.limit) return
+  if (acc.truncatedByBytes || acc.totalLines < request.offset || acc.lines.length >= request.limit) return
 
   const text = truncateLine(rawLine, request.maxLineLength)
   const bytes = lineByteSize(text, acc.lines.length)
   if (acc.outputBytes + bytes > request.maxBytes) {
     acc.truncatedByBytes = true
-    acc.done = true
     return
   }
   acc.outputBytes += bytes
@@ -102,8 +100,9 @@ function finish(acc: WindowAccumulator, request: ReadWindow, displayPath: string
 }
 
 /**
- * Build one window from streamed or whole-file chunks, enforcing line and byte caps and throwing
- * `FS_NOT_FOUND` when the requested offset is past EOF.
+ * Build one window from streamed or whole-file chunks, enforcing line and byte caps while still
+ * scanning to an exact total line count, and throwing `FS_NOT_FOUND` when the requested offset is
+ * past EOF.
  * @param chunks - decoded text chunks in file order; chunk boundaries carry no meaning.
  * @param request - the resolved window; the caller has already applied its defaults and caps.
  * @param displayPath - the caller-facing path used in the offset-out-of-range error.
@@ -137,7 +136,6 @@ export async function buildWindow(
       appendToLineBuffer(chunk.slice(startPos, newlinePos))
       flushLine()
       startPos = newlinePos + 1
-      if (acc.done) return finish(acc, request, displayPath)
     }
     appendToLineBuffer(chunk.slice(startPos))
   }

packages/fs/tool-fs/tests/read-render.spec.ts

@@ -86,6 +86,7 @@ describe('buildWindow', () => {
     it('caps output at a custom maxBytes', async () => {
       const result = await buildWindow(whole('aaaa\nbbbb\ncccc'), { offset: 1, limit: 10, maxLineLength: 2000, maxBytes: 9 }, 'f')
       expect(result.lines.map(l => l.text)).toEqual(['aaaa', 'bbbb'])
+      expect(result.totalLines).toBe(3)
       expect(result.truncatedByBytes).toBe(true)
     })
   })
@@ -105,6 +106,7 @@ describe('buildWindow', () => {
     it('caps output bytes mid-stream', async () => {
       const big = Array.from({ length: 2000 }, () => 'y'.repeat(100)).join('\n')
       const result = await buildWindow(chunked(big, 512), READ_ALL, 'f')
+      expect(result.totalLines).toBe(2000)
       expect(result.truncatedByBytes).toBe(true)
     })
 

packages/fs/tool-fs/tests/tools.spec.ts

@@ -572,6 +572,9 @@ describe('read caps are plugin config', () => {
     const { ctx, fs } = await setupWith({ readMaxBytes: 9 })
     fs.files.set('key:a.txt', 'aaaa\nbbbb\ncccc')
     const result = await call(ctx, 'read', { file_path: 'a.txt' })
+    expect(result.isError).toBe(false)
+    if (result.isError) throw new Error('expected read success')
+    expect(result.value).toMatchObject({ totalLines: 3 })
     expect(text(result)).toContain('Output capped.')
     expect(text(result)).not.toContain('cccc')
   })

packages/spill/spill-policy/README.md

@@ -30,7 +30,7 @@ This plugin registers **no service** and owns no storage or preview mechanics: p
 
 ## Scope
 
-The policy sees only the FINAL formatted surface result—not a tool's internal resource or canonical value. If a provider already truncated (e.g. `web-fetch-local.maxBodyChars`), the spill artifact holds the full formatted result the tool returned, not the full original source. Provider/resource caps stay mandatory and separate. `glob`/`grep` own item-level surface spill because their complete acquired values still exist before rendering; bash streams own acquisition-time spill. See the [tool output spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md).
+The policy sees only the FINAL formatted surface result—not a tool's internal resource or canonical value. If a provider already truncated (e.g. `web-fetch-local.maxBodyChars`), the spill artifact holds the full formatted result the tool returned, not the full original source. Provider/resource caps stay mandatory and separate. `glob`/`grep` own item-level surface spill because their complete acquired values still exist before rendering; bash streams own acquisition-time spill. The generic policy prepends its waterfall listener, then delegates, so ordinary tool-owned asynchronous projections complete before generic byte bounding regardless of plugin load order. See the [tool output spill Agent Note](../../../.agents/notes/implemented/architecture/2026-07-08-tool-output-spill-files.md).
 
 ## Model Experience
 

packages/spill/spill-policy/src/index.ts

@@ -26,10 +26,11 @@
  *   failure ⇒ log and return the original result. A spill failure must NEVER
  *   turn a successful tool call into an `isError` or hide the inline result.
  *
- * It COMPOSES with other post-execute listeners: it delegates via `next()` and
- * bounds the resulting content projection, so a hook that replaced content
- * still has its replacement bounded, while value replacements and `block`
- * decisions pass through unchanged.
+ * It COMPOSES with other post-execute listeners: its prepended listener
+ * delegates via `next()` and bounds the resulting content projection, so
+ * tool-owned asynchronous projection runs before generic bounding, a hook that
+ * replaced content still has its replacement bounded, and value replacements
+ * and `block` decisions pass through unchanged.
  *
  * @module @deepseek-ai/dsh-spill-policy
  */
@@ -176,5 +177,5 @@ export function apply(ctx: Context, config: Config): void {
     }
     const replaced: ContentBlock[] = [{ type: 'text', text: replacedText }]
     return { kind: 'accept', content: replaced, ...decision.additionalContexts ? { additionalContexts: decision.additionalContexts } : {} }
-  })
+  }, { prepend: true })
 }

packages/spill/spill-policy/tests/spill-policy.spec.ts

@@ -16,7 +16,7 @@ import type { ContentBlock } from '@deepseek-ai/dsh-llm'
 import { SessionId } from '@deepseek-ai/dsh-session'
 import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
 import ToolRegistry, { defineContentToolFixture } from '@deepseek-ai/dsh-tools'
-import type { ToolExecution, ToolExecutionToken } from '@deepseek-ai/dsh-tools'
+import type { PostToolDecision, ToolExecution, ToolExecutionToken } from '@deepseek-ai/dsh-tools'
 import { SpillLocator, SpillStore } from '@deepseek-ai/dsh-spill'
 import type { SaveTextSpill, SpillRef } from '@deepseek-ai/dsh-spill'
 import * as SpillPolicy from '@deepseek-ai/dsh-spill-policy'
@@ -61,7 +61,11 @@ function exec(name: string, session = 's1'): ToolExecution {
  * Build a context with tools + the policy, and optionally a spill backend.
  * Returns the context and the backend handle (undefined when `withSpill` false).
  */
-async function setup(config: SpillPolicy.Config, withSpill = true): Promise<{ ctx: Context; spill?: StubStore; fiber: Awaited<ReturnType<Context['plugin']>> }> {
+async function setup(
+  config: SpillPolicy.Config,
+  withSpill = true,
+  beforePolicy?: (ctx: Context) => void,
+): Promise<{ ctx: Context; spill?: StubStore; fiber: Awaited<ReturnType<Context['plugin']>> }> {
   const ctx = new Context()
   await ctx.plugin(SystemPrompt)
   await ctx.plugin(ToolRegistry)
@@ -70,6 +74,7 @@ async function setup(config: SpillPolicy.Config, withSpill = true): Promise<{ ct
     await ctx.plugin(StubStore)
     spill = ctx.spillStore as StubStore
   }
+  beforePolicy?.(ctx)
   const fiber = await ctx.plugin(SpillPolicy, config)
   return { ctx, fiber, ...spill ? { spill } : {} }
 }
@@ -272,6 +277,26 @@ describe('best-effort fallback', () => {
 })
 
 describe('composition', () => {
+  it('wraps an earlier tool-owned projection before applying the generic cap', async () => {
+    let downstreamDecision: PostToolDecision | undefined
+    const { ctx, spill } = await setup({ maxInlineBytes: 200 }, true, (target) => {
+      target.on('tools/post-execute', async (_exec, _result, next): Promise<PostToolDecision> => {
+        downstreamDecision = await next()
+        return {
+          kind: 'accept',
+          content: [{ type: 'text', text: `first page\n\nFull canonical result stored at /spill/search-results.txt.\n${'z'.repeat(500)}` }],
+        }
+      })
+    })
+    ctx.tools.register(textTool('search', 'initial capped page'))
+
+    const result = await ctx.tools.execute(exec('search'))
+
+    expect(downstreamDecision).toEqual({ kind: 'accept' })
+    expect(spill?.saves[0]?.content).toContain('Full canonical result stored at /spill/search-results.txt.')
+    expect(textOf(result.content)).toContain('Full formatted result stored at')
+  })
+
   it('bounds content a downstream post-execute listener replaced', async () => {
     const { ctx, spill } = await setup({ maxInlineBytes: 200 })
     // A later-registered listener replaces the (small) tool result with a big one;