Sfoglia il codice sorgente

fix: retain prompt parts in client catalog

creatixchu 3 settimane fa
parent
commit
b67663c583

+ 2 - 2
.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.md
-2026-08-27-steer-followup-image-delivery.md: 71f1d470fa6435758a22baaf9630f5c241aabc47
-2026-08-27-steer-followup-image-delivery.zh.md: 72c47d5e3366b181b1ffbdb112cac06292616f2f
+2026-08-27-steer-followup-image-delivery.md: e5b3f516e45588a865c87a1d8188b50208fbff33
+2026-08-27-steer-followup-image-delivery.zh.md: 37e55b7148841634e306b2a9d17f7f48b099c1eb

+ 2 - 2
.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.md

@@ -18,7 +18,7 @@ Third, the browser queue projection reduced a queued image to the text `[image]`
 
 **Closing-turn wake delivery.** `ReactLoopAgent` tracks the identities of waking sends still awaiting a claim (`pendingWakes`); claim and discard notifications prune the set. At a driver exit whose turn loop returned without throwing, a non-empty set re-wakes the driver, so a steer or follow-up that lost the race with a normally closing turn is claimed by a fresh turn. Cancellation and `agent/pre-step` rejection instead clear the set: accepted-but-unclaimed input parks until the next waking send, preserving the tested `cancel({ keepInbox: true })` semantics and keeping rejected claims from being re-offered to the rejecting policy. Injected context never enters the set. The turn-flow section of [docs/architecture.md](../../../../docs/architecture.md) records the delivery/parking rule.
 
-**Host-side subagent image admission.** `SubagentPromptRequest.content` is now upload-shaped `PromptContentPart[]` (updating the wire contract in [Web subagent conversations](../feature/2026-07-27-web-subagent-conversations.md)), whose single home moved from `dsh-api-session-controller` to `dsh-attachment`; the shared `durablePromptContent()` conversion lives in `dsh-llm/content` and is used by both the Session prompt endpoint and `SubagentRuntime.prompt`. The subagent route admits and persists image batches through `ctx.attachments` before `followup()`, and the continuation manager refuses delivery inside the per-child lock when the child's `agent.options` route resolves to a model without image input (`MODEL_DOES_NOT_SUPPORT_IMAGES`, surfaced as `attachment-error` with the same reason vocabulary as the Session route). A child without a fixed options route, or a deployment without the LLM registry, delivers and relies on the LLM layer's text-only projection. The Client forwards image parts unchanged and the `SUBAGENT_IMAGE_UNSUPPORTED` copy is gone.
+**Host-side subagent image admission.** `SubagentPromptRequest.content` is now upload-shaped `PromptContentPart[]` (updating the wire contract in [Web subagent conversations](../feature/2026-07-27-web-subagent-conversations.md)). `dsh-attachment` owns the shared upload vocabulary used by the subagent route; `dsh-api-session-controller` retains a structurally identical Client-face declaration so the generated Client Cordis catalog contains the complete prompt-part fields, with a compile-time equality test preventing drift. The shared `durablePromptContent()` conversion lives in `dsh-llm/content` and is used by both the Session prompt endpoint and `SubagentRuntime.prompt`. The subagent route admits and persists image batches through `ctx.attachments` before `followup()`, and the continuation manager refuses delivery inside the per-child lock when the child's `agent.options` route resolves to a model without image input (`MODEL_DOES_NOT_SUPPORT_IMAGES`, surfaced as `attachment-error` with the same reason vocabulary as the Session route). A child without a fixed options route, or a deployment without the LLM registry, delivers and relies on the LLM layer's text-only projection. The Client forwards image parts unchanged and the `SUBAGENT_IMAGE_UNSUPPORTED` copy is gone.
 
 **Queue presentation.** The queue mirror's text preview excludes image blocks, and the queue dock renders each durable image part as a thumbnail resolved through `ctx.uiConversation.imageUrl` — the same session-authorized read the transcript uses. Editing queued image messages stays refused (#3072).
 
@@ -34,7 +34,7 @@ Third, the browser queue projection reduced a queued image to the text `[image]`
 
 ## Testing
 
-Agent-loop tests pin the closing-turn window deterministically (a `turn/end` listener queues the send as a microtask ahead of the driver's exit continuation) for steer, follow-up, and the inject non-delivery case. Host tests cover `mode: 'steer'` image admission; subagent control tests cover ordered admission, batch refusal, non-canonical base64, and the capability refusal mapping; continuation tests cover refusal without a partial message, capable delivery, and the routeless deferral. Client tests cover unstripped forwarding, queue thumbnails (load, failure placeholder, unmount), and the image-free preview.
+Agent-loop tests pin the closing-turn window deterministically (a `turn/end` listener queues the send as a microtask ahead of the driver's exit continuation) for steer, follow-up, and the inject non-delivery case. Host tests cover `mode: 'steer'` image admission; subagent control tests cover ordered admission, batch refusal, non-canonical base64, and the capability refusal mapping; continuation tests cover refusal without a partial message, capable delivery, and the routeless deferral. Client tests cover unstripped forwarding, the catalog-visible upload declaration, queue thumbnails (load, failure placeholder, unmount), and the image-free preview.
 
 ## Consequences
 

+ 2 - 2
.agents/notes/implemented/bug-fix/2026-08-27-steer-followup-image-delivery.zh.md

@@ -18,7 +18,7 @@ agent 运行期间提交的图片没有可靠进入模型上下文(#3186),
 
 **轮次收尾期的唤醒投递。** `ReactLoopAgent` 用 `pendingWakes` 记录尚未被认领的唤醒发送的身份;认领与丢弃通知会移除对应条目。当 driver 的轮次循环无异常返回并退出时,集合非空就重新拉起 driver,输掉与正常收尾轮次竞态的 steer 或 follow-up 由新轮次认领。取消与 `agent/pre-step` 拒绝则清空该集合:已接受但未认领的输入停放到下一次唤醒发送,既保留了有测试保护的 `cancel({ keepInbox: true })` 语义,也避免把被拒绝的认领重新塞给同一个拒绝策略。注入的上下文从不进入该集合。投递与停放规则记录在 [docs/architecture.md](../../../../docs/architecture.zh.md) 的 turn-flow 一节。
 
-**Host 侧子代理图片准入。** `SubagentPromptRequest.content` 改为上传形态的 `PromptContentPart[]`(同步更新 [Web 子代理会话](../feature/2026-07-27-web-subagent-conversations.zh.md) 的 wire 契约),该类型的唯一定义处从 `dsh-api-session-controller` 移到 `dsh-attachment`;共享的 `durablePromptContent()` 转换位于 `dsh-llm/content`,Session prompt 端点与 `SubagentRuntime.prompt` 共用。子代理路由在 `followup()` 之前经 `ctx.attachments` 完成整批图片的准入与持久化;continuation 管理器在逐子级锁内,当子级 `agent.options` 路由解析到不接受图片输入的模型时拒绝投递(`MODEL_DOES_NOT_SUPPORT_IMAGES`,以与 Session 路由一致的 `attachment-error` 词汇表上抛)。子级没有固定 options 路由,或部署未挂载 LLM 注册表时照常投递,交给 LLM 层的纯文本投影。客户端原样转发图片部分,`SUBAGENT_IMAGE_UNSUPPORTED` 文案删除。
+**Host 侧子代理图片准入。** `SubagentPromptRequest.content` 改为上传形态的 `PromptContentPart[]`(同步更新 [Web 子代理会话](../feature/2026-07-27-web-subagent-conversations.zh.md) 的 wire 契约)。`dsh-attachment` 负责子代理路由使用的共享上传词汇;`dsh-api-session-controller` 保留结构相同的 Client face 声明,使生成的 Client Cordis 目录包含完整的 prompt part 字段,并用编译期等价测试防止两处定义偏离。共享的 `durablePromptContent()` 转换位于 `dsh-llm/content`,Session prompt 端点与 `SubagentRuntime.prompt` 共用。子代理路由在 `followup()` 之前经 `ctx.attachments` 完成整批图片的准入与持久化;continuation 管理器在逐子级锁内,当子级 `agent.options` 路由解析到不接受图片输入的模型时拒绝投递(`MODEL_DOES_NOT_SUPPORT_IMAGES`,以与 Session 路由一致的 `attachment-error` 词汇表上抛)。子级没有固定 options 路由,或部署未挂载 LLM 注册表时照常投递,交给 LLM 层的纯文本投影。客户端原样转发图片部分,`SUBAGENT_IMAGE_UNSUPPORTED` 文案删除。
 
 **队列展示。** 队列镜像的文本预览不再包含图片块,queue dock 把每个持久化图片部分渲染为缩略图,经 `ctx.uiConversation.imageUrl` 解析,与会话记录使用同一个会话授权读取。已排队图片消息的编辑仍然拒绝(#3072)。
 
@@ -34,7 +34,7 @@ agent 运行期间提交的图片没有可靠进入模型上下文(#3186),
 
 ## Testing
 
-agent-loop 测试确定性地钉住收尾窗口(`turn/end` 监听器把发送排为微任务,先于 driver 的退出续体执行),覆盖 steer、follow-up 与注入不投递。Host 测试覆盖 `mode: 'steer'` 的图片准入;subagent control 测试覆盖有序准入、整批拒绝、非规范 base64 与能力拒绝映射;continuation 测试覆盖拒绝时不留半条消息、能力通过时投递、无路由时的顺延。客户端测试覆盖不剥离的转发、队列缩略图(加载、失败占位、卸载)与不含图片的预览。
+agent-loop 测试确定性地钉住收尾窗口(`turn/end` 监听器把发送排为微任务,先于 driver 的退出续体执行),覆盖 steer、follow-up 与注入不投递。Host 测试覆盖 `mode: 'steer'` 的图片准入;subagent control 测试覆盖有序准入、整批拒绝、非规范 base64 与能力拒绝映射;continuation 测试覆盖拒绝时不留半条消息、能力通过时投递、无路由时的顺延。客户端测试覆盖不剥离的转发、目录可见的上传声明、队列缩略图(加载、失败占位、卸载)与不含图片的预览。
 
 ## Consequences
 

+ 2 - 4
packages/api/session-controller/src/client/contract/session.ts

@@ -7,14 +7,12 @@
  * must stub); implementation-internal entry points (history staging, wire-frame
  * dispatch) stay on the class, invisible out here.
  */
-import type {
-  AttachmentIdType, ImageAttachmentRef, PromptContentPart,
-} from '@deepseek-ai/dsh-attachment'
+import type { AttachmentIdType, ImageAttachmentRef } from '@deepseek-ai/dsh-attachment'
 import type { MessageId } from '@deepseek-ai/dsh-llm/brand'
 import type { SessionId } from '@deepseek-ai/dsh-session/types'
 import type { RemoteResult } from '@deepseek-ai/dsh-typert-protocol'
 import type { ObservableSnapshot } from '@deepseek-ai/dsh-client-store'
-import type { QueueAction, SessionRequestId } from '../../types.ts'
+import type { PromptContentPart, QueueAction, SessionRequestId } from '../../types.ts'
 import type { ClientResult } from './result.ts'
 import type { PendingSubmissionImage, SessionSnapshot } from './snapshot.ts'
 

+ 10 - 2
packages/api/session-controller/src/types.ts

@@ -1,7 +1,7 @@
 /** Browser-safe request, result, and lifecycle vocabulary for the Session Remote service. */
 
 import type {
-  AttachmentIdType, ImageAttachmentLimits, ImageAttachmentRef, PromptContentPart,
+  AttachmentIdType, ImageAttachmentLimits, ImageAttachmentRef, ImageMediaType,
 } from '@deepseek-ai/dsh-attachment'
 import type { Branded } from '@deepseek-ai/dsh-brand'
 import type { MessageId } from '@deepseek-ai/dsh-llm/brand'
@@ -67,7 +67,15 @@ export interface SessionProjectionBaseline {
 export type SessionProjectionValues = Partial<SessionProjectionMap>
   & Readonly<Record<string, SessionProjectionValue>>
 
-export type { PromptContentPart } from '@deepseek-ai/dsh-attachment'
+/** Browser-submitted prompt content; the Host promotes image bytes to durable references. */
+export type PromptContentPart =
+  | { readonly type: 'text'; readonly text: string }
+  | {
+    readonly type: 'image'
+    readonly mediaType: ImageMediaType
+    readonly data: string
+    readonly name?: string
+  }
 
 /** Complete model selection for one Session. */
 export interface ModelSelection {

+ 7 - 1
packages/api/session-controller/tests/client-contract.client.spec.ts

@@ -1,8 +1,10 @@
-import { describe, expect, it, vi } from 'vitest'
+import { describe, expect, expectTypeOf, it, vi } from 'vitest'
+import type { PromptContentPart as AttachmentPromptContentPart } from '@deepseek-ai/dsh-attachment/types'
 import {
   MutableSessionEventSource, type SessionLiveEventEntry,
 } from '../src/client/contract/events.ts'
 import { transportResult } from '../src/client/contract/result.ts'
+import type { PromptContentPart as SessionPromptContentPart } from '../src/types.ts'
 
 function entry(seq: number): SessionLiveEventEntry {
   return {
@@ -17,6 +19,10 @@ function entry(seq: number): SessionLiveEventEntry {
 }
 
 describe('Client Session contracts', () => {
+  it('keeps its catalog-visible prompt parts identical to attachment intake', () => {
+    expectTypeOf<SessionPromptContentPart>().toEqualTypeOf<AttachmentPromptContentPart>()
+  })
+
   it('publishes exact replace, prepend, and append event-window changes', () => {
     const feed = new MutableSessionEventSource()
     const listener = vi.fn()

+ 4 - 0
packages/extensions/cordis-client-runner/src/client/api-catalog.ts

@@ -622,6 +622,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [
     name: 'ProjectionsFace',
     declaration: 'export interface ProjectionsFace {\n    faceOf(key: string): ObservableSnapshot<unknown>;\n}',
   },
+  {
+    name: 'PromptContentPart',
+    declaration: 'export type PromptContentPart = {\n    readonly type: \'text\';\n    readonly text: string;\n} | {\n    readonly type: \'image\';\n    readonly mediaType: ImageMediaType;\n    readonly data: string;\n    readonly name?: string;\n};',
+  },
   {
     name: 'PromptError',
     declaration: 'export interface PromptError {\n    readonly op: \'send\' | \'stop\';\n    readonly error: ClientFailure;\n}',

+ 7 - 1
packages/extensions/cordis-client-runner/tests/api-catalog.client.spec.ts

@@ -30,9 +30,15 @@ describe('Client Cordis inspect catalog', () => {
 
   it('includes the current referenced type closure for the Sessions service', () => {
     const result = queryServiceApi('sessions') as {
-      referencedTypes: readonly { name: string }[]
+      referencedTypes: readonly { name: string; declaration: string }[]
     }
     expect(result.referencedTypes.length).toBeGreaterThan(0)
+    expect(result.referencedTypes).toEqual(expect.arrayContaining([
+      expect.objectContaining({
+        name: 'PromptContentPart',
+        declaration: expect.stringContaining("readonly type: 'image'"),
+      }),
+    ]))
     expect(result.referencedTypes.map(type => type.name)).not.toEqual(expect.arrayContaining([
       'ConversationSnapshot',
       'PendingInteraction',