Ver Fonte

fix(llm-deepseek): correct image pricing documentation and edge-case evidence

creatixchu há 1 semana atrás
pai
commit
f24bc3c832

+ 2 - 2
.agents/notes/implemented/bug-fix/2026-09-10-deepseek-image-token-calculator-v41.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-09-10-deepseek-image-token-calculator-v41.md
-2026-09-10-deepseek-image-token-calculator-v41.md: d9a8b7e72143793f13aaf3a4db26468dc0f4c162
-2026-09-10-deepseek-image-token-calculator-v41.zh.md: 26cdf23ccf2b693813648c015b5a9926cd362ab2
+2026-09-10-deepseek-image-token-calculator-v41.md: d8042b04c1ed7824720485aa3ccf584f913d0726
+2026-09-10-deepseek-image-token-calculator-v41.zh.md: 6be386da1c66c469329d03e7b86c8c0e2c22ce3a

+ 2 - 2
.agents/notes/implemented/bug-fix/2026-09-10-deepseek-image-token-calculator-v41.md

@@ -6,7 +6,7 @@ English | [中文](2026-09-10-deepseek-image-token-calculator-v41.zh.md)
 
 ## Problem
 
-`deepSeekImageTokens()` in `llm-deepseek` ported the provider's published image-token calculator in its `v4` configuration: a 384×384 scale-up floor, a 384-token cap, an 8:1 width clamp, a grid layout that adds a row for odd row counts and parity corrections, and a pad-to-4 alignment charged at its worst case. The provider's Vision guide now documents a different projection for the current Flash model: images below roughly 544×544 total pixels scale up, larger images scale down to roughly 1300×1300 total pixels, and one image costs at most 1024 tokens. The published calculator carries this as a `v41` configuration and the docs page instantiates that one. The old port therefore underpriced every retained image, an 800×800 request image by 73 tokens, so automatic compaction on image-dense sessions triggered late.
+`deepSeekImageTokens()` in `llm-deepseek` ported the provider's published image-token calculator in its `v4` configuration: a 384×384 scale-up floor, a 384-token cap, an 8:1 width clamp, a grid layout that adds a row for odd row counts and parity corrections, and a pad-to-4 alignment charged at its worst case. The provider's Vision guide now documents a different projection for the current Flash model: images below roughly 544×544 total pixels scale up, larger images scale down to roughly 1300×1300 total pixels, and one image costs at most 1024 tokens. The published calculator carries this as a `v41` configuration and the docs page instantiates that one. The old port underprices an 800×800 request image by 73 tokens, which can delay automatic compaction in sessions containing these images. The error depends on dimensions: a 640×480 image is overestimated by 3 tokens.
 
 ## Decision
 
@@ -24,4 +24,4 @@ The test vectors are re-pinned from the published calculator. The request-pricin
 
 ## Consequences
 
-Retained request images price higher, so image-dense DeepSeek sessions compact earlier and closer to the provider's real pressure. Under the harness's 640,000-pixel projection the ceiling for one image is 422 tokens at 800×800 rather than 349. The estimate no longer carries a three-token conservative margin; provider usage remains the authoritative anchor once a request completes. Sessions replayed through `llm-replay` use their fixture's `imageRequestTokens` and are unaffected.
+An 800×800 request image costs 422 tokens instead of 349, while a 640×480 image costs 206 instead of 209 and a low-budget 512×512 image costs 184 instead of 201. Compaction pressure changes with the retained image dimensions. The 640,000-pixel budget does not imply a 422-token ceiling: an 8192×1 image stays within that pixel budget and costs 1024 tokens. The estimate no longer carries a three-token conservative margin; provider usage remains the authoritative anchor once a request completes. Sessions replayed through `llm-replay` use their fixture's `imageRequestTokens` and are unaffected.

+ 2 - 2
.agents/notes/implemented/bug-fix/2026-09-10-deepseek-image-token-calculator-v41.zh.md

@@ -6,7 +6,7 @@ Status: implemented
 
 ## 问题
 
-`llm-deepseek` 中的 `deepSeekImageTokens()` 移植的是提供方公开图片 token 计算器的 `v4` 配置:384×384 放大下限、384 token 上限、8:1 宽度钳制、奇数行数额外加一行并做奇偶校正的网格布局,以及按最坏情况计价的 pad-to-4 对齐。提供方的图像理解指南现在为当前 Flash 模型记录了另一套投影规则:总像素小于约 544×544 的图片放大,更大的图片缩小到约 1300×1300 总像素,单张图片最多 1024 token。公开计算器以 `v41` 配置承载这套规则,文档页实例化的也是它。旧移植因此对每张保留图片都估低,一张 800×800 的请求图片低估 73 token,图片密集会话的自动压缩触发过晚
+`llm-deepseek` 中的 `deepSeekImageTokens()` 移植的是提供方公开图片 token 计算器的 `v4` 配置:384×384 放大下限、384 token 上限、8:1 宽度钳制、奇数行数额外加一行并做奇偶校正的网格布局,以及按最坏情况计价的 pad-to-4 对齐。提供方的图像理解指南现在为当前 Flash 模型记录了另一套投影规则:总像素小于约 544×544 的图片放大,更大的图片缩小到约 1300×1300 总像素,单张图片最多 1024 token。公开计算器以 `v41` 配置承载这套规则,文档页实例化的也是它。旧移植对一张 800×800 的请求图片低估 73 token,可能使包含这类图片的会话延迟触发自动压缩。误差取决于尺寸:一张 640×480 的图片会被高估 3 token
 
 ## 决策
 
@@ -24,4 +24,4 @@ Status: implemented
 
 ## 后果
 
-保留的请求图片计价更高,图片密集的 DeepSeek 会话会更早压缩,更接近提供方的真实压力。在 harness 的 640,000 像素投影下,单张图片的上限是 800×800 时的 422 token,而非 349。估算值不再带 3 token 的保守余量;请求完成后,提供方 usage 仍是权威锚点。经 `llm-replay` 回放的会话使用各自 fixture 的 `imageRequestTokens`,不受影响。
+800×800 请求图片的计价从 349 变为 422 token,640×480 图片从 209 变为 206,低预算下的 512×512 图片从 201 变为 184。压缩压力随保留图片的尺寸变化。640,000 像素预算不意味着 422 token 上限:8192×1 图片在该像素预算内,仍计 1024 token。估算值不再带 3 token 的保守余量;请求完成后,提供方 usage 仍是权威锚点。经 `llm-replay` 回放的会话使用各自 fixture 的 `imageRequestTokens`,不受影响。

+ 2 - 2
docs/subsystems/llm-streaming.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write docs/subsystems/llm-streaming.md
-llm-streaming.md: cadfda5553ca5d5e0df78f40b6a7a75ef7cc71a6
-llm-streaming.zh.md: 5f73d544459268ce83a30218b710dd7c29f7a614
+llm-streaming.md: 77d3313b3c1a025f756e984453bd041abd7b0534
+llm-streaming.zh.md: cac7897ab412f45bd5b5ab20c6460644f18f4ff2

+ 1 - 1
docs/subsystems/llm-streaming.md

@@ -249,7 +249,7 @@ interface LlmFailure {
 
 ## Request-image pricing
 
-An adapter whose provider charges visual tokens for request images declares per-route pricing by overriding `LlmAdapter.imageRequestPricing`, and `ctx.llm.imageRequestPricing(provider, model)` resolves it synchronously for consumers. The token meter resolves the routed model's pricing on every measurement so compaction pressure, retention, and range selection price image history as the routed request actually sends it; the DeepSeek adapter reproduces its own request projection (per-model pixel budget, oldest-first offload) and prices retained images with the published v4 vision accounting, while provider usage remains the authoritative anchor for completed requests.
+An adapter whose provider charges visual tokens for request images declares per-route pricing by overriding `LlmAdapter.imageRequestPricing`, and `ctx.llm.imageRequestPricing(provider, model)` resolves it synchronously for consumers. The token meter resolves the routed model's pricing on every measurement so compaction pressure, retention, and range selection price image history as the routed request actually sends it; the DeepSeek adapter reproduces its own request projection (per-model pixel budget, oldest-first offload) and prices retained images with the published vision accounting, while provider usage remains the authoritative anchor for completed requests.
 
 ```ts type-equiv
 /**

+ 1 - 1
docs/subsystems/llm-streaming.zh.md

@@ -251,7 +251,7 @@ interface LlmFailure {
 
 ## 请求图片定价
 
-提供方对请求图片收取视觉 token 的适配器通过覆写 `LlmAdapter.imageRequestPricing` 声明按路由的定价,消费方经 `ctx.llm.imageRequestPricing(provider, model)` 同步解析。token 计量服务在每次计量时解析路由模型的定价,使 compaction 的压力、保留与选段都按路由请求实际发送的形式为图片历史计价;DeepSeek 适配器复现自身的请求投影(按模型的像素预算、最旧优先 offload),并用官方公布的 v4 视觉计量为保留图片定价,已完成请求仍以 provider usage 为权威锚点。
+提供方对请求图片收取视觉 token 的适配器通过覆写 `LlmAdapter.imageRequestPricing` 声明按路由的定价,消费方经 `ctx.llm.imageRequestPricing(provider, model)` 同步解析。token 计量服务在每次计量时解析路由模型的定价,使 compaction 的压力、保留与选段都按路由请求实际发送的形式为图片历史计价;DeepSeek 适配器复现自身的请求投影(按模型的像素预算、最旧优先 offload),并用官方公布的视觉计量为保留图片定价,已完成请求仍以 provider usage 为权威锚点。
 
 ```ts type-equiv
 /**

+ 2 - 2
packages/llm/llm-deepseek/README.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
-README.md: 67c700bec9fbe6b9df7ab95ded5c991bf01dba1c
-README.zh.md: 43670a3947519ae1b7598900b8ae329becf9ad6a
+README.md: 41646097f80dfad9cc523595b9d2820c28fde66b
+README.zh.md: c7a1e40952dff7a804880dcd266114cecb20918f

+ 2 - 0
packages/llm/llm-deepseek/README.md

@@ -190,6 +190,8 @@ These limits define where the adapter stops and future work begins. They are cur
 - **Images are input-only durable attachments** — direct external URLs and assistant image output are not supported; DeepSeek input normally uses the Files API and uses inline base64 only for per-request recovery.
 - The default catalog pre-registers `deepseek-flash` and its text/image and in-history capabilities without probing gateway availability. Requests can fail with `INVALID_REQUEST` until the gateway enables the id. With `DEEPSEEK_API_KEY` and a supporting gateway configured, `DEEPSEEK_FLASH_E2E=1` enables the Chat Completions check in [this package's e2e suite](tests/adapter.e2e.ts).
 
+- The default request-image projection caps total pixels at 640,000, below the provider's roughly 1300×1300 processing budget, so it can discard usable detail. Each model's `imagePixelBudget` can override this default; changing the default affects request content and needs separate snapshot verification ([decision](../../../.agents/notes/implemented/bug-fix/2026-09-10-deepseek-image-token-calculator-v41.md)).
+
 <a id="dev-note"></a>
 ### Dev Note
 

+ 2 - 0
packages/llm/llm-deepseek/README.zh.md

@@ -190,6 +190,8 @@ loop 保留的响应块会追加到下一个请求,并保留其更早的可复
 - **图片是仅用于输入的持久附件**——不支持直接外部 URL 与 assistant 图片输出;DeepSeek 输入通常使用 Files API,仅在单次请求恢复时使用内联 base64。
 - 默认目录预注册 `deepseek-flash` 及其文本、图片和历史内更新能力,不探测网关可用性。网关开放该 ID 前,请求可能以 `INVALID_REQUEST` 失败。配置 `DEEPSEEK_API_KEY` 和支持该 ID 的网关后,设置 `DEEPSEEK_FLASH_E2E=1` 可启用[本包 e2e 测试文件](tests/adapter.e2e.ts)中的 Chat Completions 协议验证。
 
+- 默认请求图片投影限制为 640,000 总像素,低于提供方约 1300×1300 的处理预算,可能丢弃可用细节。每个模型的 `imagePixelBudget` 可以覆盖默认值。更改默认值会改变请求内容,需要单独验证快照([决策](../../../.agents/notes/implemented/bug-fix/2026-09-10-deepseek-image-token-calculator-v41.zh.md))。
+
 <a id="dev-note"></a>
 ### 开发备注
 

+ 3 - 3
packages/llm/llm-deepseek/src/image-tokens.ts

@@ -38,9 +38,9 @@ function gridTokens(gridHeight: number, gridWidth: number): number {
   return gridHeight * (gridWidth + 1) + 2
 }
 
-/** Token grid the padded pixel dimensions project onto. */
-function gridCells(paddedPixels: number): number {
-  return ceilDiv(intDiv(paddedPixels, PATCH_SIZE), DOWNSAMPLE_RATIO)
+/** Token-cell count along one padded pixel axis. */
+function gridCells(paddedLength: number): number {
+  return ceilDiv(intDiv(paddedLength, PATCH_SIZE), DOWNSAMPLE_RATIO)
 }
 
 /** Solve the largest grid within `budget` tokens preserving the aspect ratio. */

+ 2 - 2
packages/llm/llm-deepseek/src/request-pricing.ts

@@ -2,7 +2,7 @@
  * Provider-side request-image pricing for DeepSeek routes: reproduces the
  * adapter's deterministic request projection (per-model pixel budget,
  * oldest-first offload under the raw-byte and count budgets) and prices every
- * retained image with the published v4 vision-token accounting. Consumed
+ * retained image with the published vision-token accounting. Consumed
  * synchronously by the token meter through `LlmAdapter.imageRequestPricing`;
  * provider usage remains the authoritative anchor for completed requests.
  *
@@ -20,7 +20,7 @@ import type { DeepSeekCatalogModel, DeepSeekConnectionOptions } from './adapter.
 export const DEFAULT_MAX_REQUEST_FILES_BYTES = 128 * 1024 * 1024
 /** Provider request image-count limit. */
 export const DEFAULT_MAX_IMAGES_PER_REQUEST = 600
-/** Total-pixel budget matching DeepSeek's normal vision projection. */
+/** Default total-pixel budget for harness request-image projection. */
 export const DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET = 640_000
 /** Total-pixel budget matching provider low-detail image input. */
 export const DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET = 512 * 512

+ 3 - 6
packages/llm/llm-deepseek/tests/image-tokens.spec.ts

@@ -16,6 +16,8 @@ describe('DeepSeek image tokens', () => {
     [2000, 2000, 994],
     [5000, 5000, 994],
     [300, 50, 200],
+    [8192, 100, 593],
+    [16, 8192, 590],
   ])('prices %sx%s as %s tokens', (width, height, expected) => {
     expect(deepSeekImageTokens(width, height)).toBe(expected)
   })
@@ -33,19 +35,14 @@ describe('DeepSeek image tokens', () => {
   })
 
   it('solves a one-row grid for an extremely wide image', () => {
-    // Width-dominant aspect drives the solver's single-row branch; no
-    // aspect-ratio clamp applies, so the row fills the whole budget.
     expect(deepSeekImageTokens(9000, 1)).toBe(1024)
-    expect(deepSeekImageTokens(8192, 100)).toBe(593)
   })
 
   it('solves a one-column grid for an extremely tall image', () => {
-    // Height-dominant aspect drives the solver's single-column branch.
     expect(deepSeekImageTokens(1, 9000)).toBe(1024)
-    expect(deepSeekImageTokens(16, 8192)).toBe(590)
   })
 
-  it('converges through a second projection pass when the first is not a fixpoint', () => {
+  it('converges through repeated projection passes when the first is not a fixpoint', () => {
     expect(deepSeekImageTokens(12, 1123)).toBe(380)
     expect(deepSeekImageTokens(89, 2076)).toBe(254)
   })

+ 9 - 0
packages/llm/llm-deepseek/tests/request-pricing.spec.ts

@@ -49,6 +49,15 @@ describe('DeepSeek request-image pricing', () => {
     }])
   })
 
+  it.each([[8192, 1], [1, 8192]])('prices a %sx%s image at the token cap within the default pixel budget', (width, height) => {
+    const image = ref('thin', width, height)
+    const prices = deepSeekImageRequestPricing(connection(), 'vision').priceImages([image])
+    expect(prices).toEqual([{
+      visualTokens: 1024,
+      text: requestImageHandleText(image, { width, height }),
+    }])
+  })
+
   it('honors the low-detail pixel budget preset', () => {
     const image = ref('photo', 4096, 4096)
     const options = resolveAdapterOptions({