|
|
@@ -10,7 +10,7 @@ English | [中文](2026-07-14-provider-routed-llm-adapters.zh.md)
|
|
|
|
|
|
The conflation prevents a provider gateway from serving an open-ended model catalog. OpenRouter, for example, is one provider with many model ids, while a private OpenAI-compatible endpoint may add models without changing the Harness plugin tree. Every newly selected model currently needs to have been registered during plugin startup. The same model id can also exist at multiple providers, so model-only registration cannot state which provider the caller intended.
|
|
|
|
|
|
-`dsh-llm-pi-ai` exposed none of pi-ai's provider abstraction. It constructed an inline DeepSeek `openai-completions` model, applied DeepSeek-specific payload patches, and stamped every replayed assistant message as DeepSeek. pi-ai itself has a provider/model catalog, selects APIs such as `openai-responses`, `anthropic-messages`, and `google-generative-ai`, and preserves provider-specific response ids and reasoning/tool signatures for later turns. The Harness conversion dropped that provenance, so simply replacing the inline model with a catalog lookup would have made same-model replay and cross-provider handoff incomplete.
|
|
|
+`dsh-llm-pi-ai` exposed none of pi-ai's provider abstraction. It constructed an inline DeepSeek `openai-completions` model, applied DeepSeek-specific payload patches, and stamped every replayed assistant message as DeepSeek. pi-ai itself has a provider/model catalog, selects APIs such as `openai-responses`, `anthropic-messages`, and `google-generative-ai`, and preserves provider-specific response ids and reasoning/tool signatures for later turns. The Harness conversion dropped the provider/model route and provider response fields, so simply replacing the inline model with a catalog lookup would have made same-model replay and cross-provider handoff incomplete.
|
|
|
|
|
|
The adapter configuration also assumes one DeepSeek API key and endpoint. A generic backend needs independent credentials and endpoint overrides per provider while leaving AWS, Google ADC, OAuth, and other ambient authentication mechanisms to pi-ai.
|
|
|
|
|
|
@@ -36,15 +36,15 @@ The adapter calls pi-ai's `streamSimple()` so each catalog model chooses its reg
|
|
|
|
|
|
pi-ai's common stream options do not expose stop sequences. `dsh-llm-pi-ai` rejects a defined Harness `stop` option with `UNSUPPORTED_OPTION` rather than silently ignoring it or growing a second provider-specific payload implementation. `dsh-llm-deepseek` continues to support `stop` through its native request serializer.
|
|
|
|
|
|
-### Durable assistant provenance and replay state
|
|
|
+### Recorded assistant route and replay state
|
|
|
|
|
|
-Assistant messages carry provider-neutral provenance containing the request's `provider` and `model`, plus an optional JSON-serializable adapter replay state. A successful `assistant/message` session event records this provenance and `deriveMessages()` returns it with the assistant message. User, system, context, and tool-result messages carry no assistant provenance. The provider/model fields are authoritative loop data; an adapter owns only its opaque replay-state payload.
|
|
|
+Assistant messages carry the request's `provider` and `model`, plus an optional JSON-serializable adapter replay state. A successful `assistant/message` session event records those fields and `deriveMessages()` returns them with the assistant message. User, system, context, and tool-result messages carry no assistant route fields. The provider/model fields are authoritative loop data; an adapter owns only its opaque replay-state payload.
|
|
|
|
|
|
-A terminal successful `finish` chunk may carry replay state, and `BlockAssembler` retains it alongside usage and finish reason. The loop attaches that state to the assembled assistant provenance without exposing a response-rewrite hook. Error and aborted responses do not produce a normal assistant message and therefore do not enter future model history.
|
|
|
+A terminal successful `finish` chunk may carry replay state, and `BlockAssembler` retains it alongside usage and finish reason. The loop attaches that state to the assembled assistant message's model source without exposing a response-rewrite hook. Error and aborted responses do not produce a normal assistant message and therefore do not enter future model history.
|
|
|
|
|
|
-The pi-ai replay state is a versioned, minimal projection of its successful `AssistantMessage`: source API/provider/model, response id/model, stop reason, and index-aligned text, thinking, and tool-call signatures. It does not duplicate text or tool arguments already carried by Harness content blocks, and it omits diagnostics, timestamps, usage, and errors. On a later request, `LlmService` gives replay state to the target adapter only when the historical provider and target provider are currently owned by the same adapter instance. That adapter combines the logged Harness content with replay state when it can restore the historical response, and owns any required cross-model or cross-provider conversion. An adapter receiving replay state with an unknown version or mismatched block shape fails explicitly; a different adapter receives only provider-neutral content and provenance.
|
|
|
+The pi-ai replay state is a versioned, minimal projection of its successful `AssistantMessage`: source API/provider/model, response id/model, stop reason, and index-aligned text, thinking, and tool-call signatures. It does not duplicate text or tool arguments already carried by Harness content blocks, and it omits diagnostics, timestamps, usage, and errors. On a later request, `LlmService` gives replay state to the target adapter only when the historical provider and target provider are currently owned by the same adapter instance. That adapter combines the logged Harness content with replay state when it can restore the historical response, and owns any required cross-model or cross-provider conversion. An adapter receiving replay state with an unknown version or mismatched block shape fails explicitly; a different adapter receives only provider-neutral content plus provider/model fields.
|
|
|
|
|
|
-This state is model-visible replay input and therefore follows the existing [reconstructable-request rule](2026-07-05-reconstructable-requests.md): it is present in both the terminal `finish` chunk and the assembled `assistant/message` provenance that drives derivation. Resume and fork preserve it verbatim. Compaction that shadows the assistant message also removes its replay state from the active surface; the summary is ordinary provider-neutral content.
|
|
|
+This state is model-visible replay input and therefore follows the existing [reconstructable-request rule](2026-07-05-reconstructable-requests.md): it is present in both the terminal `finish` chunk and the assembled `assistant/message` model source that drives derivation. Resume and fork preserve it verbatim. Compaction that shadows the assistant message also removes its replay state from the active surface; the summary is ordinary provider-neutral content.
|
|
|
|
|
|
### Propagate the target through every request producer
|
|
|
|
|
|
@@ -54,7 +54,7 @@ Compaction configuration gains `summarizationProvider` beside `summarizationMode
|
|
|
|
|
|
The JSON-RPC runtime receives provider and model explicitly. Its convenience fallback mounts `dsh-llm-deepseek` only for provider `deepseek` when that provider has no registered owner; other missing providers fail without guessing an adapter.
|
|
|
|
|
|
-The on-disk session format remains the pre-release pinned version `0`, with no compatibility promise. Seed/load validation rejects request headers lacking provider and assistant messages lacking required provenance instead of accepting an old shape that can no longer reconstruct the request.
|
|
|
+The on-disk session format remains the pre-release pinned version `0`, with no compatibility promise. Seed/load validation rejects request headers and assistant messages that omit required provider/model fields instead of accepting an old shape that can no longer reconstruct the request.
|
|
|
|
|
|
## Alternatives considered
|
|
|
|
|
|
@@ -78,7 +78,7 @@ The on-disk session format remains the pre-release pinned version `0`, with no c
|
|
|
- pi-ai credentials, transport knobs, SDK timeouts, and the five-minute-default `streamIdleTimeoutMs` watchdog are scoped per provider profile. Hidden provider retries are disabled; bounded retries belong to the separately composed agent recovery policy.
|
|
|
- `dsh-llm-pi-ai` rejects stop sequences because pi-ai's common stream API cannot express them; the native DeepSeek adapter retains its stop support.
|
|
|
- Replay state is portable only within the adapter instance that owns both the historical and target providers. Cross-provider and cross-model restoration is an adapter responsibility, and another adapter receives provider-neutral history without the opaque state.
|
|
|
-- Current pre-release session JSONL requires provider/model request headers and assistant provenance. Older shapes remain version `0` but are rejected rather than migrated.
|
|
|
+- Current pre-release session JSONL requires provider/model on request headers and assistant messages. Older shapes remain version `0` but are rejected rather than migrated.
|
|
|
|
|
|
## Testing
|
|
|
|