Pārlūkot izejas kodu

fix(code-runtime-python): raise the output-budget worst-case multiple to 12 and reject the boundary

The load-time output-budget/addressSpaceMb gate used a worst-case multiple of 8,
assuming two simultaneous ~4x astral copies (the built string and its encode).
Three are live at the peak: on the newline path a single write holds the caller's
text argument, the line slice handed to push, and push's encode copy; the
settlement flush_line path held the pending chunks, their join, and that encode
copy. A budget admitted at 8x (e.g. maxLogBytes 48 MiB against addressSpaceMb 512)
could still OOM the child. The multiple is now 12, the strict `>` is `>=` so a
budget whose peak exactly equals the room left after the interpreter baseline is
rejected (that peak plus the baseline is the whole address space), and flush_line
drops the pending chunks before its push to match the newline path's
join-clear-push order. The child re-check mirror and both note sides move in step;
config-catalog is regenerated from the updated field JSDoc.
Chinesezjc 1 mēnesi atpakaļ
vecāks
revīzija
9d9525549d

+ 2 - 2
.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md
-2026-07-31-code-runtime-python-settlement-fixes.md: 8dc9a1e991f89efe0607f858dbbcc77929f582ce
-2026-07-31-code-runtime-python-settlement-fixes.zh.md: beb6e1bf4381f542ae33ed36a301ae31e3eabc10
+2026-07-31-code-runtime-python-settlement-fixes.md: 088b26397765b908cfbf3514fe020301a0a19235
+2026-07-31-code-runtime-python-settlement-fixes.zh.md: 1d45ceffda823f8cc1fb15f6cfb0a3bcef1688ad

Failā izmaiņas netiks attēlotas, jo tās ir par lielu
+ 0 - 0
.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md


Failā izmaiņas netiks attēlotas, jo tās ir par lielu
+ 0 - 0
.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.zh.md


+ 12 - 8
docs/config-catalog.md

@@ -379,24 +379,28 @@ export interface Config {
    * cleanly. Not applied on Darwin, where the dyld shared cache mapped into
    * every process at exec exceeds any practical cap and the kernel rejects
    * the call; `cpuSeconds` and `maxWallMs` still bound the run there. Bounds
-   * `maxLogBytes`/`maxValueBytes` at load on EVERY platform (not just where the
-   * limit is enforced): each budget times a worst-case Unicode expansion must
-   * fit this byte count, so a near-budget output cannot breach the address space
-   * during the child's build-and-encode.
+   * `maxLogBytes`/`maxValueBytes` at load on EVERY platform (this static check
+   * runs on Darwin too, where only the runtime `setrlimit` is skipped): each
+   * budget times a worst-case Unicode expansion must fit this byte count minus a
+   * fixed interpreter baseline, so a near-budget output cannot breach the address
+   * space during the child's build-and-encode.
    */
   addressSpaceMb?: number
   /**
    * Shared byte budget for captured log text (host-side ledger). Bounded at load
    * against `addressSpaceMb`: the child builds and encodes a near-budget entry
-   * under RLIMIT_AS, so this cap times the worst-case Unicode expansion must fit
-   * the address space (see `addressSpaceMb`).
+   * under RLIMIT_AS with several copies live at once, so this cap times the
+   * worst-case Unicode expansion must fit the address space left after the
+   * interpreter baseline (see `addressSpaceMb`) — a load-time rejection, not a
+   * runtime clamp.
    */
   maxLogBytes?: number
   /**
    * Byte cap for the completion value. Bounded at load against `addressSpaceMb`
    * the same way `maxLogBytes` is: the child builds and encodes a near-budget
-   * value under RLIMIT_AS, so this cap times the worst-case Unicode expansion
-   * must fit the address space.
+   * value under RLIMIT_AS with several copies live at once, so this cap times the
+   * worst-case Unicode expansion must fit the address space left after the
+   * interpreter baseline.
    */
   maxValueBytes?: number
   /** SIGTERM→SIGKILL grace period on kill, matching bash-local's default. */

+ 16 - 7
packages/code-runtime/code-runtime-python/py/bootstrap.py

@@ -51,11 +51,13 @@ _MAX_FALLBACK_NAME_CHARS = 200
 # Mirror of the host's output-budget/address-space gate (src/index.ts's
 # OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE and INTERPRETER_BASELINE_BYTES),
 # re-applied against the EFFECTIVE RLIMIT_AS after inheritance clamping. An astral
-# character is one character but ~4 bytes of str storage and ~4 UTF-8 bytes, live
-# at once while the ledger charges and frames it, so a budget's worst-case peak is
-# eight times its byte count; the interpreter's own footprint is reserved on top.
-# Kept in sync with the host constants by the shared reasoning, not a wire field.
-_OUTPUT_BUDGET_WORST_CASE_MULTIPLE = 8
+# character is one character but ~4 bytes of str storage and ~4 UTF-8 bytes, and
+# three such copies are live at the peak — the caller's write argument, the line
+# slice or joined pending handed to push, and the encode copy push takes — so a
+# budget's worst-case peak is twelve times its byte count; the interpreter's own
+# footprint is reserved on top. Kept in sync with the host constants by the shared
+# reasoning, not a wire field.
+_OUTPUT_BUDGET_WORST_CASE_MULTIPLE = 12
 _INTERPRETER_BASELINE_BYTES = 64 * 1024 * 1024
 
 
@@ -350,9 +352,16 @@ class _LogStream(io.TextIOBase):
         # against them.
         with self._logs.lock:
             if self._pending:
-                self._logs.push("".join(self._pending))
+                # Join, drop the chunks, THEN push — the same order the newline
+                # path uses (:232-235). Pushing before the clear would keep the
+                # pending chunks alive through `_push_locked`'s `text.encode`, so
+                # the chunks, their join, and the encode copy would all be live at
+                # once; dropping the chunks first leaves only the join and its
+                # encode, matching that path's peak.
+                line = "".join(self._pending)
                 self._pending = []
                 self._pending_chars = 0
+                self._logs.push(line)
 
 
 # ---------------------------------------------------------------------------
@@ -664,7 +673,7 @@ async def _run(channel: ProtocolChannel) -> None:
             if effective_soft != resource.RLIM_INFINITY:
                 budgetable = effective_soft - _INTERPRETER_BASELINE_BYTES
                 for _budget_key in ("maxLogBytes", "maxValueBytes"):
-                    if int(boot[_budget_key]) * _OUTPUT_BUDGET_WORST_CASE_MULTIPLE > budgetable:
+                    if int(boot[_budget_key]) * _OUTPUT_BUDGET_WORST_CASE_MULTIPLE >= budgetable:
                         raise ValueError(
                             "config.%s is too large for the inherited RLIMIT_AS of %d bytes "
                             "(a near-budget output would breach it during encode); "

+ 52 - 38
packages/code-runtime/code-runtime-python/src/index.ts

@@ -57,24 +57,28 @@ export interface Config {
    * cleanly. Not applied on Darwin, where the dyld shared cache mapped into
    * every process at exec exceeds any practical cap and the kernel rejects
    * the call; `cpuSeconds` and `maxWallMs` still bound the run there. Bounds
-   * `maxLogBytes`/`maxValueBytes` at load on EVERY platform (not just where the
-   * limit is enforced): each budget times a worst-case Unicode expansion must
-   * fit this byte count, so a near-budget output cannot breach the address space
-   * during the child's build-and-encode.
+   * `maxLogBytes`/`maxValueBytes` at load on EVERY platform (this static check
+   * runs on Darwin too, where only the runtime `setrlimit` is skipped): each
+   * budget times a worst-case Unicode expansion must fit this byte count minus a
+   * fixed interpreter baseline, so a near-budget output cannot breach the address
+   * space during the child's build-and-encode.
    */
   addressSpaceMb?: number
   /**
    * Shared byte budget for captured log text (host-side ledger). Bounded at load
    * against `addressSpaceMb`: the child builds and encodes a near-budget entry
-   * under RLIMIT_AS, so this cap times the worst-case Unicode expansion must fit
-   * the address space (see `addressSpaceMb`).
+   * under RLIMIT_AS with several copies live at once, so this cap times the
+   * worst-case Unicode expansion must fit the address space left after the
+   * interpreter baseline (see `addressSpaceMb`) — a load-time rejection, not a
+   * runtime clamp.
    */
   maxLogBytes?: number
   /**
    * Byte cap for the completion value. Bounded at load against `addressSpaceMb`
    * the same way `maxLogBytes` is: the child builds and encodes a near-budget
-   * value under RLIMIT_AS, so this cap times the worst-case Unicode expansion
-   * must fit the address space.
+   * value under RLIMIT_AS with several copies live at once, so this cap times the
+   * worst-case Unicode expansion must fit the address space left after the
+   * interpreter baseline.
    */
   maxValueBytes?: number
   /** SIGTERM→SIGKILL grace period on kill, matching bash-local's default. */
@@ -236,20 +240,24 @@ const CLOSE_REAP_MARGIN_MS = 2_000
  * as a multiple of the budget. The child's ledgers trigger on CHARACTER count
  * against a serialized-BYTE budget, and an astral character is one character but
  * four bytes of CPython `str` storage and four UTF-8 bytes — so a budget's worth
- * of astral characters is ~4x the budget in the built string and ~4x again in
- * the `encode` copy taken to measure or ship it, live at the same time (the
- * concat that briefly holds both is bounded by those two). Eight covers that
- * simultaneous pair. The interpreter baseline is NOT in this multiple — it is
- * reserved separately as {@link INTERPRETER_BASELINE_BYTES} — because it is a
- * fixed cost, not one that scales with the budget. Used to bound
- * `maxLogBytes`/`maxValueBytes` against `addressSpaceMb` at load, with a STRICT
- * `>` so a budget whose worst-case peak exactly equals the room left after the
- * baseline is rejected, so a legitimate near-budget output truncates (log) or
- * fails as `output-limit` (value) rather than breaching `RLIMIT_AS` as
- * `worker-exit`. A fixed safety invariant tying the budgets to the address
- * space, not a knob.
+ * of astral characters is ~4x the budget in each string that holds it. THREE
+ * such copies are live at the peak: on the newline path a single
+ * `sys.stdout.write(line + "\n")` holds the caller's `text` argument (alive for
+ * the whole `write` call, ~4x), the line slice `text[pos:newline]` handed to
+ * `LogBuffer.push` (~4x), and the `text.encode("utf-8")` copy `_push_locked`
+ * takes to charge and ship it (~4x); the settlement `flush_line` path holds the
+ * pending chunks, their `"".join(...)`, and that same encode copy. Twelve covers
+ * those three simultaneous ~4x copies. The interpreter baseline is NOT in this
+ * multiple — it is reserved separately as {@link INTERPRETER_BASELINE_BYTES} —
+ * because it is a fixed cost, not one that scales with the budget. Used to bound
+ * `maxLogBytes`/`maxValueBytes` against `addressSpaceMb` at load, with a `>=` so
+ * a budget whose worst-case peak exactly equals the room left after the baseline
+ * is rejected (that peak plus the baseline is the whole address space, the
+ * RLIMIT_AS edge), so a legitimate near-budget output truncates (log) or fails
+ * as `output-limit` (value) rather than breaching `RLIMIT_AS` as `worker-exit`.
+ * A fixed safety invariant tying the budgets to the address space, not a knob.
  */
-const OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE = 8
+const OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE = 12
 
 /**
  * Fixed address-space headroom reserved for the CPython interpreter itself
@@ -728,20 +736,21 @@ export class PythonCodeRuntime extends CodeRuntime {
     // `maxValueBytes` completion value under `RLIMIT_AS`, and both paths trigger
     // on CHARACTER count against a serialized-BYTE budget. An astral character is
     // one character but four bytes of `str` storage and four UTF-8 bytes, so a
-    // budget's worth of them peaks at several simultaneous ~4x copies (the built
-    // string, the concat that still references it, and the encode taken to
-    // measure or ship it). A budget approaching `addressSpaceMb` therefore makes
-    // a LEGITIMATE near-budget output breach the address space and die as
-    // `worker-exit` instead of truncating (log) or failing as `output-limit`
-    // (value). Metering every child write against the address space at runtime is
-    // the wrong fix — an exact serialized-cost check is either a full encode (the
-    // allocation being avoided) or a per-character Python loop that burns the CPU
-    // budget — so the incompatible pair is rejected at load: each budget times the
-    // worst-case multiple must fit the address space. Checked on every platform,
-    // not just where `RLIMIT_AS` is enforced: the incompatibility is a property of
-    // the config values, and the child OOMs on a Linux deployment regardless of
-    // the host that assembled the config, so a uniform load-time rejection is the
-    // fail-loud contract (Darwin skips only the runtime `setrlimit`).
+    // budget's worth of them peaks at three simultaneous ~4x copies (the caller's
+    // write argument, the line slice or joined pending handed to push, and the
+    // encode push takes to charge and ship it). A budget approaching
+    // `addressSpaceMb` therefore makes a LEGITIMATE near-budget output breach the
+    // address space and die as `worker-exit` instead of truncating (log) or
+    // failing as `output-limit` (value). Metering every child write against the
+    // address space at runtime is the wrong fix — an exact serialized-cost check
+    // is either a full encode (the allocation being avoided) or a per-character
+    // Python loop that burns the CPU budget — so the incompatible pair is rejected
+    // at load: each budget times the worst-case multiple must fit the address
+    // space. Checked on every platform, not just where `RLIMIT_AS` is enforced:
+    // the incompatibility is a property of the config values, and the child OOMs
+    // on a Linux deployment regardless of the host that assembled the config, so a
+    // uniform load-time rejection is the fail-loud contract (Darwin skips only the
+    // runtime `setrlimit`).
     const addressSpaceBytes = this.config.addressSpaceMb * 1024 * 1024
     // Room left for the peak output allocation after the interpreter's own fixed
     // footprint. A budget must fit MULTIPLE times over into THIS, not the whole
@@ -749,10 +758,15 @@ export class PythonCodeRuntime extends CodeRuntime {
     // the multiple alone would admit — cannot leave the peak plus the interpreter
     // over the limit.
     const budgetableBytes = addressSpaceBytes - INTERPRETER_BASELINE_BYTES
-    const admissibleBudget = Math.floor(budgetableBytes / OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE)
+    // The largest budget that fits: the peak (budget * MULTIPLE) must leave room,
+    // so a budget whose peak exactly equals `budgetableBytes` is rejected — that
+    // peak plus the reserved baseline is the whole address space, the RLIMIT_AS
+    // edge. `ceil(budgetableBytes / MULTIPLE) - 1` is the last integer strictly
+    // under `budgetableBytes / MULTIPLE`.
+    const admissibleBudget = Math.ceil(budgetableBytes / OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE) - 1
     for (const key of ['maxLogBytes', 'maxValueBytes'] as const) {
-      if (this.config[key] * OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE > budgetableBytes) {
-        throw new Error(`dsh-code-runtime-python: config.${key} times the ${OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE}x worst-case Unicode expansion must fit the ${budgetableBytes} bytes left after the ${INTERPRETER_BASELINE_BYTES}-byte interpreter baseline within the ${addressSpaceBytes}-byte addressSpaceMb, so a near-budget output truncates rather than breaching RLIMIT_AS as worker-exit; got ${String(this.config[key])} against a limit of ${admissibleBudget}`)
+      if (this.config[key] * OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE >= budgetableBytes) {
+        throw new Error(`dsh-code-runtime-python: config.${key} times the ${OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE}x worst-case Unicode expansion must fit within the ${budgetableBytes} bytes left after the ${INTERPRETER_BASELINE_BYTES}-byte interpreter baseline within the ${addressSpaceBytes}-byte addressSpaceMb, so a near-budget output truncates rather than breaching RLIMIT_AS as worker-exit; got ${String(this.config[key])} against a limit of ${admissibleBudget}`)
       }
     }
     ctx.effect(() => () => this.teardown(), 'python code-runtime teardown')

+ 33 - 23
packages/code-runtime/code-runtime-python/tests/runtime.spec.ts

@@ -119,7 +119,7 @@ describe('PythonCodeRuntime — seam descriptors and misuse', () => {
     // The boundary value itself loads: the bound is the largest cap a frame can
     // still carry, not one below it. It needs an address space large enough to
     // clear the separate maxValueBytes/addressSpaceMb worst-case gate (the cap
-    // times the 8x Unicode expansion must fit), so this pairs it with a 4 GiB
+    // times the 12x Unicode expansion must fit), so this pairs it with a 4 GiB
     // addressSpaceMb — the two load-time bounds are independent.
     const boundary = await ctx.plugin(PythonCodeRuntime, { maxValueBytes: admissible, addressSpaceMb: 4096 })
     await boundary.dispose()
@@ -418,11 +418,11 @@ describe('PythonCodeRuntime — inherited resource limits', () => {
     // never gets, so a near-budget output would OOM mid-run as an opaque
     // worker-exit. The bootstrap re-checks both budgets against the EFFECTIVE
     // clamped limit and fails loud at boot instead. A 128 MiB inherited limit
-    // leaves 64 MiB budgetable (8 MiB admissible), under which a 32 MiB
-    // maxLogBytes — admitted by the 512 MiB configured default — is rejected. The
-    // rejection surfaces as an 'exception' (bootstrap's setrlimit-phase failure
-    // class), not a mid-run OOM. The repro is Linux-only (macOS ignores
-    // `ulimit -v`); there the run proceeds.
+    // leaves 64 MiB budgetable (~5 MiB admissible under the 12x multiple), under
+    // which a 32 MiB maxLogBytes — admitted by the 512 MiB configured default — is
+    // rejected. The rejection surfaces as an 'exception' (bootstrap's
+    // setrlimit-phase failure class), not a mid-run OOM. The repro is Linux-only
+    // (macOS ignores `ulimit -v`); there the run proceeds.
     const dir = await mkdtemp(join(tmpdir(), 'dsh-rlimit-'))
     const wrapper = join(dir, 'python3-tight')
     await writeFile(wrapper, '#!/bin/sh\nulimit -v 131072\nexec python3 "$@"\n', { mode: 0o755 })
@@ -783,21 +783,30 @@ describe('PythonCodeRuntime — programs and bindings', () => {
     // The child builds, charges, and encodes a `maxLogBytes` log entry or a
     // `maxValueBytes` completion value under RLIMIT_AS, and both trigger on
     // character count against a serialized-byte budget — an astral character is
-    // one character but ~4 bytes stored and ~4 encoded, so a budget approaching
-    // the address space lets a legitimate near-budget output breach it and die as
-    // worker-exit. The incompatible pair is rejected at load: each budget times
-    // the worst-case multiple (8) must fit the address space LEFT after the fixed
-    // interpreter baseline. Against a 256 MiB address space that leaves 192 MiB
-    // budgetable (24 MiB admissible), so a 50 MB cap is far over; the default caps
-    // against 512 MiB are not. Both budgets are gated symmetrically — the value
-    // case sets a default-fitting maxLogBytes so the maxValueBytes check is what
-    // fires.
+    // one character but ~4 bytes stored and ~4 encoded, and THREE such copies are
+    // live at the peak (the caller's write argument, the slice/join handed to
+    // push, and the encode copy), so a budget approaching the address space lets a
+    // legitimate near-budget output breach it and die as worker-exit. The
+    // incompatible pair is rejected at load: each budget times the worst-case
+    // multiple (12) must fit the address space LEFT after the fixed interpreter
+    // baseline. Against a 256 MiB address space that leaves 192 MiB budgetable
+    // (~16 MiB admissible), so a 50 MB cap is far over; the default caps against
+    // 512 MiB are not. Both budgets are gated symmetrically — the value case sets
+    // a default-fitting maxLogBytes so the maxValueBytes check is what fires.
     const ctxLog = new Context()
     await expect(ctxLog.plugin(PythonCodeRuntime, { maxLogBytes: 50_000_000, addressSpaceMb: 256 }))
-      .rejects.toThrow(/maxLogBytes times the 8x worst-case Unicode expansion must fit/)
+      .rejects.toThrow(/maxLogBytes times the 12x worst-case Unicode expansion must fit/)
     const ctxValue = new Context()
     await expect(ctxValue.plugin(PythonCodeRuntime, { maxValueBytes: 50_000_000, addressSpaceMb: 256 }))
-      .rejects.toThrow(/maxValueBytes times the 8x worst-case Unicode expansion must fit/)
+      .rejects.toThrow(/maxValueBytes times the 12x worst-case Unicode expansion must fit/)
+    // Discriminates 12 from 8: a 48 MiB maxLogBytes against a 512 MiB address
+    // space leaves 448 MiB budgetable. 48*8 = 384 MiB fits (the old 8x multiple
+    // wrongly ADMITTED this), but 48*12 = 576 MiB does not — and this is exactly
+    // the config that OOMs, since a settlement flush holds the pending chunks,
+    // their join, and the encode copy at once (~12x). The 12x gate rejects it.
+    const ctxTwelve = new Context()
+    await expect(ctxTwelve.plugin(PythonCodeRuntime, { maxLogBytes: 48 * 1024 * 1024, addressSpaceMb: 512 }))
+      .rejects.toThrow(/maxLogBytes times the 12x worst-case Unicode expansion must fit/)
     // The default caps against the default 512 MiB address space load.
     const ok = new Context()
     const fiber = await ok.plugin(PythonCodeRuntime, { maxLogBytes: 65536, maxValueBytes: 32768, addressSpaceMb: 512 })
@@ -3588,15 +3597,16 @@ describe('PythonCodeRuntime — hostile peer', () => {
     expect(serialized).toBeLessThan(1024)
   })
 
-  it('charges the serialized cost child-side, so a control-heavy line truncates instead of breaching the address space', async () => {
+  it('charges the serialized cost child-side, so a control-heavy line truncates instead of being admitted whole', async () => {
     // The child's ledger must charge what the entry costs on the wire, not its
     // raw UTF-8 length: a NUL is one raw byte but six as its escape. A 24 MiB NUL
     // line clears the cheap char-count lower bound (24 MiB < 32 MiB budget), so
-    // charging raw bytes would ADMIT it and then encode a ~144 MiB escaped
-    // payload plus its UTF-8 copy — past the 384 MiB address space, killing the
-    // child (surfaced host-side as `worker-exit`) instead of truncating.
-    // Charging the serialized cost rejects it before any encode.
-    const { runtime } = await setup({ maxLogBytes: 32 * 1024 * 1024, addressSpaceMb: 384, maxWallMs: 20_000 })
+    // charging raw bytes would ADMIT it and emit a ~144 MiB escaped entry;
+    // charging the serialized cost (~144 MiB > the 32 MiB budget) rejects it
+    // before any encode and emits the marker instead. The address space (512 MiB,
+    // clearing the 12x load gate for a 32 MiB budget) is sized so the run loads;
+    // the gate separately guarantees a correctly-charged near-budget entry fits.
+    const { runtime } = await setup({ maxLogBytes: 32 * 1024 * 1024, addressSpaceMb: 512, maxWallMs: 20_000 })
     const result = await runtime.run({
       program: [
         'print("\\x00" * (24 * 1024 * 1024))',

Daži faili netika attēloti, jo izmaiņu fails ir pārāk liels