瀏覽代碼

fix(code-runtime-python): reject an oversized maxLogBytes at load instead of metering log capture at runtime

The child log ledger encodes an admitted entry to UTF-8 once to charge its
serialized cost, so a maxLogBytes approaching addressSpaceMb lets a legitimate
near-budget log entry breach RLIMIT_AS and die as worker-exit instead of
truncating. Two runtime fixes were tried and both traded one resource bound for
another: an exact serialized-cost check is either a full encode (the allocation
being avoided) or a per-character Python loop that burns the CPU budget (a 10 MB
write hits SIGXCPU under cpuSeconds:1). The breach is a property of the
maxLogBytes/addressSpaceMb pair, not any write, so reject the incompatible pair
at load — maxLogBytes must stay within one eighth of the addressSpaceMb byte
count — and revert _LogStream to its original character-count buffering, which
is memory-safe once the budget fits the address space. The check runs on every
platform since the incompatibility is a config-value property, not a runtime one.

Replace the child-flood regression tests (which asserted the reverted runtime
behavior) with a load-rejection test. The host-side accrueStrayCost UTF-8
per-lead validation and its tests are unaffected. Update the note and zh pair.
Chinesezjc 1 月之前
父節點
當前提交
2df88b5bbe

+ 2 - 2
.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.i18n.yaml

@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md
-2026-07-31-code-runtime-python-settlement-fixes.md: fc275bd13891c45641e69a773bbe5e05b1d294d3
-2026-07-31-code-runtime-python-settlement-fixes.zh.md: fca7d9245f60f623dc49162c0497ba5b68fedfba
+2026-07-31-code-runtime-python-settlement-fixes.md: 3ce090b8399311f3a27a03ea409379cbe55f2c54
+2026-07-31-code-runtime-python-settlement-fixes.zh.md: d14d71cb9390c6cfbdc196a0dd7323e7c6f3a53a

文件差異過大導致無法顯示
+ 0 - 0
.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md


文件差異過大導致無法顯示
+ 0 - 0
.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.zh.md


+ 13 - 101
packages/code-runtime/code-runtime-python/py/bootstrap.py

@@ -167,44 +167,10 @@ class _LogStream(io.TextIOBase):
         # ``print("x", end="")`` must not concatenate quadratically.
         self._pending: list[str] = []
         self._pending_chars = 0
-        # Running serialized JSON cost of the pending tail, maintained beside the
-        # character count so the early-flush trigger charges against ``remaining``
-        # (a serialized-byte budget) rather than undercharging control-char text.
-        # Accumulated per fragment through ``_fragment_cost_upto`` so no write
-        # re-scans the whole buffer.
-        self._pending_cost = 0
 
     def writable(self) -> bool:  # noqa: D401 -- inherited contract
         return True
 
-    @staticmethod
-    def _fragment_cost_upto(chunk: str, limit: int, start: int = 0, end: "int | None" = None) -> int:
-        # Serialized JSON cost of ``chunk[start:end]``'s characters (no enclosing
-        # quotes), indexing the str directly and STOPPING once the running total
-        # passes ``limit`` so the walk is bounded by a budget's worth of
-        # characters however large the write is. A control character serializes to
-        # up to six bytes, so a plain character count undercharges a control-char
-        # flood by up to 6x: 30M NUL characters stay under a 50 MB character budget
-        # yet serialize to ~180 MB, which the settlement flush would then allocate
-        # at once and breach RLIMIT_AS. Measuring the true serialized cost fixes
-        # that, but ``.encode`` to measure it would itself be the copy the
-        # ``_push_bounded_prefix`` path exists to avoid (a single 340 MiB write
-        # under a tight addressSpaceMb dies on that encode), and re-scanning the
-        # whole pending list per write would be quadratic under a daemon-thread
-        # flood — so the caller accumulates this per-fragment result once and the
-        # ``limit`` cap keeps each scan bounded. ``start``/``end`` weigh a
-        # sub-range without slicing it (the slice on a 340 MiB write would be that
-        # same copy); CPython ``str`` indexing is O(1) per character.
-        cost = 0
-        stop = len(chunk) if end is None else end
-        index = start
-        while index < stop:
-            cost += _json_char_cost(ord(chunk[index]))
-            if cost > limit:
-                return cost
-            index += 1
-        return cost
-
     def write(self, text: str) -> int:  # noqa: D401 -- inherited contract
         # Serialize the whole read-modify-write against the settlement flush and
         # any other thread's write: model code may spawn daemon threads that keep
@@ -241,16 +207,7 @@ class _LogStream(io.TextIOBase):
             pos = 0
             if self._pending:
                 newline = text.index("\n")
-                # Weigh the reconstructed first line by SERIALIZED cost, not
-                # character count: `_pending_cost` already holds the buffered
-                # chunks' cost, and the first line's cost is scanned up to the
-                # newline without slicing `text` (the slice on a 340 MiB write
-                # would be the copy this path avoids). A character-count check
-                # undercharged a control-char line — 30M NUL characters plus a
-                # newline pass `chars + 3 > remaining` under a 50 MB budget, then
-                # `_logs.push` would encode the 30M-char join and breach RLIMIT_AS.
-                first_line_cost = self._fragment_cost_upto(text, self._logs.remaining, end=newline)
-                if self._pending_cost + first_line_cost + 2 > self._logs.remaining:
+                if self._pending_chars + newline + 3 > self._logs.remaining:
                     # The reconstructed first line cannot fit the ledger, so
                     # LogBuffer would reject it whole: copy only the prefix that
                     # fails its cheap bound and drop the chunks. The slice is
@@ -265,7 +222,6 @@ class _LogStream(io.TextIOBase):
                     line = "".join(self._pending)
                     self._pending = []
                     self._pending_chars = 0
-                    self._pending_cost = 0
                     self._logs.push(line)
                 pos = newline + 1
             # Scan by offset and STOP once the ledger is exhausted: a single
@@ -278,17 +234,14 @@ class _LogStream(io.TextIOBase):
                 newline = text.find("\n", pos)
                 if newline < 0:
                     break
-                # Bound the SLICE by SERIALIZED cost, not character count: a line
-                # whose escaped form exceeds the ledger would be copied whole
-                # before push could reject it, and a control-char-dense line
-                # (30M NUL characters plus a newline) passes a `chars + 3 >
-                # remaining` check under a large budget yet encodes to ~6x that,
-                # so `_logs.push` would allocate the encode and breach RLIMIT_AS.
-                # `_fragment_cost_upto` scans up to the newline without slicing and
-                # stops at `remaining`, so an over-budget line takes the bounded
-                # prefix path; push still rejects that prefix on its own cheap
-                # bound, emits the marker, and never materializes the full line.
-                if self._fragment_cost_upto(text, self._logs.remaining, start=pos, end=newline) + 2 > self._logs.remaining:
+                # Bound the SLICE the same way LogBuffer bounds the encode: a
+                # first line far above the ledger would be copied whole before
+                # push could reject it, and that copy is the allocation an
+                # over-budget write cannot afford. Copy only a budget-sized
+                # prefix, which push still rejects on its own cheap bound (the
+                # prefix is longer than `remaining`), so the marker is emitted
+                # and the oversized line is never materialized.
+                if newline - pos + 3 > self._logs.remaining:
                     self._logs.push(text[pos:pos + self._logs.remaining + 4])
                     break
                 self._logs.push(text[pos:newline])
@@ -298,7 +251,6 @@ class _LogStream(io.TextIOBase):
                     tail = text[pos:]
                     self._pending.append(tail)
                     self._pending_chars = len(tail)
-                    self._pending_cost = self._fragment_cost_upto(tail, self._logs.remaining)
                 else:
                     # The ledger ran out with text still unscanned, so that text
                     # IS being dropped and the run must say so. One push is
@@ -318,21 +270,11 @@ class _LogStream(io.TextIOBase):
         else:
             self._pending.append(text)
             self._pending_chars += len(text)
-            # Add this fragment's serialized cost, capped so a single oversized
-            # write's scan stops at the budget rather than walking all of it.
-            self._pending_cost += self._fragment_cost_upto(text, self._logs.remaining)
         # A newline-free flood must hit the budget while running, not at
-        # settlement. `_pending_cost` weighs the buffered tail by its SERIALIZED
-        # cost: a control byte serializes to up to six bytes, so a character count
-        # undercharged control-char floods by up to 6x and a newline-free flood of
-        # ~30M NUL characters (each 1 char but 6 serialized bytes) stayed under a
-        # character-count trigger yet encoded to ~180 MB at settlement, breaching
-        # RLIMIT_AS. The per-fragment scan never encodes the whole buffer (a single
-        # oversized write must not be copied here, per the `_push_bounded_prefix`
-        # contract) nor re-scans the pending list per write (which would be
-        # quadratic under a daemon-thread flood), yet fires no later than a
-        # character count and strictly earlier for control-dense text.
-        if self._pending_cost > self._logs.remaining:
+        # settlement: once the buffered tail alone can no longer fit the
+        # ledger (chars lower-bound the serialized cost), push it through — LogBuffer
+        # truncates, emits the marker once, and swallows everything after.
+        if self._pending_chars > self._logs.remaining:
             self._push_bounded_prefix()
         return len(text)
 
@@ -365,7 +307,6 @@ class _LogStream(io.TextIOBase):
                 break
         self._pending = []
         self._pending_chars = 0
-        self._pending_cost = 0
         self._logs.push("".join(parts))
 
     def flush(self) -> None:  # noqa: D401 -- inherited contract
@@ -392,7 +333,6 @@ class _LogStream(io.TextIOBase):
                 self._logs.push("".join(self._pending))
                 self._pending = []
                 self._pending_chars = 0
-                self._pending_cost = 0
 
 
 # ---------------------------------------------------------------------------
@@ -1242,34 +1182,6 @@ for _escaped_byte, _surcharge in _JSON_ESCAPE_SURCHARGES:
     _JSON_BYTE_COST[_escaped_byte[0]] = 1 + _surcharge
 
 
-def _json_char_cost(code: int) -> int:
-    """Serialized JSON cost of one character, from its code point, without encoding.
-
-    Used by X a buffered write's true
-    serialized cost while scanning the str directly, so the early-flush trigger
-    charges a control character its full escaped width (a NUL is six bytes as
-    ``\\u0000``) rather than the single character a length count sees. A C0
-    control escapes to two bytes for the five shorthand forms or six for the
-    rest; ``"`` and ``\\`` escape to two; every other character stays at its raw
-    UTF-8 width (1/2/3 for the basic plane, 4 for an astral code point), which is
-    what the encoded form would hold. A lone surrogate is unreachable here — a
-    Python ``str`` character iterates as one code point and the caller's text has
-    already replaced any un-encodable surrogate.
-    """
-
-    if code < 0x20:
-        return 2 if code in (0x08, 0x09, 0x0a, 0x0c, 0x0d) else 6
-    if code == 0x22 or code == 0x5c:
-        return 2
-    if code < 0x80:
-        return 1
-    if code < 0x800:
-        return 2
-    if code < 0x10000:
-        return 3
-    return 4
-
-
 def _json_string_cost(raw: bytes) -> int:
     """UTF-8 byte length of one string's JSON form, WITHOUT building that form.
 

+ 35 - 0
packages/code-runtime/code-runtime-python/src/index.ts

@@ -216,6 +216,18 @@ const FRAME_ENVELOPE_BYTES = 64
  */
 const CLOSE_REAP_MARGIN_MS = 2_000
 
+/**
+ * The largest fraction of `addressSpaceMb` that `maxLogBytes` may claim, enforced
+ * at load. The child's log ledger encodes an admitted entry to UTF-8 once to
+ * charge its serialized cost, so a near-budget entry transiently needs the entry
+ * plus its encode copy — roughly twice `maxLogBytes` — on top of the interpreter
+ * baseline, all under `RLIMIT_AS`. One eighth leaves an 8x margin over the raw
+ * budget, comfortably past that transient at any admissible cap, so a legitimate
+ * near-budget log entry truncates instead of breaching the address space. A fixed
+ * safety invariant tying two configs together, not a deployment knob.
+ */
+const LOG_CAPTURE_ADDRESS_SPACE_HEADROOM_FRACTION = 1 / 8
+
 /**
  * Interval between process-group liveness probes while settlement waits for an
  * escalated SIGKILL to empty the group (see the `killing` branch in
@@ -676,6 +688,29 @@ export class PythonCodeRuntime extends CodeRuntime {
         throw new Error(`dsh-code-runtime-python: config.${key} must not exceed ${limit} (a payload that large cannot cross the ${FRAME_CEILING_BYTES}-byte fd-3 frame ceiling, so the run would fail as worker-exit rather than output-limit), got ${String(this.config[key])}`)
       }
     }
+    // The child's log ledger admits an entry up to `maxLogBytes` and, to charge
+    // its serialized cost, encodes it to UTF-8 once — a transient allocation of
+    // up to `maxLogBytes` more bytes (and a control-char-dense entry escapes up
+    // to sixfold on the wire, though the encode itself is the raw copy). That
+    // copy happens under `RLIMIT_AS`, so a `maxLogBytes` that approaches
+    // `addressSpaceMb` makes a legitimate near-budget log entry breach the
+    // address space and die as `worker-exit` instead of truncating. Rather than
+    // meter every child write against the address space at runtime — which trades
+    // the memory bound for a per-character CPU cost on the hot path — reject the
+    // incompatible pair at load: require `maxLogBytes` to leave the child room
+    // for the interpreter baseline plus the entry and its encode copy. The bound
+    // is `LOG_CAPTURE_ADDRESS_SPACE_HEADROOM_FRACTION` of the address space, well
+    // clear of the ~2x-plus-baseline the push path needs at the default 64 KiB
+    // cap. Checked on every platform, not just where `RLIMIT_AS` is enforced: the
+    // incompatibility is a property of the two config values, and the child OOMs
+    // on a Linux deployment regardless of the host that assembled the config, so
+    // a uniform load-time rejection is the fail-loud contract (Darwin skips only
+    // the runtime `setrlimit`, not this static check).
+    const addressSpaceBytes = this.config.addressSpaceMb * 1024 * 1024
+    const logCaptureCeiling = Math.floor(addressSpaceBytes * LOG_CAPTURE_ADDRESS_SPACE_HEADROOM_FRACTION)
+    if (this.config.maxLogBytes > logCaptureCeiling) {
+      throw new Error(`dsh-code-runtime-python: config.maxLogBytes must not exceed ${logCaptureCeiling} (${LOG_CAPTURE_ADDRESS_SPACE_HEADROOM_FRACTION} of the ${addressSpaceBytes}-byte addressSpaceMb, leaving the child room to encode a near-budget log entry without breaching RLIMIT_AS), got ${String(this.config.maxLogBytes)}`)
+    }
     ctx.effect(() => () => this.teardown(), 'python code-runtime teardown')
   }
 

+ 15 - 55
packages/code-runtime/code-runtime-python/tests/runtime.spec.ts

@@ -747,61 +747,21 @@ describe('PythonCodeRuntime — programs and bindings', () => {
     expect(result.logs.at(-1)).toBe(logTruncationMarker(4096))
   })
 
-  it('bounds a newline-free NUL flood through sys.stdout by serialized cost, not char count', async () => {
-    // `_LogStream` (the child's sys.stdout wrapper) buffers newline-free writes
-    // and early-flushes once the pending tail can no longer fit the ledger.
-    // Charging that trigger by CHARACTER count undercharged a control-char flood
-    // by up to 6x: NUL chars stay under a char-count trigger yet serialize to ~6x
-    // as many bytes, which the settlement "".join + encode then allocated at once.
-    // The program writes the flood in 1 MiB chunks (so no single argument str is
-    // itself the allocation under test) with no newline; the serialized-cost
-    // trigger flushes while running, keeping the pending tail bounded, so the run
-    // completes at the truncation marker. Pre-fix, the char-count trigger stayed
-    // dormant until ~200 MiB of chars accumulated, and the settlement encode of
-    // their ~1.2 GiB serialized form breached the 512 MiB RLIMIT_AS as a
-    // worker-exit. Driven through sys.stdout.write (not os.write, which bypasses
-    // the wrapper into host stray capture) to exercise the in-child stream.
-    // RLIMIT_AS is skipped on Darwin, so the worker-exit repro is Linux-only;
-    // on macOS this asserts the happy path, matching the control-char cases.
-    const { runtime } = await setup({ maxLogBytes: 20_000_000, addressSpaceMb: 512, maxWallMs: 30_000 })
-    const result = await runtime.run({
-      program: [
-        'import sys',
-        'chunk = "\\x00" * (1024 * 1024)',
-        'for _ in range(200):',
-        '    sys.stdout.write(chunk)',
-        'return None',
-      ].join('\n'),
-      bindings: [],
-    })
-    expect(result.error).toBeUndefined()
-    expect(result.logs.at(-1)).toBe(logTruncationMarker(20_000_000))
-  })
-
-  it('bounds a NEWLINE-terminated NUL flood through sys.stdout by serialized cost', async () => {
-    // The newline path of `_LogStream.write` scans and pushes each completed
-    // LINE. Its per-line fit check charged CHARACTER count, so a control-char
-    // line (a chunk of NULs ending in a newline) passed `chars + 3 > remaining`
-    // under a large budget yet `_logs.push` then encoded the whole line at
-    // settlement — the same RLIMIT_AS breach as the newline-free path, on a
-    // different branch. The check now weighs the line by serialized cost via
-    // `_fragment_cost_upto` (scanning to the newline without slicing), so an
-    // over-budget line takes the bounded-prefix path and the run truncates. Each
-    // 1 MiB NUL chunk is newline-terminated so it exercises the line branch;
-    // driven through sys.stdout.write, Linux-only RLIMIT_AS repro, macOS happy path.
-    const { runtime } = await setup({ maxLogBytes: 20_000_000, addressSpaceMb: 512, maxWallMs: 30_000 })
-    const result = await runtime.run({
-      program: [
-        'import sys',
-        'chunk = "\\x00" * (1024 * 1024) + "\\n"',
-        'for _ in range(200):',
-        '    sys.stdout.write(chunk)',
-        'return None',
-      ].join('\n'),
-      bindings: [],
-    })
-    expect(result.error).toBeUndefined()
-    expect(result.logs.at(-1)).toBe(logTruncationMarker(20_000_000))
+  it('rejects a maxLogBytes that could breach addressSpaceMb during log encode at load', async () => {
+    // The child's log ledger encodes an admitted entry to UTF-8 once to charge
+    // its serialized cost, so a `maxLogBytes` approaching `addressSpaceMb` lets a
+    // legitimate near-budget log entry breach RLIMIT_AS and die as worker-exit
+    // instead of truncating. The incompatible pair is rejected at load rather
+    // than metered per-write at runtime: `maxLogBytes` must stay within one
+    // eighth of the `addressSpaceMb` byte count. 50 MB against a 64 MiB address
+    // space is far over that bound; the default 64 KiB against 512 MiB is not.
+    const ctx = new Context()
+    await expect(ctx.plugin(PythonCodeRuntime, { maxLogBytes: 50_000_000, addressSpaceMb: 64 }))
+      .rejects.toThrow(/maxLogBytes must not exceed .* of the .*addressSpaceMb/)
+    // A compatible pair loads.
+    const ok = new Context()
+    const fiber = await ok.plugin(PythonCodeRuntime, { maxLogBytes: 65536, addressSpaceMb: 512 })
+    await fiber.dispose()
   })
 
   it('bounds an illegal-UTF-8 native residual by its U+FFFD-decoded cost', async () => {

部分文件因文件數量過多而無法顯示