Jelajahi Sumber

fix(tools): count kinds of code point, not code points, and cover a hostile tool name

"the two code points CPython refuses" counted classes: NUL is one code
point, unpaired surrogates are the whole 2,048-wide D800-DFFF block. Say
kinds, in both the docstring and the test comment that mirrors it, and
restore the "odd" qualifier the test comment dropped -- an even trailing
backslash run does not eat the closing quote.

The soft-keyword test title still said "only special in statement
position", which the previous commit's own three-way split contradicts
for `case`: `case_block` is a clause head inside a `match` statement, not
a statement.

Add the case the subscript tool-name path lacked. A lone surrogate is
reachable in a name through JSON.parse of MCP wire JSON, and that path
has no UNPRINTABLE / LONE_SURROGATE fallback -- only the same ES2019
well-formed stringification the Literal path leans on.
Chinesezjc 2 bulan lalu
induk
melakukan
72991bbcdb

+ 5 - 4
packages/core/tools/src/py-types.ts

@@ -263,10 +263,11 @@ function childClassName(base: string, segment: string): string {
  * by a JS parser back into the same double.
  * by a JS parser back into the same double.
  *
  *
  * `JSON.stringify` is also what keeps this path's output parseable, and it is
  * `JSON.stringify` is also what keeps this path's output parseable, and it is
- * the only thing that does. It covers both classes of hazard: the two code
- * points CPython refuses anywhere in source — NUL among the C0 controls, and
- * unpaired surrogates under ES2019 well-formed stringification, which the
- * engines range guarantees — and the ones that break this line in particular,
+ * the only thing that does. It covers both classes of hazard: the two kinds of
+ * code point CPython refuses anywhere in source — NUL among the C0 controls,
+ * and the whole D800–DFFF unpaired-surrogate block, escaped under ES2019
+ * well-formed stringification, which the engines range guarantees — and the
+ * ones that break this line in particular,
  * a bare `"` closing the literal early, a trailing odd backslash eating the
  * a bare `"` closing the literal early, a trailing odd backslash eating the
  * closing quote, and a bare LF/CR ending it before its terminator. The
  * closing quote, and a bare LF/CR ending it before its terminator. The
  * `description` path carries {@link UNPRINTABLE} and {@link LONE_SURROGATE}
  * `description` path carries {@link UNPRINTABLE} and {@link LONE_SURROGATE}

+ 26 - 7
packages/core/tools/tests/py-types.spec.ts

@@ -53,15 +53,16 @@ describe('jsonSchemaToPy', () => {
 
 
   it('leans on JSON.stringify to keep a Literal parseable', () => {
   it('leans on JSON.stringify to keep a Literal parseable', () => {
     // Nothing here escapes anything itself; `JSON.stringify` carries both
     // Nothing here escapes anything itself; `JSON.stringify` carries both
-    // classes of hazard. The two code points CPython refuses anywhere in
-    // source: NUL, and a lone surrogate under ES2019 well-formed
-    // stringification.
+    // classes of hazard. The two kinds of code point CPython refuses anywhere
+    // in source: NUL, and the D800–DFFF unpaired-surrogate block under ES2019
+    // well-formed stringification.
     expect(jsonSchemaToPy({ type: 'string', const: 'a\u0000b' })).toBe(String.raw`Literal["a\u0000b"]`)
     expect(jsonSchemaToPy({ type: 'string', const: 'a\u0000b' })).toBe(String.raw`Literal["a\u0000b"]`)
     expect(jsonSchemaToPy({ type: 'string', enum: ['a\ud800b'] })).toBe(String.raw`Literal["a\ud800b"]`)
     expect(jsonSchemaToPy({ type: 'string', enum: ['a\ud800b'] })).toBe(String.raw`Literal["a\ud800b"]`)
     // And the ones that break this line in particular: a bare quote closing
     // And the ones that break this line in particular: a bare quote closing
-    // the literal early, a trailing backslash eating the closing quote, a bare
-    // newline ending it before its terminator. Every escape it emits is also a
-    // Python escape for the same character, so the value round-trips.
+    // the literal early, a trailing ODD backslash eating the closing quote (an
+    // even run does not), a bare newline ending it before its terminator.
+    // Every escape it emits is also a Python escape for the same character, so
+    // the value round-trips.
     expect(jsonSchemaToPy({ type: 'string', const: 'say "hi"\n' })).toBe(String.raw`Literal["say \"hi\"\n"]`)
     expect(jsonSchemaToPy({ type: 'string', const: 'say "hi"\n' })).toBe(String.raw`Literal["say \"hi\"\n"]`)
     expect(jsonSchemaToPy({ type: 'string', const: 'ends\\' })).toBe(String.raw`Literal["ends\\"]`)
     expect(jsonSchemaToPy({ type: 'string', const: 'ends\\' })).toBe(String.raw`Literal["ends\\"]`)
   })
   })
@@ -368,7 +369,7 @@ describe('renderToolsSdkPy', () => {
     expect(text).not.toContain('WeirdFieldsArgs')
     expect(text).not.toContain('WeirdFieldsArgs')
   })
   })
 
 
-  it('keeps soft-keyword field names as TypedDict fields (match/case/type are only special in statement position)', () => {
+  it('keeps soft-keyword field names as TypedDict fields (each is special in exactly one syntactic position)', () => {
     const tool: ToolSdkSchema = {
     const tool: ToolSdkSchema = {
       name: 'search',
       name: 'search',
       description: 'Soft keywords as fields.',
       description: 'Soft keywords as fields.',
@@ -740,6 +741,24 @@ describe('renderToolsSdkPy', () => {
     expect(text).toContain('    pass\n')
     expect(text).toContain('    pass\n')
   })
   })
 
 
+  it('quotes a tool name through the same JSON.stringify the Literal path depends on', () => {
+    // A lone surrogate is reachable in a name — `"\ud800"` survives
+    // `JSON.parse` of MCP wire JSON — and this path has no UNPRINTABLE /
+    // LONE_SURROGATE fallback behind it, only ES2019 well-formed
+    // stringification. Raw, it would make the whole SDK block uncompilable,
+    // exactly as on the `Literal[...]` path.
+    const text = renderToolsSdkPy([
+      {
+        name: 'a\ud800b',
+        description: 'Lone surrogate in the name.',
+        parameters: parameterSchemaSpecToJsonSchema({}) as unknown as Record<string, unknown>,
+        output: { type: 'string' },
+      },
+    ])
+    expect(text).toContain(String.raw`# tools["a\ud800b"](args: dict[str, Any]) -> str`)
+    expect(text).not.toContain('\ud800')
+  })
+
   it('escapes quotes and backslashes in descriptions so the docstring stays valid Python', () => {
   it('escapes quotes and backslashes in descriptions so the docstring stays valid Python', () => {
     // A description ending in `"` or an odd backslash would otherwise merge
     // A description ending in `"` or an odd backslash would otherwise merge
     // with (or escape) the closing triple quote — and this block is Code
     // with (or escape) the closing triple quote — and this block is Code