Просмотр исходного кода

Merge branch 'master' into feat/session-completed-dot

GeeeekExplorer 1 месяц назад
Родитель
Сommit
ccc06f4e61

+ 6 - 0
.agents/notes/implemented/architecture/2026-08-06-web-shell-dist-chunk-layout.i18n.yaml

@@ -0,0 +1,6 @@
+# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
+# side as of the last confirmed-consistent state. Both languages carry equal authority;
+# after editing either side, bring the other along and re-record with:
+#   pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-06-web-shell-dist-chunk-layout.md
+2026-08-06-web-shell-dist-chunk-layout.md: 1c7b4273dc243685317b149e2fd7fddf2a6c18d1
+2026-08-06-web-shell-dist-chunk-layout.zh.md: 6f4b94e0bd7412e480458e34922273b389aa8892

+ 50 - 0
.agents/notes/implemented/architecture/2026-08-06-web-shell-dist-chunk-layout.md

@@ -0,0 +1,50 @@
+# Agent Note: Web shell dist chunk split and directory layout
+
+Status: implemented
+
+English | [中文](2026-08-06-web-shell-dist-chunk-layout.zh.md)
+
+## Problem
+
+The apps/web shell previously built into a single ~1.2 MB (minified) index chunk, roughly 80% of it vendor bytes — KaTeX, the boot grammars and the shiki engine, react-dom, the markdown pipeline — fused with all the workspace shell code (about one fifth). Any one-line shell change rehashed the whole chunk, forcing returning clients to redownload everything; `dist/assets/` was a flat single-level spread of 100-plus files (the main chunk, 23 lazy-loaded grammar chunks, 59 KaTeX font faces, and sourcemaps intermixed), impossible to navigate.
+
+## Decision
+
+`apps/web/vite.config.ts` splits the shell into two initial chunks via `manualChunks` and sorts the output into directories via naming functions; the entire configuration contains zero regexes — an exact-package-name Set, a filename list, an extension list.
+
+**Membership** (`VENDOR_PACKAGES`, by exact npm package name):
+
+- `vendor` = the three heavy rendering families: math (katex), highlight (shiki), markdown (the micromark/mdast parse pipeline — the incremental React renderer above it is workspace code and not part of this). The live membership is `VENDOR_PACKAGES`; the list is the packages workspace code **imports directly**: the remaining private transitive dependencies (the oniguruma family, @shikijs/core, character tables, dozens more) are referenced only by listed members, so rollup's chunk coloring pulls them into vendor automatically; dependencies shared with the index side fall back to index, diluting it by a few KB — not a correctness issue.
+- **Every vendor member must be react-free (the boundary invariant)**: rollup folds a module shared between the entry and a manual chunk into the manual chunk — one listed package importing react/jsx-runtime would drag the single shared react copy into vendor, away from index. The React side of markdown/math rendering is workspace code and naturally lives in index, so the whole react family stays pinned to index.
+- `index` (the default chunk) = the react family (react, react-dom, scheduler, use-sync-external-store), vendored cordis, all workspace code, and the unlisted small pieces (anser, clsx).
+- `@shikijs/langs` is special-cased: the boot grammars (`BOOT_GRAMMAR_FILES`: typescript, shellscript, json — the three that highlight.ts statically imports, all self-contained data modules with zero internal imports) go into vendor; the remaining 23 lazy-loaded grammars get no assignment and each keeps its own on-demand chunk.
+- `index.html` is wired up automatically by vite: index loads via `<script>` and vendor via `<link rel="modulepreload">`, so the two chunks fetch in parallel with no waterfall.
+
+**Directory layout** (`chunkFileNames` + `assetFileNames`):
+
+- The `assets/` root keeps only the index and vendor js (with their adjacent sourcemaps) and css.
+- Grammar chunks go under `assets/langs/`. The criterion is whether a chunk's `moduleIds` include an `@shikijs/langs` member, not the facade: the shared chunks of embedded grammars (php/ruby/mdx embed html+javascript, which rollup splits out for sharing) **have no facade**, so a facade criterion would miss them; index and vendor are excluded by name, because vendor legitimately carries the three boot grammars.
+- Fonts go under `assets/fonts/` (`FONT_EXTENSIONS`: woff2/woff/ttf; today all of them are KaTeX faces referenced by vendor.css — katex.min.css is imported by an index-side component, but CSS modules go through manualChunks like any module and follow `katex` into vendor.css; the browser fetches only woff2, on demand and only when a formula renders).
+- Sourcemaps need no arrangement: rollup writes each `.map` next to its js and references it by bare relative filename, so when a chunk moves directories its map follows automatically.
+
+All cross-directory references (index's dynamic imports into `langs/`, same-directory relative references among grammar chunks, vendor.css's relative references into `fonts/`) are emitted by the bundler, so the runtime needs zero accompanying changes; the host-side webserver serves the nested paths verbatim under its static prefix.
+
+## Alternatives considered
+
+- **Serving react and the other vendors from a CDN**: dsh web targets local/intranet hosts (often without internet access), so a CDN is simply unavailable; react is the platform seed external of every plugin bundle (the shell is its sole supplier), and switching to the CDN global-variable form would touch three places — the platform manifest, the seed, and the module table; the caching benefit is already delivered by the vendor split.
+- **An inverse catch-all rule (everything in node_modules except the react family goes to vendor)**: membership cannot be read off the configuration, and small pieces like anser/clsx get misassigned to vendor; superseded by the positive exact-package-name list.
+- **Regex family matching**: hard to read; exact package names plus rollup's automatic coloring of transitive dependencies make pattern matching unnecessary.
+- **Identifying grammar chunks by facadeModuleId**: the facade-less shared chunks of embedded grammars would go undetected and fall back to the root directory; the `moduleIds` membership criterion covers both shapes.
+- **Sheltering a react-edged rendering facade in vendor** (the historical react-markdown was one): rollup's shared-module folding would drag the single react copy into vendor, breaking the "react belongs to index" boundary; the constraint is codified as the list's boundary invariant.
+- **Lazy-loading KaTeX wholesale, or turning the boot TypeScript grammar lazy**: either would change first-frame rendering behavior (the fallback for formulas / the first code block); that trade-off is independent of the dist layout and is decided separately.
+
+## Verification
+
+The audit tool ships with the repository: `node scripts/attribute-chunk-bytes.mjs <chunk.js>` (zero-dependency sourcemap VLQ byte attribution, aggregated by npm package / workspace directory). It verifies that vendor contains no workspace bytes, that the react family (including react/jsx-runtime) sits entirely in index, and that the npm side of index retains only the react family plus anser/clsx; the lazy grammar chunk count matches the `LAZY_GRAMMARS` table one to one; the browser keyless replay case is verbatim-identical to the pre-change baseline (apart from environment-specific local reds), so the two-chunk shell loads and renders with no regression.
+
+## Consequences
+
+- A shell code change rehashes only index (about one third of the dist output); vendor (about two thirds) stays cache-stable across shell releases and is invalidated only by dependency upgrades.
+- `dist/assets/` is navigable: two js/css pairs at the root, on-demand grammars in `langs/`, fonts in `fonts/`.
+- Maintenance cost: when workspace code adds a direct import of a rendering family's facade package, `VENDOR_PACKAGES` must be updated alongside (an omission merely dilutes index, nothing breaks); when the boot grammar set grows in highlight.ts without `BOOT_GRAMMAR_FILES` following, that grammar silently lands in index, visible only to a dist audit.
+- The webserver's static surface has no compression yet, so the gzip size win is still on the table; transport-layer compression is a separate, independent decision.

+ 50 - 0
.agents/notes/implemented/architecture/2026-08-06-web-shell-dist-chunk-layout.zh.md

@@ -0,0 +1,50 @@
+# Agent Note: Web 壳产物的 chunk 切分与目录布局
+
+Status: implemented
+
+[English](2026-08-06-web-shell-dist-chunk-layout.md) | 中文
+
+## Problem
+
+apps/web 的壳此前打成单一约 1.2 MB(minified)的 index chunk,其中约八成是 vendor 字节——KaTeX、boot 语法与 shiki 引擎、react-dom、markdown 管线——与全部 workspace 壳代码(约五分之一)熔在一起。任何一行壳代码改动都让整个 chunk 换哈希,回头客户端全量重新下载;`dist/assets/` 是 100 多个文件的单层平铺(主 chunk、23 个懒加载语法 chunk、59 个 KaTeX 字体面、sourcemap 混居),无从导航。
+
+## Decision
+
+`apps/web/vite.config.ts` 以 `manualChunks` 把壳切成两个初始 chunk,并以输出命名函数归类目录;整套配置零正则——精确包名 Set、文件名清单、扩展名清单。
+
+**成员归属**(`VENDOR_PACKAGES`,按精确 npm 包名):
+
+- `vendor` = 三个重渲染家族:math(katex)、highlight(shiki)、markdown(micromark/mdast 解析管线——其上的增量 React 渲染器是 workspace 代码,不在此列)。成员以 `VENDOR_PACKAGES` 为活口径,清单 = workspace 代码**直接 import** 的包:其余私有传递依赖(oniguruma 系、@shikijs/core、字符表等数十个)只被清单成员引用,rollup 的 chunk 着色自动将其并入 vendor;与 index 侧共享的依赖回落 index,只稀释几 KB,不构成正确性问题。
+- **vendor 全员必须 react-free(边界不变量)**:rollup 会把入口与 manual chunk 共享的模块并入 manual chunk——清单里出现任何 import react/jsx-runtime 的包,唯一一份 react 副本就会被拽进 vendor、脱离 index。markdown/math 的 React 渲染侧是 workspace 代码天然住 index,react 族因此全部钉在 index。
+- `index`(默认 chunk)= react 族(react、react-dom、scheduler、use-sync-external-store)、vendored cordis、全部 workspace 代码及未列入的小件(anser、clsx)。
+- `@shikijs/langs` 特判:boot 语法(`BOOT_GRAMMAR_FILES`:typescript、shellscript、json——highlight.ts 静态 import 的三件,均为零内部 import 的自含数据模块)进 vendor;其余 23 个懒加载语法不做指派,各自保持按需 chunk。
+- `index.html` 由 vite 自动接线:index 走 `<script>`、vendor 走 `<link rel="modulepreload">`,两 chunk 并行拉取,无瀑布。
+
+**目录布局**(`chunkFileNames` + `assetFileNames`):
+
+- `assets/` 根只留 index 与 vendor 的 js(含随行 sourcemap)与 css。
+- 语法 chunk 归 `assets/langs/`。判据是 chunk 的 `moduleIds` 含 `@shikijs/langs` 成员,而非 facade:内嵌语法共享 chunk(php/ruby/mdx 内嵌 html+javascript,被 rollup 拆出共享)**没有 facade**,facade 判据会漏;index/vendor 按名排除,因 vendor 合法携带 boot 三语法。
+- 字体归 `assets/fonts/`(`FONT_EXTENSIONS`:woff2/woff/ttf;今日全部为 vendor.css 引用的 KaTeX 字面——katex.min.css 虽由 index 侧组件 import,css 模块同样经 manualChunks 归属、随 `katex` 落入 vendor.css;浏览器按需只拉 woff2,且仅在公式渲染时)。
+- sourcemap 无需安排:rollup 把 `.map` 写在各自 js 旁并以裸相对文件名引用,chunk 挪目录 map 自动跟随。
+
+跨目录引用(index 的动态 import 指向 `langs/`、语法 chunk 间同目录相对引用、vendor.css 相对引用 `fonts/`)均由构建器生成,运行时零配套改动;host 侧 webserver 按静态前缀原样服务嵌套路径。
+
+## Alternatives considered
+
+- **react 等 vendor 走 CDN**:dsh web 面向本机/内网主机(常无外网),CDN 直接不可用;react 是全部插件 bundle 的 platform seed external(壳是唯一供给方),改 CDN 全局变量形态需牵动 platform 清单/seed/模块表三处;缓存收益由 vendor 切分即可取得。
+- **反向兜底规则(node_modules 除 react 族全归 vendor)**:成员从配置上读不出来,且把 anser/clsx 类小件错归 vendor;被正向精确包名清单取代。
+- **正则家族匹配**:可读性差;精确包名 + rollup 对传递依赖的自动着色使模式匹配没有必要。
+- **以 facadeModuleId 识别语法 chunk**:无 facade 的内嵌语法共享 chunk 会漏检落回根目录;`moduleIds` 成员判据覆盖两种形态。
+- **在 vendor 里收留带 react 边的渲染门面**(历史上的 react-markdown 属此类):会经 rollup 的共享模块归并把唯一 react 副本拽进 vendor,破坏「react 归 index」的边界;该约束已成文为清单的边界不变量。
+- **KaTeX 整体懒加载、boot TypeScript 语法转懒**:会改变首帧渲染行为(公式/首个代码块的回退),是独立于产物布局的取舍,另行决策。
+
+## Verification
+
+审计工具随库:`node scripts/attribute-chunk-bytes.mjs <chunk.js>`(零依赖 sourcemap VLQ 字节归属,按 npm 包/workspace 目录聚合)。以其复核:vendor 不含任何 workspace 字节、react 族(含 react/jsx-runtime)全量位于 index、index 的 npm 侧仅剩 react 族与 anser/clsx;懒语法 chunk 数量与 `LAZY_GRAMMARS` 表一一对应;浏览器 keyless replay 用例与改动前基线逐字一致(本机环境性红除外),两 chunk 壳装载渲染无回归。
+
+## Consequences
+
+- 壳代码改动只重哈希 index(约为产物三分之一);vendor(约三分之二)跨壳版本缓存稳定,仅依赖升级时失效。
+- `dist/assets/` 可导航:根两对 js/css,`langs/` 按需语法,`fonts/` 字体。
+- 维护成本:workspace 代码新增对某渲染家族门面包的直接 import 时需同步 `VENDOR_PACKAGES`(漏列仅稀释 index,不致坏);在 highlight.ts 扩 boot 语法集而未同步 `BOOT_GRAMMAR_FILES` 时,该语法静默落入 index,仅产物审计可见。
+- webserver 静态面尚无压缩,gzip 体量是潜在值;传输层压缩是另一项独立决策。

+ 100 - 0
apps/web/vite.config.ts

@@ -18,10 +18,110 @@ function rejectStandaloneServe(): Plugin {
   }
 }
 
+/**
+ * Vendor-chunk membership, by exact npm package name — the heavy render
+ * families (math, highlight, markdown) that change only on dependency bumps.
+ * Only packages workspace code imports DIRECTLY need listing: their private
+ * transitive dependencies (oniguruma machinery, character tables, …) are
+ * imported solely by these and rollup's chunk coloring pulls them into
+ * vendor automatically. A dependency shared with index-side code falls back
+ * to index — a few kB of dilution, never a correctness problem. Anything not
+ * listed (react family, the vendored cordis workspace, tiny helpers like
+ * anser/clsx, all workspace code) stays in the default `index` chunk, so
+ * editing shell code re-hashes only index and returning clients keep the
+ * cached vendor chunk.
+ *
+ * Boundary invariant: every member must be react-free. A package that
+ * imports react/jsx-runtime must never be listed — rollup folds a module
+ * shared between the entry and a manual chunk into the manual chunk, so one
+ * react-importing member would drag the single shared react copy into
+ * vendor. The React side of markdown/math rendering is workspace code and
+ * rides index.
+ */
+const VENDOR_PACKAGES: ReadonlySet<string> = new Set([
+  // math
+  'katex',
+  // syntax highlight (@shikijs/langs is handled separately below —
+  // lazy grammars must not land here)
+  'shiki',
+  // markdown parse pipeline (micromark/mdast; the incremental React renderer
+  // over it is workspace code)
+  'mdast-util-from-markdown',
+  'mdast-util-gfm',
+  'mdast-util-math',
+  'micromark-core-commonmark',
+  'micromark-extension-gfm',
+  'micromark-extension-math',
+  'micromark-factory-space',
+  'micromark-util-character',
+  'micromark-util-classify-character',
+  'micromark-util-sanitize-uri',
+  'micromark-util-symbol',
+  'micromark-util-types',
+])
+
+/**
+ * Boot grammars statically imported by ui-primitives' highlight.ts
+ * (`@shikijs/langs/typescript` → `dist/typescript.mjs`, etc.). They live in
+ * the same package as the lazy read-card grammars, but unlike those they are
+ * part of the initial load and belong in the vendor chunk; the lazy ones must
+ * stay unassigned so each keeps its own on-demand chunk.
+ */
+const BOOT_GRAMMAR_FILES: readonly string[] = [
+  'dist/typescript.mjs',
+  'dist/shellscript.mjs',
+  'dist/json.mjs',
+]
+
+/** Font asset extensions routed to assets/fonts/ (KaTeX's woff2/woff/ttf faces today). */
+const FONT_EXTENSIONS: readonly string[] = ['.woff2', '.woff', '.ttf']
+
+/** npm package name of a resolved module id (the segment after the LAST `node_modules/` — pnpm nests the real package under an inner node_modules). */
+function npmPackageOf(id: string): string | undefined {
+  const parts = id.split('/node_modules/')
+  if (parts.length === 1) return undefined
+  const [first, second] = parts[parts.length - 1].split('/')
+  if (first.startsWith('.')) return undefined // .pnpm store segment, not a package
+  if (first.startsWith('@')) return second === undefined ? undefined : `${first}/${second}`
+  return first
+}
+
 export default defineConfig({
   plugins: [rejectStandaloneServe(), react()],
   build: {
     sourcemap: true,
+    rollupOptions: {
+      output: {
+        // Output layout: the two main chunks stay at assets/ root; lazy
+        // @shikijs/langs grammar chunks group under assets/langs/; fonts
+        // (today all KaTeX faces referenced by vendor.css) group under
+        // assets/fonts/. Sourcemaps need no arrangement: rollup writes each
+        // .map next to its js and references it by bare relative filename.
+        chunkFileNames(chunk): string {
+          // Grammar chunks are recognized by their member modules, not the
+          // facade: shared embedded-grammar chunks (e.g. html+javascript,
+          // split out because php/ruby/mdx embed them) have no facade at all.
+          // index and vendor are excluded by name — vendor legitimately
+          // carries the three boot grammars.
+          if (chunk.name === 'index' || chunk.name === 'vendor') return 'assets/[name]-[hash].js'
+          const isLangChunk = chunk.moduleIds.some(id => id.includes('/node_modules/@shikijs/langs/'))
+          return isLangChunk ? 'assets/langs/[name]-[hash].js' : 'assets/[name]-[hash].js'
+        },
+        assetFileNames(asset): string {
+          const fileName = asset.names[0] ?? ''
+          const isFont = FONT_EXTENSIONS.some(ext => fileName.endsWith(ext))
+          return isFont ? 'assets/fonts/[name]-[hash][extname]' : 'assets/[name]-[hash][extname]'
+        },
+        manualChunks(id: string): string | undefined {
+          const pkg = npmPackageOf(id)
+          if (pkg === undefined) return undefined // workspace + vendored cordis: index
+          if (pkg === '@shikijs/langs') {
+            return BOOT_GRAMMAR_FILES.some(file => id.endsWith(`/${file}`)) ? 'vendor' : undefined
+          }
+          return VENDOR_PACKAGES.has(pkg) ? 'vendor' : undefined
+        },
+      },
+    },
   },
   resolve: {
     // Workspace packages resolve to SOURCE: package.json exports point at lib

BIN
scripts/attribute-chunk-bytes.mjs