# Review Critical Fixes Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** 修复四份 review 报告交叉验证后的关键问题:CLI JSON 文件输入边界、rejected commit 投影、async 测试配置、KnowledgeQuery schema 漂移、StateManager 状态写入分叉、vector summary 覆盖、review 流程文档、安全边界。 **Architecture:** 先恢复写作主链和测试套件可信度,再扩展 RAG 投影覆盖,最后收紧本地 dashboard/backup 的安全默认值。每个任务都包含回归测试,避免只改实现不固定行为。 **Tech Stack:** Python 3.10+, pytest/pytest-asyncio, SQLite, FastAPI, Markdown skills --- ## File Structure ### Core Commit/Projection - `webnovel-writer/scripts/chapter_commit.py`: CLI entry point; rejected commit must also run state projection. - `webnovel-writer/scripts/data_modules/chapter_commit_service.py`: projection orchestration; non-accepted commits should still allow the state writer rejected branch. - `webnovel-writer/scripts/data_modules/event_projection_router.py`: writer selection; rejected commits require `state`. - `webnovel-writer/scripts/data_modules/tests/test_chapter_commit_service.py`: service-level projection behavior tests. - `webnovel-writer/scripts/data_modules/tests/test_projection_writers.py`: end-to-end projection regression tests. ### Test Infrastructure - `pytest.ini`: remove async plugin bans; keep coverage settings. - `webnovel-writer/scripts/data_modules/tests/test_rag_adapter.py`: existing async tests should run normally after config fix. ### CLI JSON Input Boundary - `webnovel-writer/scripts/data_modules/cli_args.py`: add optional base-dir containment for `@file` JSON arguments while keeping stdin/direct JSON behavior unchanged. - `webnovel-writer/scripts/data_modules/index_manager.py`: pass project-root containment to `load_json_arg()` for write/input JSON commands. - `webnovel-writer/scripts/data_modules/state_manager.py`: pass project-root containment to `load_json_arg()` for `process-chapter`. - `webnovel-writer/scripts/data_modules/sql_state_manager.py`: pass project-root containment to `load_json_arg()` for chapter entity processing. - `webnovel-writer/scripts/data_modules/memory/store.py`: pass project-root containment to `load_json_arg()` for memory upserts/imports. - `webnovel-writer/scripts/data_modules/rag_adapter.py`: pass project-root containment to `load_json_arg()` for scene indexing. - `webnovel-writer/scripts/data_modules/style_sampler.py`: pass project-root containment to `load_json_arg()` for scene extraction. - `webnovel-writer/scripts/data_modules/tests/test_coverage_boost.py`: extend existing `cli_args` tests. ### Knowledge Query - `webnovel-writer/scripts/data_modules/knowledge_query.py`: query production `relationship_events.type` while returning stable JSON for callers. - `webnovel-writer/scripts/data_modules/tests/test_knowledge_query.py`: use production schema, not mock-only `relationship_type`. ### State Manager - `webnovel-writer/scripts/data_modules/state_manager.py`: route `set_chapter_status()` through locked merge semantics. - `webnovel-writer/scripts/data_modules/tests/test_chapter_status.py`: preserve status monotonicity. - `webnovel-writer/scripts/data_modules/tests/test_state_manager_extra.py`: add merge regression for existing disk state. ### Vector Projection/RAG - `webnovel-writer/scripts/data_modules/vector_projection_writer.py`: add summary and scene chunks to commit projection, and avoid `asyncio.run()` failure when projection is invoked from an active event loop. - `webnovel-writer/scripts/data_modules/tests/test_vector_projection_writer.py`: cover summary/scene chunk generation, stable IDs, and active-event-loop storage bridge. ### Review Skill Flow - `webnovel-writer/skills/webnovel-review/SKILL.md`: switch to `review-pipeline --save-metrics`, remove obsolete second save command. - `webnovel-writer/skills/webnovel-write/SKILL.md`: verify already uses `--save-metrics`; no change unless wording drifts. ### Local Security Defaults - `webnovel-writer/dashboard/app.py`: restrict CORS to localhost origins, add file size limit for read API. - `webnovel-writer/scripts/backup_manager.py`: ensure generated `.gitignore` excludes `.env`. - `webnovel-writer/scripts/data_modules/style_sampler.py`: tolerate corrupt JSON in `tags`. - New or existing tests under `webnovel-writer/scripts/tests/` or `webnovel-writer/scripts/data_modules/tests/` for these fixes. --- ## Task 0: Bound `@file` JSON Arguments to Project Roots **Problem:** `load_json_arg("@path")` currently reads any local path. This is a local CLI feature rather than a remote vulnerability, but in Agent-driven pipelines the project root is the expected trust boundary. **Design:** Keep backward compatibility for direct callers by adding an optional `base_dir` argument. If `base_dir` is passed, `@file` must resolve inside it. `@-` stdin and direct JSON strings are unchanged. **Files:** - Modify: `webnovel-writer/scripts/data_modules/cli_args.py` - Modify: `webnovel-writer/scripts/data_modules/index_manager.py` - Modify: `webnovel-writer/scripts/data_modules/state_manager.py` - Modify: `webnovel-writer/scripts/data_modules/sql_state_manager.py` - Modify: `webnovel-writer/scripts/data_modules/memory/store.py` - Modify: `webnovel-writer/scripts/data_modules/rag_adapter.py` - Modify: `webnovel-writer/scripts/data_modules/style_sampler.py` - Test: `webnovel-writer/scripts/data_modules/tests/test_coverage_boost.py` - [ ] **Step 1: Add failing containment tests** Append these tests to the `cli_args` section of `webnovel-writer/scripts/data_modules/tests/test_coverage_boost.py`: ```python def test_load_json_arg_rejects_file_outside_base_dir(tmp_path): project = tmp_path / "project" project.mkdir() outside = tmp_path / "secret.json" outside.write_text('{"secret": true}', encoding="utf-8") with pytest.raises(ValueError, match="outside allowed directory"): load_json_arg(f"@{outside}", base_dir=project) def test_load_json_arg_allows_file_inside_base_dir(tmp_path): project = tmp_path / "project" project.mkdir() payload = project / "payload.json" payload.write_text('{"ok": true}', encoding="utf-8") assert load_json_arg(f"@{payload}", base_dir=project) == {"ok": True} def test_load_json_arg_stdin_ignores_base_dir(monkeypatch, tmp_path): monkeypatch.setattr(sys, "stdin", StringIO('{"stdin": true}')) assert load_json_arg("@-", base_dir=tmp_path) == {"stdin": True} ``` - [ ] **Step 2: Run failing cli_args tests** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_coverage_boost.py::test_load_json_arg_rejects_file_outside_base_dir webnovel-writer/scripts/data_modules/tests/test_coverage_boost.py::test_load_json_arg_allows_file_inside_base_dir webnovel-writer/scripts/data_modules/tests/test_coverage_boost.py::test_load_json_arg_stdin_ignores_base_dir -q --no-cov ``` Expected before implementation: fail with `TypeError: load_json_arg() got an unexpected keyword argument 'base_dir'`. - [ ] **Step 3: Implement optional containment helper** Update `webnovel-writer/scripts/data_modules/cli_args.py`: ```python def _resolve_json_arg_file(target: str, *, base_dir: str | Path | None = None) -> Path: path = Path(target).expanduser() if not path.is_absolute() and base_dir is not None: path = Path(base_dir) / path resolved = path.resolve() if base_dir is not None: base = Path(base_dir).expanduser().resolve() try: resolved.relative_to(base) except ValueError as exc: raise ValueError(f"json arg file outside allowed directory: {resolved}") from exc return resolved ``` Change the signature and file read branch: ```python def load_json_arg(raw: str, *, base_dir: str | Path | None = None) -> Any: """ 解析 CLI 传入的 JSON 参数,支持两种形式: - 直接 JSON 字符串:'{"a":1}' - @ 文件路径:'@data.json'(从文件读取 JSON,避免 shell 引号地狱) - 特例:'@-' 表示从 stdin 读取 - 当传入 base_dir 时,@ 文件必须位于 base_dir 内 """ if raw is None: raise ValueError("missing json arg") text = str(raw).strip() if text.startswith("@"): target = text[1:].strip() if not target: raise ValueError("invalid json arg: '@' without path") if target == "-": content = sys.stdin.read() else: content = _resolve_json_arg_file(target, base_dir=base_dir).read_text(encoding="utf-8") return json.loads(content) return json.loads(text) ``` - [ ] **Step 4: Pass project root from CLI call sites** For each call site that has `args.project_root`, pass it as `base_dir=args.project_root`. In `webnovel-writer/scripts/data_modules/index_manager.py`, update all `load_json_arg(...)` calls in command handlers, for example: ```python entities = load_json_arg(args.entities, base_dir=args.project_root) scenes = load_json_arg(args.scenes, base_dir=args.project_root) ``` and: ```python data = load_json_arg(args.data, base_dir=args.project_root) ``` In `webnovel-writer/scripts/data_modules/state_manager.py`: ```python data = load_json_arg(args.data, base_dir=args.project_root) ``` In `webnovel-writer/scripts/data_modules/sql_state_manager.py`: ```python data = load_json_arg(args.data, base_dir=args.project_root) ``` In `webnovel-writer/scripts/data_modules/memory/store.py`: ```python payload = load_json_arg(args.data, base_dir=args.project_root) ``` In `webnovel-writer/scripts/data_modules/rag_adapter.py`: ```python scenes = load_json_arg(args.scenes, base_dir=args.project_root) ``` In `webnovel-writer/scripts/data_modules/style_sampler.py`: ```python scenes = load_json_arg(args.scenes, base_dir=args.project_root) ``` If a specific module stores the resolved project root in `config.project_root` rather than `args.project_root`, use `base_dir=config.project_root`. - [ ] **Step 5: Run cli and unified CLI tests** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_coverage_boost.py webnovel-writer/scripts/data_modules/tests/test_webnovel_unified_cli.py -q --no-cov ``` Expected: all tests pass. - [ ] **Step 6: Commit** ```bash git add webnovel-writer/scripts/data_modules/cli_args.py webnovel-writer/scripts/data_modules/index_manager.py webnovel-writer/scripts/data_modules/state_manager.py webnovel-writer/scripts/data_modules/sql_state_manager.py webnovel-writer/scripts/data_modules/memory/store.py webnovel-writer/scripts/data_modules/rag_adapter.py webnovel-writer/scripts/data_modules/style_sampler.py webnovel-writer/scripts/data_modules/tests/test_coverage_boost.py git commit -m "fix: bound json file arguments to project root" ``` --- ## Task 1: Restore Rejected Commit State Projection **Problem:** `StateProjectionWriter` supports `rejected -> chapter_rejected`, but both CLI and service skip projections for non-accepted commits. **Files:** - Modify: `webnovel-writer/scripts/chapter_commit.py` - Modify: `webnovel-writer/scripts/data_modules/chapter_commit_service.py` - Modify: `webnovel-writer/scripts/data_modules/event_projection_router.py` - Test: `webnovel-writer/scripts/data_modules/tests/test_chapter_commit_service.py` - Test: `webnovel-writer/scripts/data_modules/tests/test_projection_writers.py` - [ ] **Step 1: Add failing service test for rejected projection** Append this test to `webnovel-writer/scripts/data_modules/tests/test_chapter_commit_service.py`: ```python import json def test_apply_projections_updates_state_for_rejected_commit(tmp_path): (tmp_path / ".webnovel").mkdir(parents=True, exist_ok=True) (tmp_path / ".webnovel" / "state.json").write_text("{}", encoding="utf-8") service = ChapterCommitService(tmp_path) payload = service.build_commit( chapter=7, review_result={"blocking_count": 1}, fulfillment_result={ "planned_nodes": ["进入坊市"], "covered_nodes": ["进入坊市"], "missed_nodes": [], "extra_nodes": [], }, disambiguation_result={"pending": []}, extraction_result={"state_deltas": [], "entity_deltas": [], "accepted_events": []}, ) projected = service.apply_projections(payload) state = json.loads((tmp_path / ".webnovel" / "state.json").read_text(encoding="utf-8")) assert projected["projection_status"]["state"] == "done" assert state["progress"]["chapter_status"]["7"] == "chapter_rejected" ``` - [ ] **Step 2: Run the failing test** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_chapter_commit_service.py::test_apply_projections_updates_state_for_rejected_commit -q --no-cov ``` Expected before implementation: fail because `projection_status["state"]` remains `pending` or `state.json` has no `chapter_rejected`. - [ ] **Step 3: Route rejected commits to state writer** Update `webnovel-writer/scripts/data_modules/event_projection_router.py`: ```python def required_writers(self, commit_payload: Dict) -> List[str]: writers: Set[str] = set() status = str((commit_payload.get("meta") or {}).get("status") or "") if status == "rejected": writers.add("state") return sorted(writers) if status == "accepted": writers.add("state") writers.add("index") if commit_payload.get("entity_deltas"): writers.add("index") if str(commit_payload.get("summary_text") or "").strip(): writers.add("summary") for event in commit_payload.get("accepted_events") or []: if not isinstance(event, dict): continue writers.update(self.route(event)) return sorted(writers) ``` - [ ] **Step 4: Allow service projection for rejected state only** Update the start of `ChapterCommitService.apply_projections()` in `webnovel-writer/scripts/data_modules/chapter_commit_service.py`: ```python def apply_projections(self, payload: Dict[str, Any]) -> Dict[str, Any]: status = str((payload.get("meta") or {}).get("status") or "") if status not in {"accepted", "rejected"}: return payload if status == "accepted": chapter = int((payload.get("meta") or {}).get("chapter") or 0) event_store = EventLogStore(self.project_root) payload["accepted_events"] = event_store.normalize_events( chapter, payload.get("accepted_events", []) ) event_store.write_events(chapter, payload["accepted_events"]) proposals = AmendProposalTrigger().check(chapter, payload.get("accepted_events", [])) if proposals: manager = IndexManager(DataModulesConfig.from_project_root(self.project_root)) with manager._get_conn() as conn: ensure_override_ledger_columns(conn) persist_amend_proposals(conn, chapter, proposals) conn.commit() ``` Keep the writer import block and writer loop after this block. This preserves event log writes and override proposals for accepted commits only, while letting rejected commits reach `StateProjectionWriter`. - [ ] **Step 5: Make CLI always call `apply_projections()`** Update `webnovel-writer/scripts/chapter_commit.py`: ```python service.persist_commit(payload) payload = service.apply_projections(payload) print(json.dumps(payload, ensure_ascii=False)) ``` - [ ] **Step 6: Run targeted projection tests** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_chapter_commit_service.py webnovel-writer/scripts/data_modules/tests/test_projection_writers.py -q --no-cov ``` Expected: all tests pass. - [ ] **Step 7: Commit** ```bash git add webnovel-writer/scripts/chapter_commit.py webnovel-writer/scripts/data_modules/chapter_commit_service.py webnovel-writer/scripts/data_modules/event_projection_router.py webnovel-writer/scripts/data_modules/tests/test_chapter_commit_service.py git commit -m "fix: project rejected chapter commits to state" ``` --- ## Task 2: Restore Async Pytest Execution **Problem:** `pytest.ini` disables `asyncio` and `anyio`, so `@pytest.mark.asyncio` tests fail or are not exercised correctly. **Files:** - Modify: `pytest.ini` - Verify: existing async tests under `webnovel-writer/scripts/data_modules/tests/test_rag_adapter.py` - [ ] **Step 1: Confirm current async failure** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_rag_adapter.py::test_store_and_search -q --no-cov ``` Expected before implementation: fail or warn due to disabled async plugin. - [ ] **Step 2: Update pytest config** Change `pytest.ini` to: ```ini [pytest] testpaths = webnovel-writer/scripts/data_modules/tests webnovel-writer/scripts/tests pythonpath = webnovel-writer/scripts asyncio_mode = auto addopts = -p no:debugging -p pytest_cov -q --cov --cov-report=term-missing --cov-fail-under=90 -p no:cacheprovider ``` Do not disable `pytest_asyncio` or `anyio`. - [ ] **Step 3: Run async-focused tests** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_rag_adapter.py -q --no-cov ``` Expected: async tests execute and pass. - [ ] **Step 4: Run full suite once** Run: ```bash python -m pytest -q ``` Expected: tests run with coverage enforcement. If failures remain, record exact failing tests before touching unrelated code. - [ ] **Step 5: Commit** ```bash git add pytest.ini git commit -m "test: enable pytest async plugins" ``` --- ## Task 3: Fix KnowledgeQuery Relationship Schema Drift **Problem:** Production table uses `relationship_events.type`; `KnowledgeQuery` and its tests use `relationship_type`. **Files:** - Modify: `webnovel-writer/scripts/data_modules/knowledge_query.py` - Modify: `webnovel-writer/scripts/data_modules/tests/test_knowledge_query.py` - [ ] **Step 1: Rewrite test fixture to production schema** In `test_knowledge_query.py`, replace the `relationship_events` table definition with: ```python conn.execute(""" CREATE TABLE IF NOT EXISTS relationship_events ( id INTEGER PRIMARY KEY AUTOINCREMENT, from_entity TEXT, to_entity TEXT, type TEXT NOT NULL, action TEXT DEFAULT '', polarity TEXT DEFAULT '', strength REAL DEFAULT 0.0, description TEXT, chapter INTEGER, scene_index INTEGER DEFAULT 0, evidence TEXT DEFAULT '', confidence REAL DEFAULT 1.0, created_at TEXT ) """) ``` Replace inserts with: ```python conn.execute( "INSERT INTO relationship_events (from_entity, to_entity, type, chapter) VALUES (?, ?, ?, ?)", ("hanli", "陈巧倩", "同门", 20), ) conn.execute( "INSERT INTO relationship_events (from_entity, to_entity, type, chapter) VALUES (?, ?, ?, ?)", ("hanli", "陈巧倩", "合作", 45), ) ``` Keep assertions against output key `relationship_type` for backward-compatible CLI JSON. - [ ] **Step 2: Run the failing KnowledgeQuery tests** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_knowledge_query.py -q --no-cov ``` Expected before implementation: fail with `no such column: relationship_type`. - [ ] **Step 3: Query production column with output compatibility** Update `KnowledgeQuery.entity_relationships_at_chapter()` in `knowledge_query.py`: ```python rows = conn.execute( """ SELECT from_entity, to_entity, type AS relationship_type, description, chapter FROM relationship_events WHERE (from_entity = ? OR to_entity = ?) AND chapter <= ? ORDER BY chapter ASC, id ASC """, (entity_id, entity_id, chapter), ).fetchall() ``` Leave returned JSON as: ```python "relationship_type": str(row["relationship_type"] or "").strip(), ``` - [ ] **Step 4: Run KnowledgeQuery and CLI-adjacent tests** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_knowledge_query.py webnovel-writer/scripts/data_modules/tests/test_webnovel_unified_cli.py -q --no-cov ``` Expected: pass. - [ ] **Step 5: Commit** ```bash git add webnovel-writer/scripts/data_modules/knowledge_query.py webnovel-writer/scripts/data_modules/tests/test_knowledge_query.py git commit -m "fix: query relationship events using production schema" ``` --- ## Task 4: Route Chapter Status Writes Through Locked Merge Semantics **Problem:** `set_chapter_status()` mutates in-memory state and calls `_save_state()`, bypassing `save_state()` locking and disk merge. **Files:** - Modify: `webnovel-writer/scripts/data_modules/state_manager.py` - Modify: `webnovel-writer/scripts/data_modules/tests/test_state_manager_extra.py` - Verify: `webnovel-writer/scripts/data_modules/tests/test_chapter_status.py` - [ ] **Step 1: Add failing locked-save regression tests** Append to `test_state_manager_extra.py`: ```python def test_set_chapter_status_uses_locked_save_state(temp_project, monkeypatch): manager = StateManager(temp_project, enable_sqlite_sync=False) called = {} def fake_save_state(): called["save_state"] = True def fail_direct_save(): raise AssertionError("set_chapter_status must use save_state()") monkeypatch.setattr(manager, "save_state", fake_save_state) monkeypatch.setattr(manager, "_save_state", fail_direct_save) manager.set_chapter_status(5, "chapter_drafted") assert called["save_state"] is True assert manager._pending_chapter_status == {"5": "chapter_drafted"} def test_set_chapter_status_preserves_existing_disk_state(temp_project): temp_project.state_file.write_text( json.dumps( { "progress": {"current_chapter": 4, "chapter_status": {"4": "chapter_committed"}}, "disambiguation_warnings": [{"chapter": 4, "mention": "宗主"}], }, ensure_ascii=False, ), encoding="utf-8", ) manager = StateManager(temp_project, enable_sqlite_sync=False) manager.set_chapter_status(5, "chapter_drafted") saved = json.loads(temp_project.state_file.read_text(encoding="utf-8")) assert saved["progress"]["chapter_status"]["4"] == "chapter_committed" assert saved["progress"]["chapter_status"]["5"] == "chapter_drafted" assert saved["disambiguation_warnings"] == [{"chapter": 4, "mention": "宗主"}] ``` - [ ] **Step 2: Run status tests before implementation** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_chapter_status.py webnovel-writer/scripts/data_modules/tests/test_state_manager_extra.py::test_set_chapter_status_uses_locked_save_state webnovel-writer/scripts/data_modules/tests/test_state_manager_extra.py::test_set_chapter_status_preserves_existing_disk_state -q --no-cov ``` Expected before implementation: `test_set_chapter_status_uses_locked_save_state` fails because current code calls `_save_state()` directly and has no `_pending_chapter_status`. - [ ] **Step 3: Add pending chapter status field** In `StateManager.__init__`, after `_pending_progress_chapter`, add: ```python self._pending_chapter_status: Dict[str, str] = {} ``` Update `has_pending` in `save_state()` to include: ```python self._pending_chapter_status, ``` - [ ] **Step 4: Merge pending chapter statuses inside `save_state()`** Inside the locked `with lock:` block, immediately after `progress` is normalized for progress updates or before disambiguation merge, add: ```python if self._pending_chapter_status: progress = disk_state.get("progress", {}) if not isinstance(progress, dict): progress = {} disk_state["progress"] = progress chapter_status = progress.get("chapter_status") if not isinstance(chapter_status, dict): chapter_status = {} progress["chapter_status"] = chapter_status chapter_status.update(self._pending_chapter_status) progress["last_updated"] = self._now_progress_timestamp() ``` - [ ] **Step 5: Clear pending chapter statuses only after successful write** Where `save_state()` clears other pending structures after `atomic_write_json`, add: ```python self._pending_chapter_status.clear() ``` Place it beside the existing pending clears, not before SQLite sync error handling. - [ ] **Step 6: Update `set_chapter_status()`** Replace the final mutation/write block with: ```python progress = self._state.setdefault("progress", {}) chapter_status = progress.setdefault("chapter_status", {}) chapter_status[str(chapter)] = status self._pending_chapter_status[str(chapter)] = status self.save_state() ``` Keep monotonicity checks unchanged. - [ ] **Step 7: Run StateManager status tests** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_chapter_status.py webnovel-writer/scripts/data_modules/tests/test_state_manager_extra.py -q --no-cov ``` Expected: pass. - [ ] **Step 8: Commit** ```bash git add webnovel-writer/scripts/data_modules/state_manager.py webnovel-writer/scripts/data_modules/tests/test_state_manager_extra.py git commit -m "fix: merge chapter status updates under state lock" ``` --- ## Task 5: Add Summary/Scene Chunks and Safe Async Bridge to Vector Projection **Problem:** Commit projection writes event/entity chunks only. RAG already supports `summary` and `scene`, but commit projection does not feed them. The writer also calls `asyncio.run()` directly, which fails if this synchronous projection path is ever invoked inside an active event loop. **Files:** - Modify: `webnovel-writer/scripts/data_modules/vector_projection_writer.py` - Modify: `webnovel-writer/scripts/data_modules/tests/test_vector_projection_writer.py` - [ ] **Step 1: Add failing test for summary and scene chunks** Append to `test_vector_projection_writer.py`: ```python def test_collect_chunks_includes_summary_and_scenes(): writer = VectorProjectionWriter.__new__(VectorProjectionWriter) payload = { "meta": {"chapter": 47, "status": "accepted"}, "summary_text": "韩立在坊市发现丹方线索。", "scenes": [ {"index": 1, "summary": "韩立入坊市观察摊位", "location": "坊市"}, {"scene_index": 2, "content": "陈巧倩暗中提醒韩立有人跟踪。"}, ], "accepted_events": [], "entity_deltas": [], } chunks = writer._collect_chunks(payload) by_type = {chunk["chunk_type"]: chunk for chunk in chunks} assert by_type["summary"]["chunk_id"] == "ch0047_summary" assert by_type["summary"]["parent_chunk_id"] is None assert by_type["scene"]["parent_chunk_id"] == "ch0047_summary" assert any(chunk["scene_index"] == 2 for chunk in chunks if chunk["chunk_type"] == "scene") ``` - [ ] **Step 2: Run failing vector test** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_vector_projection_writer.py::test_collect_chunks_includes_summary_and_scenes -q --no-cov ``` Expected before implementation: fail because no summary/scene chunks exist. - [ ] **Step 3: Add summary chunk collection** At the start of `_collect_chunks()` after `chunk_counts`, add: ```python summary_text = str(commit_payload.get("summary_text") or "").strip() summary_chunk_id = f"ch{chapter:04d}_summary" if chapter > 0 else "" if chapter > 0 and summary_text: chunks.append({ "chunk_id": summary_chunk_id, "chapter": chapter, "scene_index": 0, "content": summary_text, "chunk_type": "summary", "parent_chunk_id": None, "source_file": f"commit:chapter_{chapter:03d}", }) ``` - [ ] **Step 4: Add scene chunk collection** After the summary block, add: ```python for idx, scene in enumerate(commit_payload.get("scenes") or [], start=1): if not isinstance(scene, dict): continue scene_index = int(scene.get("scene_index") or scene.get("index") or idx) text = str(scene.get("summary") or scene.get("content") or "").strip() location = str(scene.get("location") or "").strip() if location and text: text = f"{location}:{text}" if not text: continue chunk_id = self._chunk_id("scene", chapter, scene_index) chunks.append({ "chunk_id": chunk_id, "chapter": chapter, "scene_index": scene_index, "content": text, "chunk_type": "scene", "parent_chunk_id": summary_chunk_id or None, "source_file": f"commit:chapter_{chapter:03d}", }) ``` - [ ] **Step 5: Run vector and RAG tests** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_vector_projection_writer.py webnovel-writer/scripts/data_modules/tests/test_rag_adapter.py -q --no-cov ``` Expected: pass before the async bridge change, except any failures introduced by summary/scene implementation should be fixed before continuing. - [ ] **Step 6: Add active event loop regression test** Append to `test_vector_projection_writer.py`: ```python import pytest @pytest.mark.asyncio async def test_run_store_coro_works_inside_active_event_loop(): writer = VectorProjectionWriter.__new__(VectorProjectionWriter) async def store(): return 3 assert writer._run_store_coro(store()) == 3 ``` - [ ] **Step 7: Run failing active-loop test** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_vector_projection_writer.py::test_run_store_coro_works_inside_active_event_loop -q --no-cov ``` Expected before implementation: fail because `_run_store_coro` does not exist. - [ ] **Step 8: Implement safe coroutine bridge** In `webnovel-writer/scripts/data_modules/vector_projection_writer.py`, add imports: ```python import threading from collections.abc import Coroutine ``` Add this helper method to `VectorProjectionWriter`: ```python def _run_store_coro(self, coro: Coroutine[Any, Any, int]) -> int: try: asyncio.get_running_loop() except RuntimeError: return int(asyncio.run(coro) or 0) result: dict[str, Any] = {} def runner() -> None: try: result["value"] = asyncio.run(coro) except Exception as exc: result["error"] = exc thread = threading.Thread(target=runner, daemon=True) thread.start() thread.join() if "error" in result: raise result["error"] return int(result.get("value") or 0) ``` Then change `_store_chunks()` from: ```python stored = asyncio.run(adapter.store_chunks(chunks)) return stored ``` to: ```python return self._run_store_coro(adapter.store_chunks(chunks)) ``` This keeps the synchronous public API intact while moving the coroutine to a short-lived thread when the caller already owns an event loop. - [ ] **Step 9: Run vector and RAG tests** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_vector_projection_writer.py webnovel-writer/scripts/data_modules/tests/test_rag_adapter.py -q --no-cov ``` Expected: pass. - [ ] **Step 10: Commit** ```bash git add webnovel-writer/scripts/data_modules/vector_projection_writer.py webnovel-writer/scripts/data_modules/tests/test_vector_projection_writer.py git commit -m "feat: project summaries and scenes to vectors safely" ``` --- ## Task 6: Unify Review Skill Metrics Flow **Problem:** CLI supports `review-pipeline --save-metrics`; `webnovel-review` still documents the old two-step flow. **Files:** - Modify: `webnovel-writer/skills/webnovel-review/SKILL.md` - Verify: `webnovel-writer/skills/webnovel-write/SKILL.md` - [ ] **Step 1: Update common mistakes** In `webnovel-review/SKILL.md`, replace: ```markdown - ❌ 把 report 文件生成等同于已落库(`save-review-metrics` 未跑) ``` with: ```markdown - ❌ 把 report 文件生成等同于已落库(`review-pipeline --save-metrics` 未跑) ``` - [ ] **Step 2: Replace Step 5 command block** Replace the two-command standard flow with: ```bash python -X utf8 "${SCRIPTS_DIR}/webnovel.py" --project-root "${PROJECT_ROOT}" review-pipeline \ --chapter {chapter_num} \ --review-results "${PROJECT_ROOT}/.webnovel/tmp/review_results.json" \ --metrics-out "${PROJECT_ROOT}/.webnovel/tmp/review_metrics.json" \ --report-file "审查报告/第{chapter_num}章审查报告.md" \ --save-metrics ``` Replace the requirement: ```markdown - `review-pipeline` 生成的 `review_metrics.json` 必须可直接写入 `review_metrics` 表 ``` with: ```markdown - `review-pipeline --save-metrics` 必须完成报告生成、metrics 文件输出、`review_metrics` 表写入 ``` - [ ] **Step 3: Grep for obsolete instruction** Run: ```bash rg -n "save-review-metrics|--save-metrics" webnovel-writer/skills/webnovel-review/SKILL.md webnovel-writer/skills/webnovel-write/SKILL.md ``` Expected: `webnovel-review` no longer instructs a separate `index save-review-metrics` call; both skills mention `--save-metrics`. - [ ] **Step 4: Commit** ```bash git add webnovel-writer/skills/webnovel-review/SKILL.md git commit -m "docs: unify review metrics persistence flow" ``` --- ## Task 7: Tighten Local Dashboard and Backup Safety Defaults **Problem:** Dashboard allows `*` CORS while exposing local project text. Backup-created `.gitignore` does not exclude `.env`. **Files:** - Modify: `webnovel-writer/dashboard/app.py` - Modify: `webnovel-writer/scripts/backup_manager.py` - Test: add `webnovel-writer/scripts/tests/test_dashboard_security.py` - Test: add `webnovel-writer/scripts/tests/test_backup_manager.py` - [ ] **Step 1: Add dashboard security tests** Create `webnovel-writer/scripts/tests/test_dashboard_security.py`: ```python from fastapi.testclient import TestClient from dashboard.app import create_app def test_dashboard_cors_allows_localhost_origin(tmp_path): (tmp_path / ".webnovel").mkdir(parents=True) (tmp_path / ".webnovel" / "state.json").write_text("{}", encoding="utf-8") app = create_app(tmp_path) client = TestClient(app) response = client.options( "/api/project/info", headers={ "Origin": "http://localhost:5173", "Access-Control-Request-Method": "GET", }, ) assert response.headers["access-control-allow-origin"] == "http://localhost:5173" def test_dashboard_cors_rejects_untrusted_origin(tmp_path): (tmp_path / ".webnovel").mkdir(parents=True) (tmp_path / ".webnovel" / "state.json").write_text("{}", encoding="utf-8") app = create_app(tmp_path) client = TestClient(app) response = client.options( "/api/project/info", headers={ "Origin": "https://example.com", "Access-Control-Request-Method": "GET", }, ) assert "access-control-allow-origin" not in response.headers ``` - [ ] **Step 2: Restrict CORS origins** In `dashboard/app.py`, add module-level constant: ```python LOCAL_CORS_ORIGINS = [ "http://localhost", "http://localhost:5173", "http://localhost:8000", "http://127.0.0.1", "http://127.0.0.1:5173", "http://127.0.0.1:8000", ] ``` Change middleware setup to: ```python app.add_middleware( CORSMiddleware, allow_origins=LOCAL_CORS_ORIGINS, allow_methods=["GET"], allow_headers=["*"], ) ``` - [ ] **Step 3: Add file read size guard** In `file_read()`, before `read_text`, add: ```python max_bytes = 2 * 1024 * 1024 if resolved.stat().st_size > max_bytes: raise HTTPException(413, "文件过大,无法预览") ``` - [ ] **Step 4: Add backup gitignore test** Create `webnovel-writer/scripts/tests/test_backup_manager.py`: ```python import subprocess from backup_manager import GitBackupManager def test_backup_manager_gitignore_excludes_env(tmp_path, monkeypatch): calls = [] def fake_run(args, cwd=None, check=False, capture_output=False, text=False): calls.append(args) if args == ["git", "init"]: (tmp_path / ".git").mkdir() return subprocess.CompletedProcess(args=args, returncode=0, stdout="", stderr="") monkeypatch.setattr("backup_manager.is_git_available", lambda: True) monkeypatch.setattr("backup_manager.subprocess.run", fake_run) GitBackupManager(str(tmp_path)) gitignore = (tmp_path / ".gitignore").read_text(encoding="utf-8") assert ".env" in gitignore assert ".env.*" in gitignore assert "!.env.example" in gitignore ``` - [ ] **Step 5: Update backup `.gitignore` template** In `backup_manager.py`, add this block to the generated `.gitignore`: ```gitignore # Env (keep .env.example) .env .env.* !.env.example ``` - [ ] **Step 6: Run security tests** Run: ```bash python -m pytest webnovel-writer/scripts/tests/test_dashboard_security.py webnovel-writer/scripts/tests/test_backup_manager.py -q --no-cov ``` Expected: pass. - [ ] **Step 7: Commit** ```bash git add webnovel-writer/dashboard/app.py webnovel-writer/scripts/backup_manager.py webnovel-writer/scripts/tests/test_dashboard_security.py webnovel-writer/scripts/tests/test_backup_manager.py git commit -m "fix: tighten dashboard cors and backup gitignore defaults" ``` --- ## Task 8: Harden StyleSampler Tag JSON Parsing **Problem:** A corrupt `tags` JSON value in `style_samples.db` crashes listing. **Files:** - Modify: `webnovel-writer/scripts/data_modules/style_sampler.py` - Modify: `webnovel-writer/scripts/data_modules/tests/test_style_sampler_cli.py` - [ ] **Step 1: Add corrupt tag regression test** Append to `test_style_sampler_cli.py`: ```python def test_style_sampler_ignores_corrupt_tag_json(temp_project): sampler = StyleSampler(temp_project) with sampler._get_conn() as conn: conn.execute( """ INSERT INTO style_samples (id, chapter, scene_type, content, score, tags, created_at) VALUES (?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP) """, ("bad-tags", 1, SceneType.BATTLE.value, "战斗描写" * 50, 0.8, "[bad-json"), ) conn.commit() samples = sampler.get_best_samples(limit=5) assert samples[0].id == "bad-tags" assert samples[0].tags == [] ``` If the schema column order differs, inspect `_ensure_db()` in `style_sampler.py` and adjust column names exactly. - [ ] **Step 2: Run failing test** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_style_sampler_cli.py::test_style_sampler_ignores_corrupt_tag_json -q --no-cov ``` Expected before implementation: fail with `json.JSONDecodeError`. - [ ] **Step 3: Add safe JSON helper** In `style_sampler.py`, add: ```python def _safe_json_list(raw) -> list: if not raw: return [] try: value = json.loads(raw) except (TypeError, json.JSONDecodeError): return [] return value if isinstance(value, list) else [] ``` Change row mapping from: ```python tags=json.loads(row[5]) if row[5] else [], ``` to: ```python tags=_safe_json_list(row[5]), ``` - [ ] **Step 4: Run style sampler tests** Run: ```bash python -m pytest webnovel-writer/scripts/data_modules/tests/test_style_sampler_cli.py -q --no-cov ``` Expected: pass. - [ ] **Step 5: Commit** ```bash git add webnovel-writer/scripts/data_modules/style_sampler.py webnovel-writer/scripts/data_modules/tests/test_style_sampler_cli.py git commit -m "fix: tolerate corrupt style sample tags" ``` --- ## Task 9: Final Verification **Files:** read-only verification, plus fixes if tests reveal regressions. - [ ] **Step 1: Compile Python sources** Run: ```bash python -m compileall -q webnovel-writer/scripts webnovel-writer/dashboard ``` Expected: exit code 0. - [ ] **Step 2: Run focused regression suite** Run: ```bash python -m pytest \ webnovel-writer/scripts/data_modules/tests/test_chapter_commit_service.py \ webnovel-writer/scripts/data_modules/tests/test_coverage_boost.py \ webnovel-writer/scripts/data_modules/tests/test_projection_writers.py \ webnovel-writer/scripts/data_modules/tests/test_knowledge_query.py \ webnovel-writer/scripts/data_modules/tests/test_chapter_status.py \ webnovel-writer/scripts/data_modules/tests/test_state_manager_extra.py \ webnovel-writer/scripts/data_modules/tests/test_vector_projection_writer.py \ webnovel-writer/scripts/data_modules/tests/test_style_sampler_cli.py \ webnovel-writer/scripts/tests/test_dashboard_security.py \ webnovel-writer/scripts/tests/test_backup_manager.py \ -q --no-cov ``` Expected: all pass. - [ ] **Step 3: Run full suite** Run: ```bash python -m pytest -q ``` Expected: all tests pass and coverage is at least 90%. - [ ] **Step 4: Inspect review skill command consistency** Run: ```bash rg -n "save-review-metrics|review-pipeline|--save-metrics" webnovel-writer/skills/webnovel-review/SKILL.md webnovel-writer/skills/webnovel-write/SKILL.md ``` Expected: both review flows use `review-pipeline --save-metrics`; no active instruction requires separate `index save-review-metrics`. - [ ] **Step 5: Commit final test adjustments if needed** If verification required small test-only fixes: ```bash git add git commit -m "test: cover critical review fixes" ``` If no further changes were needed, do not create an empty commit. --- ## Out of Scope for This Plan - Full `/webnovel-resync` implementation for manually edited chapters. - Full automatic review-fix loop. - Full SQLite migration framework. - Large refactors of `index_manager.py`, `status_reporter.py`, or `DataModulesConfig`. - Dashboard frontend performance work beyond security defaults. - Anti-AI/style quality product line improvements. These should be planned separately after the correctness and test-trust fixes land.