feat(security): v2 ensemble tuning — label-first voting + SOLO_CONTENT_BLOCK

Cuts Haiku classifier false-positive rate from 44.1% → 22.9% on
BrowseSafe-Bench smoke. Detection trades from 67.3% → 56.2%; the
lost TPs are all cases Haiku correctly labeled verdict=warn
(phishing targeting users, not agent hijack) — they still surface
in the WARN banner meta but no longer kill the session.

Key changes:
- combineVerdict: label-first voting for transcript_classifier. Only
  meta.verdict==='block' block-votes; verdict==='warn' is a soft
  signal. Missing meta.verdict never block-votes (backward-compat).
- Hallucination guard: verdict='block' at confidence < LOG_ONLY (0.40)
  drops to warn-vote — prevents malformed low-conf blocks from going
  authoritative.
- New THRESHOLDS.SOLO_CONTENT_BLOCK = 0.92 decoupled from BLOCK (0.85).
  Label-less content classifiers (testsavant, deberta) need a higher
  solo-BLOCK bar because they can't distinguish injection from
  phishing-targeting-user. Transcript keeps label-gated solo path
  (verdict=block AND conf >= BLOCK).
- THRESHOLDS.WARN bumped 0.60 → 0.75 — borderline fires drop out of
  the 2-of-N ensemble pool.
- Haiku model pinned (claude-haiku-4-5-20251001). `claude -p` spawns
  from os.tmpdir() so project CLAUDE.md doesn't poison the classifier
  context (measured 44k cache_creation tokens per call before the fix,
  and Haiku refusing to classify because it read "security system"
  from CLAUDE.md and went meta).
- Haiku timeout 15s → 45s. Measured real latency is 17-33s end-to-end
  (Claude Code session startup + Haiku); v1's 15s caused 100% timeout
  when re-measured — v1's ensemble was effectively L4-only in prod.
- Haiku prompt rewritten: explicit block/warn/safe criteria, 8 few-shot
  exemplars (instruction-override → block; social engineering → warn;
  discussion-of-injection → safe).

Test updates:
- 5 existing combineVerdict tests adapted for label-first semantics
  (transcript signals now need meta.verdict to block-vote).
- 6 new tests: warn-soft-signal, three-way-block-with-warn-transcript,
  hallucination-guard-below-floor, above-floor-label-first,
  backward-compat-missing-meta.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

browse/test/security.test.ts

@@ -54,12 +54,12 @@ describe('combineVerdict — ensemble rule', () => {
 
   test('both ML layers at WARN → BLOCK (ensemble agreement)', () => {
     const r = combineVerdict([
-      { layer: 'testsavant_content', confidence: 0.7 },
-      { layer: 'transcript_classifier', confidence: 0.65 },
+      { layer: 'testsavant_content', confidence: 0.8 },
+      { layer: 'transcript_classifier', confidence: 0.78, meta: { verdict: 'block' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');
-    expect(r.confidence).toBe(0.65); // min of the two
+    expect(r.confidence).toBe(0.78); // min of the two
   });
 
   test('single layer >= BLOCK (no cross-confirm) → WARN, NOT block', () => {
@@ -67,7 +67,7 @@ describe('combineVerdict — ensemble rule', () => {
     // shouldn't kill sessions without a second opinion.
     const r = combineVerdict([
       { layer: 'testsavant_content', confidence: 0.95 },
-      { layer: 'transcript_classifier', confidence: 0.1 },
+      { layer: 'transcript_classifier', confidence: 0.1, meta: { verdict: 'safe' } },
     ]);
     expect(r.verdict).toBe('warn');
     expect(r.reason).toBe('single_layer_high');
@@ -75,8 +75,8 @@ describe('combineVerdict — ensemble rule', () => {
 
   test('single layer >= WARN → WARN (other layer low)', () => {
     const r = combineVerdict([
-      { layer: 'testsavant_content', confidence: 0.7 },
-      { layer: 'transcript_classifier', confidence: 0.2 },
+      { layer: 'testsavant_content', confidence: 0.8 },
+      { layer: 'transcript_classifier', confidence: 0.2, meta: { verdict: 'safe' } },
     ]);
     expect(r.verdict).toBe('warn');
     expect(r.reason).toBe('single_layer_medium');
@@ -101,7 +101,7 @@ describe('combineVerdict — ensemble rule', () => {
     const r = combineVerdict([
       { layer: 'testsavant_content', confidence: 0.3 },
       { layer: 'testsavant_content', confidence: 0.8 },
-      { layer: 'transcript_classifier', confidence: 0.75 },
+      { layer: 'transcript_classifier', confidence: 0.75, meta: { verdict: 'block' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');
@@ -110,20 +110,25 @@ describe('combineVerdict — ensemble rule', () => {
   // --- 3-way ensemble (DeBERTa opt-in) ---
 
   test('3-way: DeBERTa + testsavant at WARN → BLOCK (two ML classifiers agreeing)', () => {
+    // Two scalar-layer block-votes; transcript offers no vote.
     const r = combineVerdict([
-      { layer: 'testsavant_content', confidence: 0.7 },
-      { layer: 'deberta_content', confidence: 0.65 },
-      { layer: 'transcript_classifier', confidence: 0.1 },
+      { layer: 'testsavant_content', confidence: 0.8 },
+      { layer: 'deberta_content', confidence: 0.78 },
+      { layer: 'transcript_classifier', confidence: 0.1, meta: { verdict: 'safe' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');
   });
 
   test('3-way: only deberta fires alone → WARN (no cross-confirm)', () => {
+    // deberta at 0.95 is >= SOLO_CONTENT_BLOCK (0.92) → single_layer_high
+    // path. For user-input mode (no toolOutput opt), it degrades to WARN
+    // (SO-FP mitigation). Confidence bumped from 0.9 to 0.95 to stay above
+    // the new SOLO_CONTENT_BLOCK threshold.
     const r = combineVerdict([
       { layer: 'testsavant_content', confidence: 0.1 },
-      { layer: 'deberta_content', confidence: 0.9 },
-      { layer: 'transcript_classifier', confidence: 0.1 },
+      { layer: 'deberta_content', confidence: 0.95 },
+      { layer: 'transcript_classifier', confidence: 0.1, meta: { verdict: 'safe' } },
     ]);
     expect(r.verdict).toBe('warn');
     expect(r.reason).toBe('single_layer_high');
@@ -131,15 +136,15 @@ describe('combineVerdict — ensemble rule', () => {
 
   test('3-way: all three ML layers at WARN → BLOCK with min confidence', () => {
     const r = combineVerdict([
-      { layer: 'testsavant_content', confidence: 0.7 },
-      { layer: 'deberta_content', confidence: 0.65 },
-      { layer: 'transcript_classifier', confidence: 0.8 },
+      { layer: 'testsavant_content', confidence: 0.8 },
+      { layer: 'deberta_content', confidence: 0.76 },
+      { layer: 'transcript_classifier', confidence: 0.82, meta: { verdict: 'block' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');
-    // Confidence reports the MIN of the WARN+ signals (most conservative
-    // estimate of agreed-upon signal strength)
-    expect(r.confidence).toBe(0.65);
+    // Confidence reports the MIN of the contributing block-votes
+    // (most conservative estimate of agreed-upon signal strength).
+    expect(r.confidence).toBe(0.76);
   });
 
   test('DeBERTa disabled (confidence 0, meta.disabled) does not degrade verdict', () => {
@@ -148,9 +153,9 @@ describe('combineVerdict — ensemble rule', () => {
     // identically to a safe/absent signal — never let the zero drag
     // down what testsavant + transcript would have said.
     const r = combineVerdict([
-      { layer: 'testsavant_content', confidence: 0.7 },
+      { layer: 'testsavant_content', confidence: 0.8 },
       { layer: 'deberta_content', confidence: 0, meta: { disabled: true } },
-      { layer: 'transcript_classifier', confidence: 0.7 },
+      { layer: 'transcript_classifier', confidence: 0.8, meta: { verdict: 'block' } },
     ]);
     expect(r.verdict).toBe('block');
     expect(r.reason).toBe('ensemble_agreement');