Files
gstack/test/helpers/claude-pty-runner.unit.test.ts
Garry Tan 5d4fe7df07 v1.31.0.0 fix: delete AskUserQuestion fallback (root cause of forever war) + harness primitives (#1390)
* test: add multi-finding batching regression test (periodic tier)

Adds a periodic-tier E2E that catches the May 2026 transcript bug shape
the existing single-finding gate-tier floor test cannot detect: a model
that fires one AskUserQuestion and then batches the remaining findings
into a single "## Decisions to confirm" plan write + ExitPlanMode.

Why a separate test from skill-e2e-plan-eng-finding-floor: the gate-tier
floor (runPlanSkillFloorCheck) exits on the first AUQ render and returns
success, so a once-then-batch model would pass it trivially. This test
uses runPlanSkillCounting at periodic tier with N-AUQ tracking and
asserts >= 3 distinct review-phase AUQs on a 4-finding seeded plan.

- test/fixtures/forcing-finding-seeds.ts: FORCING_BATCHING_ENG fixture
  (4 distinct non-trivial findings spread across Architecture, Code
  Quality, Tests, Performance — mirrors the D1-D4 transcript shape)
- test/skill-e2e-plan-eng-multi-finding-batching.test.ts: new test
- test/helpers/touchfiles.ts: registered in BOTH E2E_TOUCHFILES and
  E2E_TIERS (touchfiles.test.ts asserts exact equality)

Test will fail on baseline today because today's model uses the preamble
fallback to batch findings; passes after the architectural fix lands in
a follow-up commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: expand plan-mode pass envelopes to accept BLOCKED path

Three existing plan-mode regression tests previously codified the
preamble fallback as a valid PASS path under --disallowedTools
AskUserQuestion: outcome=plan_ready was accepted only when the model
wrote a "## Decisions to confirm" section. The forever-war fix deletes
that fallback, so this assertion would fail post-deletion.

Expanded envelope accepts EITHER:
- 'plan_ready' WITH (## Decisions section [legacy] OR BLOCKED string
  visible in TTY [post-fix])
- 'exited' WITH BLOCKED string visible in TTY [post-fix]

The legacy ## Decisions branch stays in the envelope so these tests
keep passing on today's code (where the fallback still exists) and
on tomorrow's code (where the model reports BLOCKED instead). Once
the deletion has been on main long enough that the cache flushes,
the legacy branch can be removed in a follow-up.

Failure signals (regression we DO want to catch) unchanged:
auto_decided / silent_write / timeout / exited-without-BLOCKED /
plan_ready-without-(decisions OR BLOCKED).

- test/skill-e2e-plan-ceo-plan-mode.test.ts (test 2 only)
- test/skill-e2e-autoplan-auto-mode.test.ts
- test/skill-e2e-plan-design-plan-mode.test.ts

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: delete AskUserQuestion fallback (root cause of forever war)

The /plan-eng-review skill failed to fire AskUserQuestion on a real
plan review and surfaced 4 calibration decisions via prose instead.
Investigation traced this to a "fallback when neither variant is
callable" clause in the preamble that the model rationalizes around
as a general escape hatch from "fanning out round-trip AUQs," even
when an AUQ variant IS callable. Codex review confirmed the fallback
exists in 8 inline sites with 2 surviving escape hatches the original
narrowing missed (a "genuinely trivial" exception duplicated across
all 4 plan-* templates, and a "outside plan mode, output as prose
and stop" branch in the preamble itself).

Net deletion in skill text. Closes both branches of the deleted
fallback (plan-file write AND prose-and-stop) and the trivial-fix
exception with a single hard rule:

  If no AskUserQuestion variant appears in your tool list, this
  skill is BLOCKED. Stop, report `BLOCKED — AskUserQuestion
  unavailable`, and wait for the user.

Honest about being a model directive, not a runtime guard — none of
the PTY harness helpers enforce BLOCKED today. The architectural
improvement is that the model has fewer alternatives to obey it
against. Runtime enforcement is a follow-up TODO.

Sources changed:
- scripts/resolvers/preamble/generate-ask-user-format.ts: delete both
  fallback branches; replace with 1-line BLOCKED rule
- scripts/resolvers/preamble/generate-completion-status.ts: delete
  fallback in generatePlanModeInfo
- plan-eng-review/SKILL.md.tmpl: delete fallback at Step 0 + Sections
  1-4 (5 instances) + delete trivial-fix exception
- office-hours/SKILL.md.tmpl: delete fallback in approach-selection
- plan-ceo-review/SKILL.md.tmpl: delete trivial-fix exception
- plan-design-review/SKILL.md.tmpl: delete trivial-fix exception
- plan-devex-review/SKILL.md.tmpl: delete trivial-fix exception

Generated SKILL.md regen lands in a follow-up commit per the bisect
convention (template changes separate from regenerated output).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: regenerate SKILL.md after fallback deletion

Regenerates all 47 generated SKILL.md files (default + 7 host adapters)
after the template/resolver edits in the prior commit. Pure mechanical
output of `bun run gen:skill-docs`; no hand-edits.

Verifies fallback deletion landed across the entire skill surface:
- zero hits for "Decisions to confirm" in canonical SKILL.md / .tmpl
- zero hits for "no AskUserQuestion variant is callable"
- zero hits for "genuinely trivial"
- BLOCKED rule present in 42 generated SKILL.md (every Tier-2+ skill)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(harness): detect prose-rendered AskUserQuestion in plan mode

When --disallowedTools AskUserQuestion is set and no MCP variant is
callable, the model surfaces decisions as visible prose options
("A) ... B) ... C) ..." or "1. ... 2. ... 3. ...") rather than via the
native numbered-prompt UI. isNumberedOptionListVisible doesn't catch
these because the ❯ cursor sits on the empty input prompt rather than
on option 1, so runPlanSkillObservation and runPlanSkillFloorCheck
would time out at 5-10 minutes per test even though the model was
correctly waiting for user input.

This was exposed by the v1.28 fallback deletion: pre-deletion the
model used the preamble fallback to silently auto-resolve to
plan_ready in this scenario. Post-deletion the model correctly
surfaces the question and waits, but the harness couldn't tell.

isProseAUQVisible matches:
  - 2+ distinct lettered options at line starts (A/B/C/D form)
  - 3+ distinct numbered options at line starts WITHOUT a `❯ 1.`
    cursor (so it doesn't double-fire on native numbered prompts)

Wired into:
  - classifyVisible (used by runPlanSkillObservation) → returns
    outcome='asked' instead of timeout
  - runPlanSkillFloorCheck → counts as auq_observed (floor met)

8 new unit tests in claude-pty-runner.unit.test.ts cover the lettered
shape, numbered shape, threshold edges, native-cursor exclusion, and
mid-prose false-positive guard.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(harness): LLM judge for waiting-vs-working PTY state + snapshot logs

Regex detectors (isNumberedOptionListVisible, isProseAUQVisible) are
fast and free, but PTY rendering quirks fragment prose AUQ option
lists across logical lines that no regex can reliably reassemble.
When detection misses, polling loops time out at the full budget
even though the model is correctly waiting for user input.

Adds judgePtyState — a Haiku-graded trichotomy classifier:
  - waiting: agent surfaced a question/options, sitting at input prompt
  - working: spinner / tool calls / generation in progress
  - hung:    stopped without surfacing anything (rare crash signal)

Wired as a fallback into the polling loops of runPlanSkillObservation
and runPlanSkillFloorCheck: after 60s with no regex hit, snapshot the
TTY every 30s and call the judge. On 'waiting' verdict, return
outcome=asked / auq_observed early. On 'working' or 'hung', enrich the
eventual timeout summary with the verdict so failures are diagnosable.

Implementation:
  - Spawns `claude -p --model claude-haiku-4-5 --max-turns 1` synchronously
    with prompt piped via stdin (subscription auth, no API key env required)
  - In-process cache keyed by SHA-1 of normalized last-4KB so identical
    spinner-frame snapshots don't re-charge
  - Best-effort JSONL log to ~/.gstack/analytics/pty-judge.jsonl with
    timestamp, testName, state, reasoning, hash, judge wall time
  - 30s timeout per call; returns state='unknown' with diagnostic on any
    failure mode (timeout, malformed JSON, missing claude binary)

Snapshot logging: when GSTACK_PTY_LOG=1 is set, dump last 4KB of visible
TTY at every judge tick to ~/.gstack/analytics/pty-snapshots/<test>-
<elapsed>ms.txt — postmortem trail for debugging flakes.

Cost: ~$0.0005 per call; ~10 calls per 5-min test budget; ~$0.005 per
test added in worst case (only when regex detectors miss).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: accept prose-AUQ visible as third valid surface in plan-mode envelopes

The first re-run after wiring the LLM judge revealed that the model also
emits a third surface I hadn't anticipated: a properly-formatted question
with options ("Pick A, B, or C in your reply") rendered as prose AND
followed by ExitPlanMode (outcome=plan_ready). The migrated tests only
accepted (## Decisions section) OR (BLOCKED string) — neither matched
this case, so the test failed even though the user clearly saw the
question.

Three valid surfaces now:
  1. `## Decisions to confirm` section in plan file (legacy fallback path,
     still valid through migration window)
  2. `BLOCKED — AskUserQuestion` string in TTY (post-v1.28 BLOCKED rule)
  3. Numbered/lettered options visible in TTY as prose (post-v1.28 prose
     rendering — uses the existing isProseAUQVisible detector)

Also fixes assertReportAtBottomIfPlanWritten to be tolerant of:
  - Missing files (path detected from TTY but file not persisted) — was
    throwing ENOENT on plan_design_plan_mode and plan_ceo_plan_mode test 1
  - 'asked' outcome (smoke test exited at first AUQ before the model
    reached the report-writing step) — was throwing on the 1 fail in the
    plan-eng-plan-mode --disallowedTools test

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: drop GSTACK REVIEW REPORT contract from --disallowedTools migrations

The plan-ceo / plan-design --disallowedTools migrated tests called
assertReportAtBottomIfPlanWritten as the final assertion, but that
contract is for full multi-section review completions. Under
--disallowedTools AskUserQuestion the model can't run the full
review (no AUQ tools to ask findings questions through), so it exits
at Step 0 with either prose-AUQ rendering or the legacy decisions
fallback. A plan file written in that mode WON'T have a GSTACK
REVIEW REPORT section — the workflow never reached the report-writing
step.

The contract is still enforced by the periodic finding-count tests
(skill-e2e-plan-{ceo,eng,design,devex}-finding-count.test.ts), which
DO run the full review end-to-end and assert report-at-bottom there.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(harness): high-water-mark prose-AUQ tracking across polling iterations

The autoplan E2E surfaces a brief prose-AUQ window (model emits options,
waits ~30s for non-existent test responder, then resumes thinking) that
the existing polling loop misses: by judge-tick time the buffer has
moved into spinner state, so the LLM judge correctly reports 'working'
and the loop times out at 5min.

Adds two flags tracked across polling iterations:
  - proseAUQEverObserved: set true the first tick isProseAUQVisible
    returns true on the recent buffer
  - waitingEverObserved: set true on the first LLM judge 'waiting' verdict

At timeout, if either flag is set, return outcome='asked' with a
summary explaining the historical signal. The model DID surface the
question — we just missed the live-state window.

Snapshot logged with tag='prose-auq-surfaced' when GSTACK_PTY_LOG=1
for postmortem trace.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: migrate plan-eng-plan-mode test 2 envelope to match other plan-mode tests

The plan-ceo, plan-design, and autoplan plan-mode tests under
--disallowedTools all moved to the same surface-visibility envelope
(decisions section OR BLOCKED string OR prose-AUQ visible) and dropped
the GSTACK REVIEW REPORT contract because the workflow can't complete
without AUQ tools. plan-eng-plan-mode test 2 had been left on the old
envelope and was the last failing test.

This commit migrates it to match. Also lifts 'exited' out of the failure
list and into a guarded path (acceptable when surface-visible).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(harness): isProseAUQVisible — gate numbered path on tail, not full buffer

The numbered-options branch of isProseAUQVisible deferred to
isNumberedOptionListVisible whenever a `❯ 1.` cursor was visible in the
full buffer. But the boot trust dialog (`❯ 1. Yes, trust`) lives in
scrollback for the entire run, so this gate suppressed prose-numbered
detection for any session that had the trust prompt at startup —
i.e., every E2E run after the first user-trust acceptance.

Fix: check only the last 4KB tail. Native-UI deferral applies when
the cursor list is CURRENTLY rendered, not historically present in
scrollback.

Adds a regression test that puts the trust dialog in early scrollback
+ 5KB filler + a current prose-AUQ render, asserts true.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(harness): isProseAUQVisible — lower numbered threshold to 2 (matches lettered)

The 4KB tail window often contains only options 2-4 of a 4-option
numbered prose AUQ because the model emits the question header + option 1
several KB earlier in the buffer. The threshold of 3 distinct numbered
markers caused the detector to miss real prose AUQs whenever option 1
had scrolled out.

Threshold 2 matches the lettered branch and is still tightly gated by:
- Line-start anchoring (no false positives on inline `1.` references)
- No-cursor gate (defers to native UI when ❯ 1. is currently rendered)
- The 4KB tail window itself (prose-AUQ rendering happens at the end of
  the model's response, so options are clustered in the tail)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: expose high-water-mark flags through PlanSkillObservation

The 2KB obs.evidence window often misses the prose-AUQ moment because
ExitPlanMode UI ("Ready to execute" + numbered approve/reject prompt)
pushes the model's earlier option list out of the tail by the time
outcome=plan_ready fires. Tests checking "did the user see a question"
need to consult historical state, not just the truncated final tail.

Adds two optional fields to PlanSkillObservation:
  - proseAUQEverObserved: true if isProseAUQVisible was true at any tick
  - waitingEverObserved: true if the LLM judge ever returned 'waiting'

The 4 plan-mode --disallowedTools tests now check these flags as part
of the surfaceVisible computation:
    isProseAUQVisible(obs.evidence) || obs.proseAUQEverObserved === true
    blockedVisible || proseAUQVisible || obs.waitingEverObserved === true

This catches the autoplan / plan-ceo / plan-eng case where the model
surfaces options briefly, fails to get a response, then keeps thinking
— eventually emitting ExitPlanMode and pushing options out of evidence.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(plan-ceo): bump --disallowedTools test timeout to 10 min

Last 5 runs showed the model under --disallowedTools spending the full
5-min budget in 'high effort thinking' before surfacing options. The LLM
judge correctly reports state=working at every 30s tick, so the
high-water-mark fallback never fires.

10-min budget gives the model 20 judge windows to eventually surface
the question. Outer bun timeout bumped accordingly to 660s (inner +60s).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(plan-ceo): pre-prime --disallowedTools test with concrete plan content

Root cause of the persistent timeout: under --disallowedTools, the model
can't fire the AUQ tool to ask "what should I review?" — it has to
prose-render that question. Prose-rendering a 4-option choice requires
the model to first enumerate every option, which spent the full 5min
budget in 'high effort thinking' (8 consecutive 'state=working' verdicts
from the LLM judge).

Fix: pass initialPlanContent (already supported by runPlanSkillObservation)
with a CEO-review-shaped seed plan (vague success metric, missing
premise, scope creep smell). The model now has concrete material to
critique on entry, bypasses the scope-deliberation loop, and moves
directly to surfacing Step 0 / Section 1 findings — the actual
behavior we want to regression-test.

Reverted timeout from 600_000 back to 300_000 since the 5-min budget
is plenty when the model has a real plan to work with.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: delete --disallowedTools AskUserQuestion-blocked test variants

These tests simulated a fictional environment that doesn't exist in
production. Real Conductor sessions launch claude with
`--disallowedTools AskUserQuestion` AND register
`mcp__conductor__AskUserQuestion` — the model has the MCP variant. But
the tests passed `--disallowedTools` without standing up any MCP server,
so they tested "model behavior with NO AUQ available," which no real
user state produces.

Combined with bare `/plan-ceo-review` invocation (no follow-up content),
this forced the model into a 5+ minute deliberation loop trying to
prose-render a question with options it had to first invent. The result
was persistent flakes that consumed nine paid E2E runs trying to fix
"the model takes too long" — but the actual problem was the test
configuration, not the model.

Removals:
- test/skill-e2e-autoplan-auto-mode.test.ts (deleted; the entire file
  was a single AUQ-blocked test)
- test/skill-e2e-plan-ceo-plan-mode.test.ts test 2 (the migrated
  --disallowedTools test); test 1 (baseline plan-mode smoke) stays
- test/skill-e2e-plan-design-plan-mode.test.ts test 2 (same shape);
  test 1 stays
- test/skill-e2e-plan-eng-plan-mode.test.ts test 2 (same shape); test 1
  (baseline) and test 3 (STOP-gate with seeded plan, different
  contract) stay
- test/helpers/touchfiles.ts: autoplan-auto-mode entry removed
- test/touchfiles.test.ts: assertion count + commentary updated

Coverage retained: test 1 of each plan-mode file already verifies the
model fires AUQ; the periodic finding-count tests verify per-finding
AUQ cadence end-to-end. The harness improvements landed during this
debugging cycle (isProseAUQVisible regex, LLM judge, snapshot logging,
high-water-mark tracking, ENOENT-tolerant assertReportAtBottomIfPlanWritten)
all stay — they're useful for the remaining plan-mode tests that can
also encounter prose rendering and slow-thinking phases.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v1.31.0.0)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 17:01:13 -07:00

922 lines
32 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/**
* Deterministic unit tests for claude-pty-runner.ts behavior changes.
*
* Free-tier (no EVALS=1 needed). Runs in <1s on every `bun test`. Catches
* harness plumbing bugs before stochastic PTY runs surface them.
*
* Two surface areas tested:
*
* 1. Permission-dialog short-circuit in 'asked' classification: a TTY frame
* that matches BOTH isPermissionDialogVisible AND isNumberedOptionListVisible
* must NOT be classified as a skill question — permission dialogs render
* as numbered lists too, but they're not what we're guarding.
*
* 2. Env passthrough surface: runPlanSkillObservation accepts an `env`
* option and threads it to launchClaudePty. We can't fully exercise the
* spawn pipeline without paying for a PTY session, but we CAN verify the
* option exists in the type signature and that calling without env still
* works (no regression).
*
* The PTY test (skill-e2e-plan-ceo-plan-mode.test.ts) is the integration
* check; this file is the cheap deterministic guard for the harness primitives
* those tests stand on.
*/
import { describe, test, expect } from 'bun:test';
import {
isPermissionDialogVisible,
isNumberedOptionListVisible,
isProseAUQVisible,
isPlanReadyVisible,
parseNumberedOptions,
classifyVisible,
TAIL_SCAN_BYTES,
optionsSignature,
parseQuestionPrompt,
auqFingerprint,
COMPLETION_SUMMARY_RE,
assertReviewReportAtBottom,
ceoStep0Boundary,
engStep0Boundary,
designStep0Boundary,
devexStep0Boundary,
type ClaudePtyOptions,
type AskUserQuestionFingerprint,
} from './claude-pty-runner';
describe('isPermissionDialogVisible', () => {
test('matches "Bash command requires permission" prompts', () => {
const sample = `
Some preamble output
Bash command \`gstack-config get telemetry\` requires permission to run.
1. Yes
2. Yes, and always allow
3. No, abort
`;
expect(isPermissionDialogVisible(sample)).toBe(true);
});
test('matches "allow all edits" file-edit prompts', () => {
// Isolated to the "allow all edits" clause only — no overlapping
// "Do you want to proceed?" co-trigger, so this asserts the clause works.
const sample = `
Edit to ~/.gstack/config.yaml
1. Yes
2. Yes, allow all edits during this session
3. No
`;
expect(isPermissionDialogVisible(sample)).toBe(true);
});
test('matches the "Do you want to proceed?" file-edit confirmation by itself', () => {
// Separate fixture so weakening this clause is detected by a dedicated test.
const sample = `
Edit to ~/.gstack/config.yaml
Do you want to proceed?
1. Yes
2. No
`;
expect(isPermissionDialogVisible(sample)).toBe(true);
});
test('matches workspace-trust "always allow access to" prompt', () => {
const sample = `
Do you trust the files in this folder?
1. Yes, proceed
2. Yes, and always allow access to /Users/me/repo
3. No, exit
`;
expect(isPermissionDialogVisible(sample)).toBe(true);
});
test('does NOT match a skill AskUserQuestion list', () => {
const sample = `
D1 — Premise challenge: do users actually want this?
1. Yes, validated
2. No, premise is wrong
3. Need more info
`;
expect(isPermissionDialogVisible(sample)).toBe(false);
});
test('does NOT match a plan-ready confirmation', () => {
const sample = `
Ready to execute the plan?
1. Yes
2. No, keep planning
`;
expect(isPermissionDialogVisible(sample)).toBe(false);
});
test('does NOT match a skill question that contains the bare phrase "Do you want to proceed?"', () => {
// Co-trigger requirement: "Do you want to proceed?" alone is not enough.
// It must appear with "Edit to <path>" or "Write to <path>" to count as
// a permission dialog. This guards against a skill question like
// "Do you want to proceed with HOLD SCOPE?" being mis-classified.
const sample = `
Choose your scope mode for this review.
Do you want to proceed?
1. HOLD SCOPE
2. SCOPE EXPANSION
3. SELECTIVE EXPANSION
`;
expect(isPermissionDialogVisible(sample)).toBe(false);
});
test('does NOT mis-match when adversarial prose includes "Edit to <path>" alongside the bare proceed phrase', () => {
// Adversarial fixture: a skill question whose body legitimately mentions
// "Edit to <path>" in prose AND ends with "Do you want to proceed?". The
// current co-trigger regex would mis-classify this as a permission
// dialog. We DO want this test to fail until the regex is tightened
// further (e.g., proximity constraint, or anchoring "Edit to" to a
// line-start). For now this is documented as a known limitation: a
// skill question that talks about "Edit to" in prose IS still treated
// as a permission dialog. The test asserts the current behavior so a
// future fix can flip it intentionally.
const sample = `
Plan: I will Edit to ./plan.md to capture the decision.
Do you want to proceed?
1. HOLD SCOPE
2. SCOPE EXPANSION
`;
// KNOWN LIMITATION: the co-trigger fires here. Documented as a
// post-merge follow-up. Flip this assertion once the regex tightens.
expect(isPermissionDialogVisible(sample)).toBe(true);
});
});
describe('isNumberedOptionListVisible', () => {
test('matches a basic 1. + 2. cursor list', () => {
const sample = `
1. Option one
2. Option two
3. Option three
`;
expect(isNumberedOptionListVisible(sample)).toBe(true);
});
test('returns false on a single-option prompt', () => {
const sample = `
1. Only option
`;
expect(isNumberedOptionListVisible(sample)).toBe(false);
});
test('returns false when no cursor renders', () => {
const sample = `
Just some prose with 1. a numbered point and 2. another.
`;
expect(isNumberedOptionListVisible(sample)).toBe(false);
});
test('overlaps permission dialogs (this is why D5 short-circuits)', () => {
// The whole point of D5: this string matches BOTH classifiers, so the
// runner must consult isPermissionDialogVisible to disambiguate.
const sample = `
Bash command \`do-thing\` requires permission to run.
1. Yes
2. No
`;
expect(isNumberedOptionListVisible(sample)).toBe(true);
expect(isPermissionDialogVisible(sample)).toBe(true);
});
});
describe('isProseAUQVisible', () => {
test('matches 4 lettered options A) B) C) D) at line starts (plan-eng prose AUQ shape)', () => {
const sample = `
What would you like me to review? Options:
A) Point me at an existing design doc or plan file (path).
B) Describe new work you're planning — I'll explore the codebase.
C) You meant /review for the diff already on this branch.
D) Something else (tell me).
Recommendation: A if you have a doc in mind, otherwise B.
`;
expect(isProseAUQVisible(sample)).toBe(true);
});
test('matches 2 lettered options (minimum threshold)', () => {
const sample = `
A) First option
B) Second option
`;
expect(isProseAUQVisible(sample)).toBe(true);
});
test('matches 3 numbered options 1. 2. 3. without 1. cursor (autoplan prose AUQ shape)', () => {
const sample = `
What's the task? A few options:
1. You have a plan idea in mind — describe it.
2. You want to review an existing plan elsewhere.
3. You meant a different command — /plan-ceo-review etc.
`;
expect(isProseAUQVisible(sample)).toBe(true);
});
test('returns false when 1. cursor is present in the recent tail (native UI handled by isNumberedOptionListVisible)', () => {
const sample = `
1. First option
2. Second option
3. Third option
`;
expect(isProseAUQVisible(sample)).toBe(false);
});
test('does NOT suppress numbered-prose detection when 1. is only in early scrollback (trust dialog)', () => {
// Boot trust dialog rendered 1. Yes at startup, then a long body of
// model output, then prose-rendered numbered options now. The historic
// 1. is in the full buffer but NOT in the recent tail. Should detect
// the prose AUQ.
const trustHeader = ' 1. Yes, trust\n 2. No\n';
const filler = 'x'.repeat(5000); // pushes trust dialog out of last 4KB tail
const proseAUQ = `\n 1. Review the docs\n 2. Investigate the code\n 3. Defer to next session\n \n`;
const sample = trustHeader + filler + proseAUQ;
expect(isProseAUQVisible(sample)).toBe(true);
});
test('returns false on single lettered option', () => {
const sample = `
A) Only one option mentioned in passing.
`;
expect(isProseAUQVisible(sample)).toBe(false);
});
test('matches 2 numbered options (threshold matches lettered branch — tails miss option 1)', () => {
const sample = `
1. First note.
2. Second note.
`;
expect(isProseAUQVisible(sample)).toBe(true);
});
test('returns false on a single numbered option', () => {
const sample = `
1. Only one option mentioned.
`;
expect(isProseAUQVisible(sample)).toBe(false);
});
test('does not match mid-prose lettered text like "(see option B) above"', () => {
const sample = `
This refers to (see option B) above and also to point A) earlier.
`;
// The B) and A) markers are mid-line, not at line starts, so they don't count.
expect(isProseAUQVisible(sample)).toBe(false);
});
test('matches with leading whitespace and prefix on options', () => {
const sample = `
A) Option with whitespace prefix
B) Option with cursor prefix
C) Another option
`;
expect(isProseAUQVisible(sample)).toBe(true);
});
test('returns false on plain text with no option markers', () => {
expect(isProseAUQVisible('Just some plain text output from the model.')).toBe(false);
expect(isProseAUQVisible('')).toBe(false);
});
});
describe('classifyVisible (runtime path through the runner classifier)', () => {
// These tests call the actual classifier so a future contributor who
// reorders branches (e.g. moves the permission short-circuit before
// isPlanReadyVisible) is caught deterministically.
test('skill question → returns asked', () => {
const visible = `
D1 — Choose your scope mode
1. HOLD SCOPE
2. SCOPE EXPANSION
3. SELECTIVE EXPANSION
4. SCOPE REDUCTION
`;
const result = classifyVisible(visible);
expect(result?.outcome).toBe('asked');
});
test('permission dialog (Bash) → returns null (skip, keep polling)', () => {
const visible = `
Bash command \`gstack-update-check\` requires permission to run.
1. Yes
2. No
`;
expect(isNumberedOptionListVisible(visible)).toBe(true); // pre-filter
expect(classifyVisible(visible)).toBeNull(); // post-filter
});
test('plan-ready confirmation → returns plan_ready (wins over asked)', () => {
const visible = `
Ready to execute the plan?
1. Yes, proceed
2. No, keep planning
`;
const result = classifyVisible(visible);
expect(result?.outcome).toBe('plan_ready');
});
test('silent write to unsanctioned path → returns silent_write', () => {
const visible = `
⏺ Write(src/app/dangerous-write.ts)
⎿ Wrote 42 lines
`;
const result = classifyVisible(visible);
expect(result?.outcome).toBe('silent_write');
expect(result?.summary).toContain('src/app/dangerous-write.ts');
});
test('write to sanctioned path (.claude/plans) → returns null (allowed)', () => {
const visible = `
⏺ Write(/Users/me/.claude/plans/some-plan.md)
⎿ Wrote 42 lines
`;
expect(classifyVisible(visible)).toBeNull();
});
test('write while a permission dialog is on screen → returns null (gated, not silent, not asked)', () => {
const visible = `
⏺ Write(src/app/edit-with-permission.ts)
Edit to src/app/edit-with-permission.ts
Do you want to proceed?
1. Yes
2. No
`;
// The numbered prompt is a permission dialog (Edit to + Do you want to proceed?);
// silent_write is suppressed because a numbered prompt is visible, AND
// 'asked' is suppressed because the prompt is a permission dialog.
expect(classifyVisible(visible)).toBeNull();
});
test('write while a real skill question is on screen → returns asked (write is captured but not silent)', () => {
const visible = `
⏺ Write(src/app/foo.ts)
D1 — Choose your scope mode
1. HOLD SCOPE
2. SCOPE EXPANSION
`;
// The numbered prompt is a skill question, not a permission dialog;
// silent_write is suppressed (numbered prompt is visible) and the
// outcome is 'asked' — Step 0 fired.
const result = classifyVisible(visible);
expect(result?.outcome).toBe('asked');
});
test('idle / no signals → returns null', () => {
const visible = `
Some prose without any classifier signals.
`;
expect(classifyVisible(visible)).toBeNull();
});
test('TAIL_SCAN_BYTES is exported as 1500', () => {
// Shared between runner and routing test; a regression that desyncs the
// recent-tail window would surface here.
expect(TAIL_SCAN_BYTES).toBe(1500);
});
// D4-B: strictPlanWrites detector. Catches the transcript bug where the
// model writes findings to the plan file before any AskUserQuestion fires.
test('strictPlanWrites: plan write before any AUQ → wrote_findings_before_asking', () => {
const visible = `
⏺ Edit(/Users/me/.claude/plans/some-plan.md)
⎿ Updated 12 lines
`;
const result = classifyVisible(visible, { strictPlanWrites: true });
expect(result?.outcome).toBe('wrote_findings_before_asking');
expect(result?.summary).toContain('.claude/plans/some-plan.md');
});
test('strictPlanWrites: plan write AFTER an AUQ render → not flagged', () => {
// AUQ renders first, then the model writes the plan post-answer. This is
// the legitimate end-of-workflow flow and must NOT trigger the detector.
const visible = `
D1 — Some scope question
1. Option A
2. Option B
⏺ Edit(/Users/me/.claude/plans/some-plan.md)
⎿ Updated 12 lines
`;
const result = classifyVisible(visible, { strictPlanWrites: true });
// Outcome is 'asked' (the numbered list rendered); the post-AUQ plan
// write is ignored by the detector.
expect(result?.outcome).toBe('asked');
});
test('strictPlanWrites: AUQ first then plan write — write_pos > auq_pos → not flagged', () => {
// Same scenario, more explicit ordering: the regex finds the write at a
// position AFTER the numbered list. Detector lets it through.
const visible = [
'D1 — Choose your approach',
'',
' 1. Approach A',
' 2. Approach B',
'',
'⏺ Write(/Users/me/.claude/plans/draft.md)',
'⎿ Wrote 42 lines',
].join('\n');
const result = classifyVisible(visible, { strictPlanWrites: true });
expect(result?.outcome).toBe('asked');
});
test('strictPlanWrites: only a permission dialog visible → plan write still flagged', () => {
// A permission dialog 1./2. is NOT an AUQ; pre-AUQ plan writes still
// hit the detector even when a permission prompt is on screen.
const visible = `
⏺ Edit(/Users/me/.claude/plans/some-plan.md)
Edit to /Users/me/.claude/plans/some-plan.md
Do you want to proceed?
1. Yes
2. No
`;
const result = classifyVisible(visible, { strictPlanWrites: true });
expect(result?.outcome).toBe('wrote_findings_before_asking');
});
test('strictPlanWrites OFF: plan write before AUQ → returns null (legacy behavior preserved)', () => {
const visible = `
⏺ Edit(/Users/me/.claude/plans/some-plan.md)
⎿ Updated 12 lines
`;
// Without strictPlanWrites, the sanctioned-path list lets this through.
expect(classifyVisible(visible)).toBeNull();
});
});
describe('parseNumberedOptions', () => {
test('extracts options from a clean cursor list', () => {
const visible = `
1. HOLD SCOPE
2. SCOPE EXPANSION
`;
const opts = parseNumberedOptions(visible);
expect(opts).toHaveLength(2);
expect(opts[0]).toEqual({ index: 1, label: 'HOLD SCOPE' });
expect(opts[1]).toEqual({ index: 2, label: 'SCOPE EXPANSION' });
});
test('returns empty array on prose-with-numbers (no cursor)', () => {
expect(parseNumberedOptions('text 1. one 2. two')).toEqual([]);
});
test('extracts options when the cursor is INLINE with prompt header (box-layout)', () => {
// Real /plan-ceo-review rendering: the TTY's cursor-positioning escapes
// collapse divider + header + prompt + cursor onto one logical line.
// Subsequent options (2..7) still start their own lines.
const visible = [
'────────────────────────────────────────',
'☐ Review scope What scope do you want me to CEO-review? 1. The branch\'s diff vs main',
' Review the full branch: ~10K LOC.',
'2. A specific plan file or design doc',
' You point me at a file (path) and I review that.',
'3. An idea you\'ll describe inline',
'4. Cancel — wrong skill',
'5. Type something.',
'────────────────────────────────────────',
'6. Chat about this',
'7. Skip interview and plan immediately',
].join('\n');
const opts = parseNumberedOptions(visible);
expect(opts).toHaveLength(7);
expect(opts[0]).toEqual({ index: 1, label: "The branch's diff vs main" });
expect(opts[1]?.index).toBe(2);
expect(opts[6]?.index).toBe(7);
expect(opts[6]?.label).toBe('Skip interview and plan immediately');
});
test('inline-cursor and start-of-line cursor both produce 7 options for the box-layout case', () => {
// The inline path captures option 1 from the cursor line itself; the
// subsequent-lines path captures 2..7 with the existing optionRe.
const inlineLayout = [
'header text 1. first option',
'2. second',
'3. third',
].join('\n');
expect(parseNumberedOptions(inlineLayout)).toEqual([
{ index: 1, label: 'first option' },
{ index: 2, label: 'second' },
{ index: 3, label: 'third' },
]);
const cleanLayout = [
' 1. first option',
' 2. second',
' 3. third',
].join('\n');
expect(parseNumberedOptions(cleanLayout)).toEqual([
{ index: 1, label: 'first option' },
{ index: 2, label: 'second' },
{ index: 3, label: 'third' },
]);
});
});
describe('runPlanSkillObservation env passthrough surface', () => {
test('ClaudePtyOptions exposes env: Record<string, string>', () => {
// Type-level guard: this file would fail to compile if the env field
// were removed or its shape regressed. The actual env merge happens in
// launchClaudePty's spawn call (`env: { ...process.env, ...opts.env }`),
// so a regression where `env: opts.env` gets dropped from the
// runPlanSkillObservation -> launchClaudePty handoff is only caught by
// the live PTY test, not here.
const opts: ClaudePtyOptions = {
env: { QUESTION_TUNING: 'false', EXPLAIN_LEVEL: 'default' },
};
expect(opts.env).toEqual({ QUESTION_TUNING: 'false', EXPLAIN_LEVEL: 'default' });
});
});
// ────────────────────────────────────────────────────────────────────────────
// Per-finding count primitives — Section 3 unit tests #1#5, #7, #12.
// ────────────────────────────────────────────────────────────────────────────
describe('optionsSignature', () => {
test('returns a "|"-joined `index:label` string for a clean list', () => {
const sig = optionsSignature([
{ index: 1, label: 'HOLD SCOPE' },
{ index: 2, label: 'SCOPE EXPANSION' },
]);
expect(sig).toBe('1:HOLD SCOPE|2:SCOPE EXPANSION');
});
test('order-independent: shuffled inputs produce the same signature', () => {
// parseNumberedOptions already returns sorted, but defensive sort means
// a future caller that hands us shuffled input still produces a stable
// dedupe signature.
const a = optionsSignature([
{ index: 2, label: 'B' },
{ index: 1, label: 'A' },
{ index: 3, label: 'C' },
]);
const b = optionsSignature([
{ index: 1, label: 'A' },
{ index: 2, label: 'B' },
{ index: 3, label: 'C' },
]);
expect(a).toBe(b);
});
test('empty list returns empty string', () => {
expect(optionsSignature([])).toBe('');
});
test('single-item list returns just that entry', () => {
expect(optionsSignature([{ index: 1, label: 'Only' }])).toBe('1:Only');
});
});
describe('parseQuestionPrompt', () => {
test('captures 1-line prompt above the cursor', () => {
const visible = `
D1 — Pick a mode
1. HOLD SCOPE
2. SCOPE EXPANSION
`;
const prompt = parseQuestionPrompt(visible);
expect(prompt).toBe('D1 — Pick a mode');
});
test('captures multi-line prompt above the cursor', () => {
const visible = `
D2 — Approach selection
Which architecture should we follow?
1. Bypass existing helper
2. Reuse existing helper
`;
const prompt = parseQuestionPrompt(visible);
// Multi-line prompts get joined with single spaces.
expect(prompt).toContain('D2 — Approach selection');
expect(prompt).toContain('Which architecture should we follow?');
});
test('returns "" when no cursor is rendered', () => {
expect(parseQuestionPrompt('Just some prose.\nNo cursor.')).toBe('');
});
test('truncates to 240 chars', () => {
const longPrompt = 'A'.repeat(500);
const visible = `${longPrompt}\n\n 1. yes\n 2. no`;
expect(parseQuestionPrompt(visible).length).toBeLessThanOrEqual(240);
});
test('does not pull text from a previous numbered list above', () => {
const visible = `
1. previous answered question
2. previous option two
D2 — A new question text
1. fresh option A
2. fresh option B
`;
const prompt = parseQuestionPrompt(visible);
// Stops at the previous numbered-list line; should NOT contain "previous answered question".
expect(prompt).toContain('D2 — A new question text');
expect(prompt).not.toContain('previous answered question');
});
test('normalizes whitespace (collapses runs of spaces and tabs)', () => {
const visible = `D1 — Spaced out
1. yes
2. no`;
expect(parseQuestionPrompt(visible)).toBe('D1 — Spaced out');
});
test('inline-cursor box-layout: extracts prompt text BEFORE 1. on the cursor line', () => {
// Real /plan-ceo-review rendering: divider + ☐ header + prompt text +
// cursor are all on one logical line because TTY cursor-positioning
// escapes collapse the box layout under stripAnsi.
const visible = [
'──────────────────',
'☐ Review scope What scope do you want me to CEO-review? 1. The branch\'s diff vs main',
'2. A specific plan file',
'3. An idea inline',
].join('\n');
const prompt = parseQuestionPrompt(visible);
// Should extract "Review scope" and the prompt text, dropping the ☐ box-drawing sigil.
expect(prompt).toContain('Review scope');
expect(prompt).toContain('What scope do you want me to CEO-review?');
expect(prompt).not.toContain('');
expect(prompt).not.toMatch(/^☐/);
});
});
describe('auqFingerprint', () => {
test('returns the same fingerprint for identical inputs', () => {
const opts = [
{ index: 1, label: 'A' },
{ index: 2, label: 'B' },
];
expect(auqFingerprint('hello', opts)).toBe(auqFingerprint('hello', opts));
});
test('different prompts with shared option labels produce DIFFERENT fingerprints', () => {
// The collision regression Codex F1 caught: option-label-only fingerprints
// collapsed multiple distinct findings into one when they shared menu shape.
const sharedOpts = [
{ index: 1, label: 'Add to plan' },
{ index: 2, label: 'Defer' },
{ index: 3, label: 'Build now' },
];
const fpFinding1 = auqFingerprint('D5 — Architecture: bypass helper?', sharedOpts);
const fpFinding2 = auqFingerprint('D6 — Tests: zero coverage?', sharedOpts);
expect(fpFinding1).not.toBe(fpFinding2);
});
test('same prompt with different options produces DIFFERENT fingerprints', () => {
const prompt = 'D1 — Pick a mode';
const fpA = auqFingerprint(prompt, [
{ index: 1, label: 'HOLD SCOPE' },
{ index: 2, label: 'SCOPE EXPANSION' },
]);
const fpB = auqFingerprint(prompt, [
{ index: 1, label: 'HOLD SCOPE' },
{ index: 2, label: 'SCOPE REDUCTION' },
]);
expect(fpA).not.toBe(fpB);
});
test('whitespace-only differences in prompt do NOT change the fingerprint', () => {
// Same content, different rendering whitespace (TTY redraw artifact)
// must produce the same fingerprint so dedupe survives reflow.
const opts = [{ index: 1, label: 'A' }, { index: 2, label: 'B' }];
const fpA = auqFingerprint('Pick a mode', opts);
const fpB = auqFingerprint('Pick a mode', opts);
expect(fpA).toBe(fpB);
});
test('empty prompt + same options collide (caller must guard against this)', () => {
// Documents the contract: empty-prompt fingerprints WILL collide if the
// caller fingerprints them. runPlanSkillCounting must skip empty-prompt
// AUQs and re-poll instead.
const opts = [{ index: 1, label: 'A' }];
expect(auqFingerprint('', opts)).toBe(auqFingerprint('', opts));
});
});
describe('COMPLETION_SUMMARY_RE', () => {
test('matches GSTACK REVIEW REPORT heading', () => {
expect(COMPLETION_SUMMARY_RE.test('## GSTACK REVIEW REPORT')).toBe(true);
});
test('matches Completion Summary heading (ceo + eng)', () => {
expect(COMPLETION_SUMMARY_RE.test('## Completion Summary')).toBe(true);
expect(COMPLETION_SUMMARY_RE.test('## Completion summary')).toBe(true);
});
test('matches Status: clean (CEO review-log shape)', () => {
expect(COMPLETION_SUMMARY_RE.test('Status: clean')).toBe(true);
expect(COMPLETION_SUMMARY_RE.test('Status: issues_open')).toBe(true);
});
test('matches VERDICT: line', () => {
expect(COMPLETION_SUMMARY_RE.test('VERDICT: CLEARED — Eng Review passed')).toBe(true);
});
test('does NOT match prose mentions of "verdict" mid-line', () => {
// VERDICT must be at the start of a line to count.
expect(COMPLETION_SUMMARY_RE.test('the final verdict: undecided')).toBe(false);
});
});
describe('assertReviewReportAtBottom', () => {
test('passes when REVIEW REPORT is the only/last ## heading', () => {
const content = `# Plan
## Context
stuff
## Approach
more stuff
## GSTACK REVIEW REPORT
| col | col |
`;
const r = assertReviewReportAtBottom(content);
expect(r.ok).toBe(true);
});
test('fails when REVIEW REPORT is missing', () => {
const content = `# Plan
## Context
stuff
`;
const r = assertReviewReportAtBottom(content);
expect(r.ok).toBe(false);
expect(r.reason).toMatch(/no GSTACK REVIEW REPORT/);
});
test('fails when REVIEW REPORT exists but a ## heading follows it', () => {
const content = `# Plan
## GSTACK REVIEW REPORT
| col | col |
## Late Section
oops
`;
const r = assertReviewReportAtBottom(content);
expect(r.ok).toBe(false);
expect(r.reason).toMatch(/trailing ## heading/);
expect(r.trailingHeadings).toEqual(['## Late Section']);
});
test('passes when only ### subheadings follow REVIEW REPORT (deeper nesting allowed)', () => {
const content = `## GSTACK REVIEW REPORT
### Cross-model tension
- F1: resolved
- F2: resolved
`;
const r = assertReviewReportAtBottom(content);
expect(r.ok).toBe(true);
});
test('fails with multiple trailing ## headings reported', () => {
const content = `## GSTACK REVIEW REPORT
## First trailing
## Second trailing
`;
const r = assertReviewReportAtBottom(content);
expect(r.ok).toBe(false);
expect(r.trailingHeadings).toHaveLength(2);
});
});
describe('Step0BoundaryPredicate per-skill', () => {
// Helper to build a synthetic fingerprint for predicate tests.
function fp(promptSnippet: string, optionLabels: string[]): AskUserQuestionFingerprint {
const options = optionLabels.map((label, i) => ({ index: i + 1, label }));
return {
signature: auqFingerprint(promptSnippet, options),
promptSnippet,
options,
observedAtMs: 0,
preReview: true,
};
}
describe('ceoStep0Boundary', () => {
test('FIRES on Step 0F mode-pick AUQ (HOLD SCOPE in options)', () => {
const f = fp('Pick a mode', ['HOLD SCOPE', 'SCOPE EXPANSION', 'SELECTIVE EXPANSION', 'SCOPE REDUCTION']);
expect(ceoStep0Boundary(f)).toBe(true);
});
test('FIRES on scope-selection AUQ with "Skip interview" option (skip-interview path)', () => {
// After calibration run 1: plan-ceo's first AUQ is scope-selection,
// and we route via "Skip interview and plan immediately" to bypass
// Step 0 entirely. Boundary must fire on this AUQ so subsequent
// AUQs go to reviewCount.
const f = fp(
'What scope do you want me to CEO-review?',
[
"The branch's diff vs main",
'A specific plan file',
"An idea you'll describe inline",
'Cancel — wrong skill',
'Type something.',
'Chat about this',
'Skip interview and plan immediately',
],
);
expect(ceoStep0Boundary(f)).toBe(true);
});
test('does NOT fire on premise challenge AUQs', () => {
const f = fp('D1 — Premise check: is this the right problem?', ['Yes', 'No', 'Other']);
expect(ceoStep0Boundary(f)).toBe(false);
});
test('does NOT fire on review-section AUQs', () => {
const f = fp('Architecture: bypass helper?', ['Reuse existing', 'Roll new', 'Defer']);
expect(ceoStep0Boundary(f)).toBe(false);
});
});
describe('engStep0Boundary', () => {
test('FIRES on cross-project learnings prompt', () => {
const f = fp('Enable cross-project learnings on this machine?', ['Yes', 'No']);
expect(engStep0Boundary(f)).toBe(true);
});
test('FIRES on scope reduction recommendation', () => {
const f = fp('Scope reduction recommendation: cut to MVP?', ['Reduce', 'Proceed', 'Modify']);
expect(engStep0Boundary(f)).toBe(true);
});
test('does NOT fire on review-section AUQs', () => {
const f = fp('Architecture: shared mutable state?', ['Refactor', 'Defer', 'Skip']);
expect(engStep0Boundary(f)).toBe(false);
});
});
describe('designStep0Boundary', () => {
test('FIRES on design system / posture mention', () => {
const f = fp('Pick a design posture for this review', ['Polish', 'Triage', 'Expansion']);
expect(designStep0Boundary(f)).toBe(true);
});
test('FIRES on first-dimension prompt', () => {
const f = fp('First dimension: visual hierarchy. Score?', ['7', '8', '9']);
expect(designStep0Boundary(f)).toBe(true);
});
test('does NOT fire on later dimension AUQs', () => {
const f = fp('Spacing dimension score?', ['7', '8', '9']);
expect(designStep0Boundary(f)).toBe(false);
});
});
describe('devexStep0Boundary', () => {
test('FIRES on developer persona selection', () => {
const f = fp('Pick the target persona for this review', ['Senior backend', 'Junior frontend', 'Other']);
expect(devexStep0Boundary(f)).toBe(true);
});
test('FIRES on TTHW target prompt', () => {
const f = fp('What is the TTHW target for first run?', ['<5 min', '<15 min', '<30 min']);
expect(devexStep0Boundary(f)).toBe(true);
});
test('does NOT fire on review-section AUQs', () => {
const f = fp('Friction point: 5-min CI wait. Address?', ['Now', 'Defer', 'Skip']);
expect(devexStep0Boundary(f)).toBe(false);
});
});
});