test: /review hardening — NOT-READY env isolation, workdir cleanup, perf

Applied from the adversarial subagent pass during /review on this branch:

- test/benchmark-cli.test.ts — new "NOT READY path fires when auth env
  vars are stripped" test. The default dry-run test always showed OK on
  dev machines with auth, hiding regressions in the remediation-hint
  branch. Stripped env (no auth vars, HOME→empty tmpdir) now force-
  exercises gpt + gemini NOT READY paths and asserts every NOT READY
  line includes a concrete remediation hint (install/login/export).
  (claude adapter's os.homedir() call is Bun-cached; the 2-of-3 adapter
  coverage is sufficient to exercise the branch.)

- test/taste-engine.test.ts — session-cap test rewritten to seed the
  profile with 50 entries + one real CLI call, instead of 55 sequential
  subprocess spawns. Same coverage (FIFO eviction at the boundary), ~5s
  faster CI time. Also pins first-casing-wins on the Geist/GEIST merge
  assertion — bumpPref() keeps the first-arrival casing, so the test
  documents that policy.

- test/skill-e2e-benchmark-providers.test.ts — workdir creation moved
  from module-load into beforeAll, cleanup added in afterAll. Previous
  shape leaked a /tmp/bench-e2e-* dir every CI run.

- test/publish-dry-run.test.ts — removed unused empty test/helpers
  mkdirSync from the sandbox setup. The bin doesn't import from there,
  so the empty dir was a footgun for future maintainers.

- test/helpers/providers/gpt.ts — expanded the inline comment on
  `--skip-git-repo-check` to explicitly note that `-s read-only` is now
  load-bearing safety (the trust prompt was the secondary boundary;
  removing read-only while keeping skip-git-repo-check would be unsafe).

Net: 45 passing tests (was 44), session-cap test 5s faster, one real
regression surface covered that didn't exist before.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-04-18 06:54:09 +08:00
parent 6a8b637669
commit 620f5dbaea
5 changed files with 90 additions and 13 deletions

View File

@@ -18,7 +18,7 @@
* - Multi-turn tool-using prompts — our single-turn smoke skips `toolCalls > 0`
*/
import { describe, test, expect } from 'bun:test';
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { ClaudeAdapter } from './helpers/providers/claude';
import { GptAdapter } from './helpers/providers/gpt';
import { GeminiAdapter } from './helpers/providers/gemini';
@@ -41,9 +41,20 @@ const gpt = new GptAdapter();
const gemini = new GeminiAdapter();
// Use a temp working directory so provider CLIs can't accidentally touch the repo.
const workdir = fs.mkdtempSync(path.join(os.tmpdir(), 'bench-e2e-'));
// Created in beforeAll / cleaned in afterAll so concurrent CI runs don't leak.
let workdir: string;
describeIfEvals('multi-provider benchmark adapters (live)', () => {
beforeAll(() => {
workdir = fs.mkdtempSync(path.join(os.tmpdir(), 'bench-e2e-'));
});
afterAll(() => {
if (workdir && fs.existsSync(workdir)) {
fs.rmSync(workdir, { recursive: true, force: true });
}
});
test('claude: available() returns structured ok/reason', async () => {
const check = await claude.available();
expect(check).toHaveProperty('ok');