mirror of
https://github.com/garrytan/gstack.git
synced 2026-05-08 13:39:45 +08:00
docs: update BROWSER.md + TODOS.md for v1.28.0.0
BROWSER.md picks up a "Headed mode + proxy + browser-native downloads (v1.28.0.0)" subsection inside Real-browser mode plus the new source-map entries (socks-bridge.ts, proxy-config.ts, proxy-redact.ts, xvfb.ts, stealth.ts). TODOS.md anti-bot-stealth item updated to reflect the v1.28 narrowing — the "fake plugins" line is no longer accurate. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
64
BROWSER.md
64
BROWSER.md
@@ -49,7 +49,7 @@ $B connect # headed Chromium + Side Panel extension
|
||||
5. [Snapshot system + ref-based selection](#snapshot-system)
|
||||
6. [Browser-skills runtime](#browser-skills-runtime)
|
||||
7. [Domain-skills (per-site agent notes)](#domain-skills)
|
||||
8. [Real-browser mode (`$B connect`)](#real-browser-mode)
|
||||
8. [Real-browser mode (`$B connect`)](#real-browser-mode) — including [`--headed` + `--proxy` + `--navigate` (v1.28.0.0)](#headed-mode--proxy--browser-native-downloads-v12800)
|
||||
9. [Side Panel + sidebar agent](#side-panel--sidebar-agent)
|
||||
10. [Pair-agent — remote agents over an ngrok tunnel](#pair-agent)
|
||||
11. [Authentication + tokens](#authentication)
|
||||
@@ -545,6 +545,63 @@ When in real-browser mode, `/qa` and `/design-review` automatically skip
|
||||
cookie import prompts and headless workarounds — the headed browser already
|
||||
has whatever session you logged into.
|
||||
|
||||
### Headed mode + proxy + browser-native downloads (v1.28.0.0)
|
||||
|
||||
Three coordinated flags for sites that block headless browsers, fingerprint
|
||||
Playwright defaults, or sit behind authenticated upstream proxies:
|
||||
|
||||
```bash
|
||||
# Visible Chromium. Auto-spawns Xvfb on Linux containers without DISPLAY.
|
||||
$B --headed goto https://example.com
|
||||
|
||||
# SOCKS5 with auth — Chromium can't prompt for SOCKS5 creds, so $B runs a
|
||||
# local 127.0.0.1 bridge that handles the auth handshake.
|
||||
$B --proxy socks5://user:pass@residential.proxy.host:1080 goto https://example.com
|
||||
|
||||
# HTTP/HTTPS proxy passes through to Chromium directly.
|
||||
$B --proxy http://corp-proxy:3128 goto https://example.com
|
||||
|
||||
# Browser-native download for Content-Disposition, redirect chains, anti-bot
|
||||
# CDNs where page.request.fetch() falls over.
|
||||
$B download "https://protected.example.com/file" /tmp/file.bin --navigate
|
||||
|
||||
# Combined.
|
||||
$B --headed --proxy socks5://user:pass@host:1080 \
|
||||
download "https://protected.example.com/file" /tmp/file.bin --navigate
|
||||
```
|
||||
|
||||
**Credential policy.** Pass creds via the URL (`socks5://user:pass@host`) OR
|
||||
the env vars `BROWSE_PROXY_USER` / `BROWSE_PROXY_PASS` — never both. `$B`
|
||||
refuses with a clear hint when both are set; silent override created
|
||||
"works on my machine" debugging traps.
|
||||
|
||||
**Daemon discipline.** `--proxy` and `--headed` are daemon-startup config.
|
||||
A running daemon with config A meeting a new invocation with config B exits
|
||||
1 with a `browse disconnect` hint instead of silently restarting and dropping
|
||||
tab state, cookies, or sessions.
|
||||
|
||||
**Stealth scope.** When `--headed` or `--proxy` are set, `$B` masks
|
||||
`navigator.webdriver` only — via Chromium's
|
||||
`--disable-blink-features=AutomationControlled` plus a small init script.
|
||||
We do NOT fake `navigator.plugins`, `navigator.languages`, or `window.chrome`
|
||||
— modern fingerprinters check those for consistency, and synthesizing fixed
|
||||
values can flag MORE bot-like, not less. ChromeDriver's `cdc_` runtime
|
||||
artifacts and the Permissions API patch are still cleaned up.
|
||||
|
||||
**Container support.** `--headed` on Linux without `DISPLAY` walks the
|
||||
display range (`:99`, `:100`, ...) until `xdpyinfo` reports a free slot,
|
||||
then spawns Xvfb. Cleanup-on-disconnect validates the recorded PID's
|
||||
`/proc/<pid>/cmdline` matches `Xvfb` AND start-time matches before sending
|
||||
any signal — no PID-reuse footguns. Skips spawn entirely when
|
||||
`WAYLAND_DISPLAY` is set (Chromium uses Wayland natively). Standard
|
||||
Debian/Ubuntu containers work out of the box; minimal images (alpine,
|
||||
distroless) may need fonts/dbus/gtk libs for headed Chromium to render.
|
||||
|
||||
**Failure modes.** SOCKS5 upstream rejected or unreachable — fail-fast at
|
||||
startup with a redacted error after 3 retries (5s budget). Mid-stream
|
||||
upstream drop — bridge kills the affected client connection only; no
|
||||
transport retries that could corrupt browser traffic.
|
||||
|
||||
---
|
||||
|
||||
## Side Panel + sidebar agent
|
||||
@@ -1117,6 +1174,11 @@ browse/
|
||||
│ ├── cli.ts # Thin client — reads state, sends HTTP, prints
|
||||
│ ├── server.ts # Bun HTTP daemon — routes commands, dual-listener
|
||||
│ ├── browser-manager.ts # Chromium lifecycle, tabs, ref map, crash detection
|
||||
│ ├── socks-bridge.ts # Local 127.0.0.1 SOCKS5 bridge that handles auth handshakes Chromium can't speak
|
||||
│ ├── proxy-config.ts # --proxy URL parsing + cred resolution (URL vs env, fail-fast on both)
|
||||
│ ├── proxy-redact.ts # Cred-redaction helper for any proxy URL surfaced to logs/errors
|
||||
│ ├── xvfb.ts # Xvfb auto-spawn + orphan cleanup with PID + start-time validation
|
||||
│ ├── stealth.ts # navigator.webdriver mask + cdc_ cleanup + Permissions API patch
|
||||
│ ├── browse-client.ts # Canonical SDK — what skills import as _lib/browse-client.ts
|
||||
│ ├── snapshot.ts # AX tree → @e/@c refs → Locator map; -D/-a/-C handling
|
||||
│ ├── read-commands.ts # Non-mutating: text, html, links, js, css, is, dialog, ...
|
||||
|
||||
2
TODOS.md
2
TODOS.md
@@ -1562,7 +1562,7 @@ Shipped in v0.6.5. TemplateContext in gen-skill-docs.ts bakes skill name into pr
|
||||
|
||||
**What:** Write a postinstall script that patches Playwright's CDP layer to suppress `Runtime.enable` and use `addBinding` for context ID discovery, same approach as rebrowser-patches. Eliminates the `navigator.webdriver`, `cdc_` markers, and other CDP artifacts that sites like Google use to detect automation.
|
||||
|
||||
**Why:** Our current stealth patches (UA override, navigator.webdriver=false, fake plugins) work on most sites but Google still triggers captchas. The real detection is at the CDP protocol level. rebrowser-patches proved the approach works but their patches target Playwright 1.52.0 and don't apply to our 1.58.2. We need our own patcher using string matching instead of line-number diffs. 6 files, ~200 lines of patches total.
|
||||
**Why:** Our current stealth narrows to `navigator.webdriver` masking + ChromeDriver `cdc_` runtime cleanup + Permissions API patch (v1.28.0.0 narrowed it from also faking plugins/languages, since modern fingerprinters punish inconsistent fakes more than they punish admitted defaults). That's enough for most sites but Google still triggers captchas, because the real detection is at the CDP protocol level. rebrowser-patches proved the approach works but their patches target Playwright 1.52.0 and don't apply to our 1.58.2. We need our own patcher using string matching instead of line-number diffs. 6 files, ~200 lines of patches total.
|
||||
|
||||
**Context:** Full analysis of rebrowser-patches source: patches 6 files in `playwright-core/lib/server/` (crConnection.js, crDevTools.js, crPage.js, crServiceWorker.js, frames.js, page.js). Key technique: suppress `Runtime.enable` (the main CDP detection vector), use `Runtime.addBinding` + `CustomEvent` trick to discover execution context IDs without it. Our extension communicates via Chrome extension APIs, not CDP Runtime, so it should be unaffected. Write E2E tests that verify: (1) extension still loads and connects, (2) Google.com loads without captcha, (3) sidebar chat still works.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user