hai/gstack
1
0
Fork 0
mirror of https://github.com/garrytan/gstack.git synced 2026-05-19 19:02:29 +08:00
Code Issues Packages Projects Releases Wiki Activity

fix: unbuffer Python stdout in codex --json streaming

Browse Source 
Python fully buffers stdout when piped (not a TTY). The
`codex exec --json | python3 -c "..."` pattern meant zero output
visible until process exit — users saw nothing for 30+ minutes.

Add PYTHONUNBUFFERED=1 env var, python3 -u flag, and flush=True
to all print() calls in all three Python parser blocks (Challenge,
Consult new session, Consult resumed session).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-03-26 12:17:02 -06:00
parent 997f7b1da6
commit ec069b9d07
2 changed files with 30 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
codex/SKILL.md
View File

@@ -520,7 +520,7 @@ With focus (e.g., "security"):
2. Run codex exec with **JSONL output** to capture reasoning traces and tool calls (5-minute timeout): 2. Run codex exec with **JSONL output** to capture reasoning traces and tool calls (5-minute timeout):
```bash ```bash
codex exec "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>/dev/null | python3 -c " codex exec "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>/dev/null | PYTHONUNBUFFERED=1 python3 -u -c "
import sys, json import sys, json
for line in sys.stdin: for line in sys.stdin:
line = line.strip() line = line.strip()
@@ -533,17 +533,17 @@ for line in sys.stdin:
itype = item.get('type','') itype = item.get('type','')
text = item.get('text','') text = item.get('text','')
if itype == 'reasoning' and text: if itype == 'reasoning' and text:
print(f'[codex thinking] {text}') print(f'[codex thinking] {text}', flush=True)
print() print(flush=True)
elif itype == 'agent_message' and text: elif itype == 'agent_message' and text:
print(text) print(text, flush=True)
elif itype == 'command_execution': elif itype == 'command_execution':
cmd = item.get('command','') cmd = item.get('command','')
if cmd: print(f'[codex ran] {cmd}') if cmd: print(f'[codex ran] {cmd}', flush=True)
elif t == 'turn.completed': elif t == 'turn.completed':
usage = obj.get('usage',{}) usage = obj.get('usage',{})
tokens = usage.get('input_tokens',0) + usage.get('output_tokens',0) tokens = usage.get('input_tokens',0) + usage.get('output_tokens',0)
if tokens: print(f'\ntokens used: {tokens}') if tokens: print(f'\ntokens used: {tokens}', flush=True)
except: pass except: pass
" "
``` ```
@@ -605,7 +605,7 @@ THE PLAN:
For a **new session:** For a **new session:**
```bash ```bash
codex exec "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>"$TMPERR" | python3 -c " codex exec "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>"$TMPERR" | PYTHONUNBUFFERED=1 python3 -u -c "
import sys, json import sys, json
for line in sys.stdin: for line in sys.stdin:
line = line.strip() line = line.strip()
@@ -615,31 +615,31 @@ for line in sys.stdin:
t = obj.get('type','') t = obj.get('type','')
if t == 'thread.started': if t == 'thread.started':
tid = obj.get('thread_id','') tid = obj.get('thread_id','')
if tid: print(f'SESSION_ID:{tid}') if tid: print(f'SESSION_ID:{tid}', flush=True)
elif t == 'item.completed' and 'item' in obj: elif t == 'item.completed' and 'item' in obj:
item = obj['item'] item = obj['item']
itype = item.get('type','') itype = item.get('type','')
text = item.get('text','') text = item.get('text','')
if itype == 'reasoning' and text: if itype == 'reasoning' and text:
print(f'[codex thinking] {text}') print(f'[codex thinking] {text}', flush=True)
print() print(flush=True)
elif itype == 'agent_message' and text: elif itype == 'agent_message' and text:
print(text) print(text, flush=True)
elif itype == 'command_execution': elif itype == 'command_execution':
cmd = item.get('command','') cmd = item.get('command','')
if cmd: print(f'[codex ran] {cmd}') if cmd: print(f'[codex ran] {cmd}', flush=True)
elif t == 'turn.completed': elif t == 'turn.completed':
usage = obj.get('usage',{}) usage = obj.get('usage',{})
tokens = usage.get('input_tokens',0) + usage.get('output_tokens',0) tokens = usage.get('input_tokens',0) + usage.get('output_tokens',0)
if tokens: print(f'\ntokens used: {tokens}') if tokens: print(f'\ntokens used: {tokens}', flush=True)
except: pass except: pass
" "
``` ```
For a **resumed session** (user chose "Continue"): For a **resumed session** (user chose "Continue"):
```bash ```bash
codex exec resume <session-id> "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>"$TMPERR" | python3 -c " codex exec resume <session-id> "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>"$TMPERR" | PYTHONUNBUFFERED=1 python3 -u -c "
<same python streaming parser as above> <same python streaming parser as above, with flush=True on all print() calls>
" "
``` ```

30
codex/SKILL.md.tmpl
View File

@@ -159,7 +159,7 @@ With focus (e.g., "security"):
2. Run codex exec with **JSONL output** to capture reasoning traces and tool calls (5-minute timeout): 2. Run codex exec with **JSONL output** to capture reasoning traces and tool calls (5-minute timeout):
```bash ```bash
codex exec "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>/dev/null | python3 -c " codex exec "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>/dev/null | PYTHONUNBUFFERED=1 python3 -u -c "
import sys, json import sys, json
for line in sys.stdin: for line in sys.stdin:
line = line.strip() line = line.strip()
@@ -172,17 +172,17 @@ for line in sys.stdin:
itype = item.get('type','') itype = item.get('type','')
text = item.get('text','') text = item.get('text','')
if itype == 'reasoning' and text: if itype == 'reasoning' and text:
print(f'[codex thinking] {text}') print(f'[codex thinking] {text}', flush=True)
print() print(flush=True)
elif itype == 'agent_message' and text: elif itype == 'agent_message' and text:
print(text) print(text, flush=True)
elif itype == 'command_execution': elif itype == 'command_execution':
cmd = item.get('command','') cmd = item.get('command','')
if cmd: print(f'[codex ran] {cmd}') if cmd: print(f'[codex ran] {cmd}', flush=True)
elif t == 'turn.completed': elif t == 'turn.completed':
usage = obj.get('usage',{}) usage = obj.get('usage',{})
tokens = usage.get('input_tokens',0) + usage.get('output_tokens',0) tokens = usage.get('input_tokens',0) + usage.get('output_tokens',0)
if tokens: print(f'\ntokens used: {tokens}') if tokens: print(f'\ntokens used: {tokens}', flush=True)
except: pass except: pass
" "
``` ```
@@ -244,7 +244,7 @@ THE PLAN:
For a **new session:** For a **new session:**
```bash ```bash
codex exec "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>"$TMPERR" | python3 -c " codex exec "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>"$TMPERR" | PYTHONUNBUFFERED=1 python3 -u -c "
import sys, json import sys, json
for line in sys.stdin: for line in sys.stdin:
line = line.strip() line = line.strip()
@@ -254,31 +254,31 @@ for line in sys.stdin:
t = obj.get('type','') t = obj.get('type','')
if t == 'thread.started': if t == 'thread.started':
tid = obj.get('thread_id','') tid = obj.get('thread_id','')
if tid: print(f'SESSION_ID:{tid}') if tid: print(f'SESSION_ID:{tid}', flush=True)
elif t == 'item.completed' and 'item' in obj: elif t == 'item.completed' and 'item' in obj:
item = obj['item'] item = obj['item']
itype = item.get('type','') itype = item.get('type','')
text = item.get('text','') text = item.get('text','')
if itype == 'reasoning' and text: if itype == 'reasoning' and text:
print(f'[codex thinking] {text}') print(f'[codex thinking] {text}', flush=True)
print() print(flush=True)
elif itype == 'agent_message' and text: elif itype == 'agent_message' and text:
print(text) print(text, flush=True)
elif itype == 'command_execution': elif itype == 'command_execution':
cmd = item.get('command','') cmd = item.get('command','')
if cmd: print(f'[codex ran] {cmd}') if cmd: print(f'[codex ran] {cmd}', flush=True)
elif t == 'turn.completed': elif t == 'turn.completed':
usage = obj.get('usage',{}) usage = obj.get('usage',{})
tokens = usage.get('input_tokens',0) + usage.get('output_tokens',0) tokens = usage.get('input_tokens',0) + usage.get('output_tokens',0)
if tokens: print(f'\ntokens used: {tokens}') if tokens: print(f'\ntokens used: {tokens}', flush=True)
except: pass except: pass
" "
``` ```
For a **resumed session** (user chose "Continue"): For a **resumed session** (user chose "Continue"):
```bash ```bash
codex exec resume <session-id> "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>"$TMPERR" | python3 -c " codex exec resume <session-id> "<prompt>" -C "$(git rev-parse --show-toplevel)" -s read-only -c 'model_reasoning_effort="xhigh"' --enable web_search_cached --json 2>"$TMPERR" | PYTHONUNBUFFERED=1 python3 -u -c "
<same python streaming parser as above> <same python streaming parser as above, with flush=True on all print() calls>
" "
``` ```