codex exec(non-interactive codex) processes launch N of them simultaneously in the background and split arbitrary tasks among them, collecting each result as a file. Throughput is controlled by the single spawn count (PARALLEL). The general-purpose sibling skill of the image-only [codex-imagegen].
§0. When to use this skill
| Situation | Tool |
|---|---|
| Single task, interactive | Just codex directly |
| Mass-generate N images | codex-imagegen skill ($imagegen + session folder collection) |
| Split N arbitrary tasks to run simultaneously | This skill (code modification, analysis, summary, review, generation…) |
Key difference: imagegen's output lands in ~/.codex/generated_images/and has a collection race, but general tasks are received directly into per-task files with -o(output-last-message), so there is no collection race.
⚠️ The limit is per ChatGPT account. Even if you increase spawns, the account rate limit (tokens/requests per minute) is not duplicated. Excessive automated mass calls risk session invalidation (
refresh_token_invalidated) —codex execis the official path and thus relatively safe, but unlimited spawning is forbidden.
§1. Core mechanism — one codex exec = one task
codex exec \
--skip-git-repo-check \
--dangerously-bypass-approvals-and-sandbox \
-C /path/to/workdir \
-o /path/to/result.md \
"여기에 작업 지시문"
Required/useful flags (codex-cli 0.139.0 basis):
| Flag | Purpose |
|---|---|
-o, --output-last-message <FILE> |
Save the agent's final message to a file — the standard for batch collection (no need to parse stdout) |
--json |
Output events as JSONL to stdout (for parsing reasoning and message streams) |
--output-schema <FILE> |
Force the final response into JSON Schema form — structured result collection |
-C, --cd <DIR> |
Specify task root — the core of parallel isolation |
--add-dir <DIR> |
Additional writable directories |
-s, --sandbox <mode> |
read-only / workspace-write / danger-full-access |
--dangerously-bypass-approvals-and-sandbox |
Skip all approvals and sandboxing (unattended batch) |
--skip-git-repo-check |
Run even outside a git repo |
--ephemeral |
Leave no session file — for parallel bulk execution, ~/.codex prevent non-interactivity issues |
-m, --model <MODEL> |
Specify model |
-i, --image <FILE> |
Attach image |
resume |
Continue from a previous session (codex exec resume --last) |
- The result goes into the
-ofile. Safe batch pattern:read-onlyto analyze/investigate,workspace-write+ modify in an isolated directory. - Unattended execution requires
--dangerously-bypass-approvals-and-sandbox(removes approval prompts). However, this also disables the sandbox, so to go safely, instead consider the combination of-s workspace-write -a never(no approval prompts + workspace-write only).
Connection check (1 test):
codex exec --skip-git-repo-check -s read-only -o /tmp/t.md "이 디렉토리 구조를 3줄로 요약해" -C .
cat /tmp/t.md
§2. Number of spawns — auto-scaling + file conflict isolation
PARALLEL = 동시에 살아있는 codex 프로세스 수. Default auto — it estimates workers according to the number of tasks, and if healthy, automatically ramps up to the cap.
PARALLEL=auto(default): target =min(작업수, MAX),MAXif unspecified,min(16, CPU-1).STARTstarts at (default 3), and +1 everyRAMP_EVERY(default 5) successes; if 429 appears, growth stops. Manual is fixed atPARALLEL=4.- Read-only tasks(analysis, review, summary, research) have no conflicts, so you can just launch them in the same repo with auto (
-s read-only). - File-modifying taskswill conflictif multiple codex instances modify the same repo at the same time. Isolation required:
- per-task directory: if each task touches only its own files/folders, separate with jsonl
cwd(→-C) - git worktree: if the same repo must be modified in parallel, for each task use
git worktree addto create a separate worktree → each codex works in its own worktree → merge when done
- per-task directory: if each task touches only its own files/folders, separate with jsonl
- CLI spawning uses a full codex per process, so memory/CPU is the real bottleneck. That is why auto caps based on CPU and ramps up gradually. Measure rate in steady state (measuring during burst intervals underestimates).
§3. Reference runner (general-purpose, results collected as files)
Input JSONL: per line {id, prompt, cwd?, schema?}. Results are OUTDIR/<id>.md(or when schema, .json).
#!/usr/bin/env python3
# codex_spawn_runner.py — 깡코덱스 codex exec를 자동 스케일링 병렬 스폰, 결과를 파일로 회수
# 실행: TASKS=tasks.jsonl OUTDIR=./out SANDBOX=read-only python3 codex_spawn_runner.py
# PARALLEL=auto(기본) : 작업 수에 맞춰 워커 자동 산정 + 헬시하면 상한까지 램프업
# PARALLEL=4 : 수동 고정. MAX=12 : 자동 상한 override. START=3 : 시작 워커.
import json, os, subprocess, sys, time, threading
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path
TASKS = Path(os.environ.get("TASKS", "tasks.jsonl"))
OUTDIR = Path(os.environ.get("OUTDIR", "./out"))
PARALLEL = os.environ.get("PARALLEL", "auto")
TIMEOUT = int(os.environ.get("TIMEOUT", "600")) # 단일 작업 상한(초)
SANDBOX = os.environ.get("SANDBOX", "read-only") # read-only|workspace-write|danger-full-access
MODEL = os.environ.get("MODEL", "") # 비우면 codex 기본
BYPASS = os.environ.get("BYPASS", "0") == "1" # 1이면 승인·샌드박스 전부 생략
RAMP_EVERY = int(os.environ.get("RAMP_EVERY", "5")) # 성공 N건마다 워커 +1
def auto_target(todo):
"""작업 수 → 목표 워커. codex exec는 I/O 바운드(이미지 API 대기 ~30-45s)라 CPU가
아니라 RAM·계정(250 IPM)이 한도 → 여유 RAM 기반 캡(HARD_CAP 천장 32). MAX로 직접 고정.
예: 여유 6GB·0.4GB/proc → ~15. 큰 머신이면 HARD_CAP까지. 계정은 보통 헤드룸 큼."""
cap = int(os.environ.get("MAX", "0"))
if not cap:
try:
free_gb = os.sysconf("SC_AVPHYS_PAGES") * os.sysconf("SC_PAGE_SIZE") / 1e9
per = float(os.environ.get("RAM_PER_PROC_GB", "0.4"))
cap = max(2, int(free_gb / per))
except (ValueError, OSError, AttributeError):
cap = (os.cpu_count() or 4) * 2 # 비리눅스 폴백(I/O 바운드라 코어×2)
cap = min(int(os.environ.get("HARD_CAP", "32")), cap)
return max(1, min(todo, cap))
class AutoScaler:
"""start 워커로 시작해 헬시하면 RAMP_EVERY 성공마다 +1, target까지. 스로틀 시 성장 정지."""
def __init__(self, target):
self.target = target
self.permits = max(1, min(int(os.environ.get("START", "3")), target))
self.sem = threading.Semaphore(self.permits)
self.lock = threading.Lock()
self.ok = 0
def __enter__(self): self.sem.acquire(); return self
def __exit__(self, *a): self.sem.release()
def live(self): return self.permits
def success(self):
with self.lock:
if self.permits < self.target:
self.ok += 1
if self.ok >= RAMP_EVERY:
self.ok = 0; self.permits += 1; self.sem.release()
def throttle(self):
with self.lock:
self.ok = 0
def _is_throttle(err):
e = (err or "").lower()
return "429" in e or "rate limit" in e or "too many requests" in e
def run_one(item, scaler):
tid = str(item["id"])
schema = item.get("schema")
out = OUTDIR / (f"{tid}.json" if schema else f"{tid}.md")
if out.exists(): # resume
return (tid, "skip", 0)
out.parent.mkdir(parents=True, exist_ok=True)
cmd = ["codex", "exec", "--skip-git-repo-check", "--ephemeral", "-o", str(out)]
if BYPASS:
cmd.append("--dangerously-bypass-approvals-and-sandbox")
else:
cmd += ["-s", SANDBOX]
if MODEL:
cmd += ["-m", MODEL]
if item.get("cwd"):
cmd += ["-C", item["cwd"]] # 격리 디렉토리
if schema:
sp = OUTDIR / f".{tid}.schema.json"
sp.write_text(json.dumps(schema)); cmd += ["--output-schema", str(sp)]
cmd.append(item["prompt"])
with scaler: # 동시 실행 슬롯(천장은 스케일러가 조절)
t0 = time.time()
try:
r = subprocess.run(cmd, stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE,
timeout=TIMEOUT, text=True)
except subprocess.TimeoutExpired:
return (tid, "timeout", time.time()-t0)
if _is_throttle(r.stderr): scaler.throttle()
if out.exists() and out.stat().st_size > 0:
scaler.success()
return (tid, "ok", time.time()-t0)
err = (r.stderr or "").strip().splitlines()[-1:] or [""]
return (tid, f"fail({err[0][:100]})", time.time()-t0)
def main():
items = [json.loads(l) for l in TASKS.read_text().splitlines() if l.strip()]
items = [it for it in items
if not (OUTDIR/(f"{it['id']}.json" if it.get('schema') else f"{it['id']}.md")).exists()]
todo = len(items)
if PARALLEL.isdigit():
target = max(1, int(PARALLEL)); mode = f"manual={target}"
else:
target = auto_target(todo); mode = f"auto→{target}(RAM기반)"
if not todo: print("[done] 처리할 작업 없음."); return 0
scaler = AutoScaler(target)
print(f"[spawn] todo={todo} workers={mode} start={scaler.live()} sandbox={'bypass' if BYPASS else SANDBOX}", flush=True)
ok=fail=0; t0=time.time()
with ThreadPoolExecutor(max_workers=target) as ex:
futs={ex.submit(run_one,it,scaler):it["id"] for it in items}
for f in as_completed(futs):
tid,status,el=f.result()
if status=="ok":
ok+=1; print(f"[ok] {tid} ({el:.0f}s) · 워커 {scaler.live()}/{target}", flush=True)
elif status!="skip":
fail+=1; print(f"[fail#{fail}] {tid} ({el:.0f}s) {status}", flush=True)
print(f"\n=== done: {ok} ok / {fail} fail / {(time.time()-t0)/60:.1f}min · peak 워커 {scaler.live()}/{target} ===")
return 0 if fail==0 else 1
if __name__=="__main__":
sys.exit(main())
minimal bash variant (xargs -Pis the number of spawns, results are collected with -o):
# tasks.jsonl: {"id","prompt","cwd"}
mkdir -p out
jq -c '.' tasks.jsonl | xargs -P 4 -I{} bash -c '
j={}; id=$(jq -r .id <<<"$j"); cwd=$(jq -r ".cwd // \".\"" <<<"$j")
out="out/$id.md"; [ -s "$out" ] && exit 0 # resume
codex exec --skip-git-repo-check --ephemeral -s read-only -C "$cwd" \
-o "$out" "$(jq -r .prompt <<<"$j")" </dev/null >/dev/null 2>&1 \
&& echo "OK $id" || echo "FAIL $id"
'
§4. Structured output (--output-schema)
When results need to be collected as consistent JSON (aggregation, post-processing), give a schema per task:
// tasks.jsonl 한 줄
{"id":"f001","cwd":"./repo","prompt":"이 파일의 보안 이슈를 찾아라",
"schema":{"type":"object","properties":{
"findings":{"type":"array","items":{"type":"object","properties":{
"severity":{"type":"string"},"file":{"type":"string"},"note":{"type":"string"}}}}},
"required":["findings"]}}
→ codex outputs the last message according to this schema, and -o out/f001.jsonfalls into it. The runner automatically saves with .json extension.
§5. resume · retry · cleanup · safety
- resume: if the result file exists, automatically skip. After interruption, just run again → only the remaining ones.
- Do not retry individual failures infinitely within a run — retry in the next pass resume. Consider one pass complete as termination.
- Beware of self-kill bugs:
pkill -f codex_spawn_runnerkills the shell executing that command itself. Process-limited:pkill -9 -f "python3.*codex_spawn_runner" # 러너 pkill -9 -f "codex exec" # 떠있는 워커들 - session file:
--ephemeralif not used, it accumulates in~/.codex/sessions. If running many in parallel,--ephemeralis recommended (excluding tasks that need resume). - Isolation safety: Do not run file-modifying tasks in parallel in the same repo without isolation. If unsure, first
-s read-only. - Unattended risk:
--dangerously-bypass-approvals-and-sandboxalso disables the sandbox. Do not use it for untrusted prompts; use-s workspace-writeisolation.
§6. Checklist
codex exec -s read-only -o /tmp/t.md "..."1 test → whether a result file is createdtasks.jsonlformat:{id, prompt, cwd?, schema?}one per line- Determine task nature: if read-only, isolation unnecessary / if modifying, isolate per-task dir or worktree
PARALLEL=autoSet to (default) to auto-scale according to the number of jobs and CPUs — manual isPARALLEL=N- Run the Python runner (results
-ofile retrieval, so no race) - For failures, rerun with resume; on exit,
pkill -9 -f "codex exec"to clean up the remainder