codex-imagegen

Codex Image-Gen — CLI Parallel Spawn Orchestrator

codex exec(non-interactive codex CLI) as background processes, launching N at once to pull images, and the PNGs codex drops into its own session folder are collected to the destination without races. Spawn count (PARALLEL) alone controls throughput.

§0. What It Is

codex CLI를 launch several as-is to mass-produce images. For a single image, just codexdirectly, and this skill is when pulling dozens to hundreds split up used.

• Concurrency = N OS processeslaunched (ThreadPool / xargs -P). PARALLEL alone controls throughput.
• Auth = codex uses the CLI login state as-is (no token injection needed).
• Output = the PNGs codex drops into ~/.codex/generated_images/are collected to the destination.

⚠️ Limits are per ChatGPT account. Increasing spawns does not duplicate the account limit (250 IPM). Excessive automated mass calls risk session invalidation (refresh_token_invalidated, recovery =codex login re-login) — codex execis the official path and relatively safe, but infinite spawning is a no-go.

§1. Core Mechanism — One codex exec = One Image

Non-interactive single call:

codex exec \
  --skip-git-repo-check \
  --dangerously-bypass-approvals-and-sandbox \
  "Use \$imagegen to generate ONE image.
Aspect ratio: 2:3
Size: 1024x1536
Prompt: <프롬프트>
After generation, do NOT run any shell commands. Just generate and end your turn."

• --skip-git-repo-check : allow execution even outside a git repo
• --dangerously-bypass-approvals-and-sandbox : unattended execution without approval prompts (required for batch)
• At the end of the instruction, "do not run shell, end the turn" must be included — otherwise codex tries to move files itself and gets tangled.
• codex built-in $imagegencreates the image and saves it to ~/.codex/generated_images/{session-uuid}/ig_{40자SHA1}.png .
• Average 3045 sec/image (at 1024); detail-heavy/2048 takes longer. Single-call timeout is 240s recommended.

Connection check (1-image test):

codex exec 'Use $imagegen to generate a 1024x1024 red square test image. End turn immediately after.'
ls -lt ~/.codex/generated_images/**/ig_*.png | head -3

§2. Spawn Count — Auto Scaling (Default)

PARALLEL = 동시에 살아있는 codex 프로세스 수. Default auto — workers are calculated automatically to match the job count, and if things are running healthy it automatically increases up to the cap. No need to hardcode a number.

• PARALLEL=auto(default): target workers = min(작업수, 캡). codex exec is I/O-bound waiting ~30-45 sec for the image APIso it is not CPU cores but RAM and the account that are the real limits → 캡 = based on free RAM(여유GB ÷ RAM_PER_PROC_GB(0.4), ceiling HARD_CAP=32). E.g., 6GB free → ~15; on a big machine, up to 32.
• Ramp-up: START(default 3) → every RAMP_EVERY(default 5) successes +1 → up to the target. If 429/rate limit appears in stderr, growth pauses.
• Manual fixed/expand: PARALLEL=40In this case, lock that value (auto off). To keep it automatic and only raise the ceiling, HARD_CAP=64, if the process is lightweight, RAM_PER_PROC_GB=0.25.

limit	값	meaning
account	250 IPM	~125 workers to saturate → usually headroom is large(the machine hits the limit first)
RAM	free ÷ 0.4GB	actual bottleneck — OOM when exceeded. auto fits this
ceiling	HARD_CAP=32	large machine cap (can override)

• i.e. "since they're subagents, just spin up 100 of them" → the account allows it, but a 7GB box dies from OOM. auto sets it to ~15 based on RAM, and to really increase it further you must increase RAM first.

§3. Retrieval (move) — there's a race trap here

codex drops PNGs into its own session folder, so they must be moved to the destination. Existing move_outputs.py는 "one globally latest ig_*.png after my start time" is picked up:

~/.codex/generated_images/{session}/ig_*.png  중  mtime > after_ts  인 최신 1개

🐛 Race bug (important): this logic does not filter by session uuid does not(the original comment's "session uuid based → race safe" is not true). With parallel spawning, worker A's move can steal worker B's imagethat finished at a similar time. If PARALLEL=1 it's safe, if 2 or more it goes wrong.

Solution — claimed set + lock to guarantee 1:1. A file retrieved once cannot be picked up again (reflected in the §4 runner below). Alternative: codex exec parse the session id from stdout (version dependent, not recommended).

Retrieval failure = ~/.codex/generated_images/no new file in → usually moderation refusal(§5).

§4. Reference runner (race-fixed version)

batch_runner_parallel.pya race-safe fixed version. Input is {id, prompt, ar, size, output_path} JSONL.

#!/usr/bin/env python3
# codex_imagegen_runner.py — codex exec를 자동 스케일링 병렬 스폰 → 레이스-세이프 회수
# 실행: PROMPTS=prompts.jsonl OUTDIR=./out python3 codex_imagegen_runner.py
#   PARALLEL=auto(기본) : 작업 수에 맞춰 워커 자동 산정 + 헬시하면 상한까지 램프업
#   PARALLEL=8          : 수동 고정.   MAX=12 : 자동 상한 override.   START=3 : 시작 워커.
import json, os, subprocess, sys, time, threading, shutil
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path

PROMPTS  = Path(os.environ.get("PROMPTS", "prompts.jsonl"))
OUTDIR   = Path(os.environ.get("OUTDIR", "./out"))
PARALLEL = os.environ.get("PARALLEL", "auto")
TIMEOUT  = int(os.environ.get("TIMEOUT", "240"))
RAMP_EVERY = int(os.environ.get("RAMP_EVERY", "5"))   # 성공 N건마다 워커 +1
CODEX_IMG = Path.home() / ".codex" / "generated_images"

def auto_target(todo):
    """작업 수 → 목표 워커. codex exec는 I/O 바운드(이미지 API 대기 ~30-45s)라 CPU가
    아니라 RAM·계정(250 IPM)이 한도 → 여유 RAM 기반 캡(HARD_CAP 천장 32). MAX로 직접 고정.
    예: 여유 6GB·0.4GB/proc → ~15. 큰 머신이면 HARD_CAP까지. 계정은 보통 헤드룸 큼."""
    cap = int(os.environ.get("MAX", "0"))
    if not cap:
        try:
            free_gb = os.sysconf("SC_AVPHYS_PAGES") * os.sysconf("SC_PAGE_SIZE") / 1e9
            per = float(os.environ.get("RAM_PER_PROC_GB", "0.4"))
            cap = max(2, int(free_gb / per))
        except (ValueError, OSError, AttributeError):
            cap = (os.cpu_count() or 4) * 2          # 비리눅스 폴백(I/O 바운드라 코어×2)
        cap = min(int(os.environ.get("HARD_CAP", "32")), cap)
    return max(1, min(todo, cap))

class AutoScaler:
    """start 워커로 시작해 헬시하면 RAMP_EVERY 성공마다 +1, target까지. 스로틀 시 성장 정지."""
    def __init__(self, target):
        self.target = target
        self.permits = max(1, min(int(os.environ.get("START", "3")), target))
        self.sem = threading.Semaphore(self.permits)
        self.lock = threading.Lock()
        self.ok = 0
    def __enter__(self): self.sem.acquire(); return self
    def __exit__(self, *a): self.sem.release()
    def live(self): return self.permits
    def success(self):
        with self.lock:
            if self.permits < self.target:
                self.ok += 1
                if self.ok >= RAMP_EVERY:
                    self.ok = 0; self.permits += 1; self.sem.release()  # 천장 +1
    def throttle(self):
        with self.lock:
            self.ok = 0  # 429 보이면 성장 일시정지(개별 호출은 codex가 백오프)

_lock = threading.Lock()
_claimed = set()   # 이미 회수한 png 경로 — 레이스 방지 핵심
def newest_unclaimed(after_ts):
    with _lock:
        best = None
        if CODEX_IMG.exists():
            for sess in CODEX_IMG.iterdir():
                if not sess.is_dir(): continue
                for png in sess.glob("ig_*.png"):
                    p = str(png)
                    if p in _claimed: continue
                    try: m = png.stat().st_mtime
                    except OSError: continue
                    if m > after_ts and (best is None or m > best[1]): best = (png, m)
        if best:
            _claimed.add(str(best[0])); return best[0]
        return None

def _is_throttle(err):
    e = (err or "").lower()
    return "429" in e or "rate limit" in e or "too many requests" in e

def run_one(item, scaler):
    pid = item["id"]; out = OUTDIR / item["output_path"]
    if out.exists(): return (pid, "skip", 0)   # resume
    instr = (f"Use $imagegen to generate ONE image.\n"
             f"Aspect ratio: {item.get('ar','1:1')}\n"
             f"Size: {item.get('size','1024x1536')}\n"
             f"Prompt: {item['prompt']}\n"
             f"After generation, do NOT run any shell commands. Just generate and end your turn.")
    with scaler:                               # 동시 실행 슬롯(천장은 스케일러가 조절)
        before = time.time() - 1
        try:
            r = subprocess.run(
                ["codex","exec","--skip-git-repo-check","--dangerously-bypass-approvals-and-sandbox", instr],
                stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE,
                timeout=TIMEOUT, text=True)
        except subprocess.TimeoutExpired:
            return (pid, "timeout", time.time()-before)
        if _is_throttle(r.stderr): scaler.throttle()
        deadline = time.time() + 30; src = None
        while time.time() < deadline:
            src = newest_unclaimed(before)
            if src: break
            time.sleep(1)
    if not src:
        return (pid, "rejected/no-image", time.time()-before)   # 대개 모더레이션 거부
    out.parent.mkdir(parents=True, exist_ok=True); shutil.move(str(src), str(out))
    scaler.success()
    return (pid, "ok", time.time()-before)

def main():
    items = [json.loads(l) for l in PROMPTS.read_text().splitlines() if l.strip()]
    items = [it for it in items if not (OUTDIR/it["output_path"]).exists()]   # resume
    todo = len(items)
    if PARALLEL.isdigit():
        target = max(1, int(PARALLEL)); mode = f"manual={target}"
    else:
        target = auto_target(todo); mode = f"auto→{target}(RAM기반)"
    if not todo: print("[done] 처리할 작업 없음."); return 0
    scaler = AutoScaler(target)
    print(f"[spawn] todo={todo} workers={mode} start={scaler.live()} ramp=+1/{RAMP_EVERY}ok", flush=True)
    ok=fail=0; t0=time.time()
    with ThreadPoolExecutor(max_workers=target) as ex:
        futs={ex.submit(run_one,it,scaler):it["id"] for it in items}
        for f in as_completed(futs):
            pid,status,el=f.result()
            if status=="ok":
                ok+=1
                if ok%10==0:
                    rate=ok/max(time.time()-t0,1)*60
                    print(f"[progress] {ok}/{todo} ok · {rate:.1f}/min · 워커 {scaler.live()}/{target}", flush=True)
            elif status!="skip":
                fail+=1; print(f"[fail#{fail}] {pid} ({el:.0f}s) {status}", flush=True)
    print(f"\n=== done: {ok} ok / {fail} fail / {(time.time()-t0)/60:.1f}min · peak 워커 {scaler.live()}/{target} ===")
    return 0 if fail==0 else 1

if __name__=="__main__":
    sys.exit(main())

minimal bash variant (when you want to see only "true background spawning" without dependencies — xargs -Pis the spawn count):

# prompts.jsonl: 한 줄당 {"id","prompt","size","output_path"}
jq -c '.' prompts.jsonl | xargs -P 3 -I{} bash -c '
  j={}; id=$(jq -r .id <<<"$j"); out=$(jq -r .output_path <<<"$j")
  [ -f "$out" ] && exit 0                       # resume
  before=$(date +%s)
  codex exec --skip-git-repo-check --dangerously-bypass-approvals-and-sandbox \
    "Use \$imagegen to generate ONE image. Size: $(jq -r .size <<<"$j"). Prompt: $(jq -r .prompt <<<"$j"). Do not run shell. End turn." \
    </dev/null >/dev/null 2>&1
  png=$(find ~/.codex/generated_images -name "ig_*.png" -newermt "@$before" | head -1)
  [ -n "$png" ] && { mkdir -p "$(dirname "$out")"; mv "$png" "$out"; echo "OK $id"; } || echo "FAIL $id"
'
# ⚠️ 이 bash판은 §3 레이스를 막지 못함 — PARALLEL>1이면 파이썬 러너(claimed 락) 권장

§5. Handling moderation refusal

• On refusal, codex outputs I'm unable to generate this image due to content policy restrictions. and such and does not create a new png → in the runner rejected/no-image.
• no bypassing (making it explicit). Toning down the expression to advertising/campaign languageoften gets it through (e.g. "gravure/wet-look" → clean campaign description, "lingerie" → "loungewear" framing).
• Extract only the rejected parts, soften the prompt, and resubmit:

# 실패 id 목록으로 jsonl 필터링 → 프롬프트 수정 → resume 재실행

§6. resume · retry · cleanup

• resume: the runner output_path automatically skips if it exists. If you just run it again after interruption, it processes only what's left.
• No infinite retries for individual failures within a run — they are naturally retried in the next pass's resume. Treat completion of 1 pass as the end.
• Never post-process the characters: Do not patch cuts with wrong or missing text using PIL/ImageMagick/SVG overlays — the font, kerning, and tone will look out of place and ruin the result. Fix the prompt for just that cut (make the typography concrete → 2048x2048 + quality high → reduce the amount of copy) and re-submit to re-render within the image.
• Beware of the suicide bug: pkill -f codex_spawn_runner kills the shell itself that ran that command (including the string in the command line). Always limit to the process:

  pkill -9 -f "python3.*codex_spawn_runner"   # 러너만
  pkill -9 -f "codex exec"                      # 떠있는 codex 워커들
  

• Disk: ~/.codex/generated_images/ accumulates as unreclaimed leftovers. Clean up periodically, and before a batch check for 2GB+ free .
• Long-running in the background: Run the runner itself in the background and track progress via logs (the harness's run_in_background or nohup ... &).

§7. Prompt and size rules (gpt-image-2)

• Only fixed size sets: 1024x1024 / 1536x1024 / 1024x1536 / 1792x1024 / 1024x1792 / 2048x2048. 512x512 are rejected. Map 4:5, 16:9, etc. to the nearest values (16:9→1792x1024, 9:16→1024x1792, 2:3→1024x1536).
• quality: low/medium/high. For comics, text, and fine lines, high + 2048 otherwise it gets smeared. For general use, medium.
• Do not use negative prompts: "no ~" negations degrade quality. Replace with positive descriptions.
• If text rendering is needed, specify "render text exactly" in the instruction. Korean rendering is a strength.

§8. Checklist

1. codex execTest with 1 image → ~/.codex/generated_images/check that the png lands in
2. prompts.jsonl format: {id, prompt, ar, size, output_path} one line each
3. Set PARALLEL to match your account (Plus 1 / Pro start at 3~), check for 2GB+ disk
4. Run the Python runner (claimed lock) — the bash version races when PARALLEL>1
5. Check progress/failure logs → tone down the rejected ones and re-submit with resume
6. On exit, pkill -9 -f "codex exec"clean up leftover workers and clean up leftover pngs