runtime-monitor

Runtime Monitor

Use this skill when the task is real workload monitoring, not vague status-checking.

The canonical operator surfaces are:

• joelclaw jobs status
• joelclaw queue depth
• joelclaw queue control status
• joelclaw queue observe
• joelclaw restate status
• joelclaw restate cron status
• pi extension tool: runtime_jobs_monitor

Load order

Before doing runtime monitor work:

1. load system-architecture
2. load pi-tui-design if you are building or changing the monitor widget/TUI
3. load joelclaw for CLI operator flows

What to use when

First glance

joelclaw jobs status --hours 1 --count 10

This is the first operator answer to:

• can the system take more work right now?
• is the queue/runtime layer healthy?
• is Dkron alive?
• is Inngest still healthy during transition?

Queue-specific investigation

joelclaw queue depth
joelclaw queue control status --hours 1
joelclaw queue observe --hours 1
joelclaw queue stats --since <iso|ms>

Use these once jobs status tells you the queue layer is the interesting bit.

Runtime substrate investigation

joelclaw restate status
joelclaw restate cron status
joelclaw runs --count 20 --hours 1

Use these when the top-level surface says Restate, Dkron, or transitional Inngest needs a closer look.

Async pi monitor

The loaded pi extension at packages/pi-extensions/inngest-monitor/index.ts exposes:

• runtime_jobs_monitor
• inngest_send
• inngest_runs

Start

{"action":"start","interval":5,"report":true}

Effects:

• polls joelclaw jobs status in the background
• paints a persistent TUI widget
• emits OTEL on runtime severity changes and meaningful workload-state changes
• sends hidden follow-up summaries when the runtime meaningfully changes or the monitor stops/times out

Status

{"action":"status"}

Returns the latest runtime snapshot:

• overall status/summary
• queue depth
• active pause count
• Restate / Dkron / Inngest state

Stop

{"action":"stop"}

Stops the background poller and sends a final follow-up summary.

Rules

• joelclaw jobs status is the aggregated truth surface; do not rebuild the same picture by hand unless the command is wrong.
• If jobs status is noisy or misleading, report the defect and use bounded underlying evidence. Change the CLI only when implementation is in scope.
• For TUI work, prefer compact status blocks with explicit severity and short summaries. No decorative sludge.
• Any runtime monitor widget line must be clamped to the active terminal width with pi-tui truncation (truncateToWidth / visibleWidth). Pi will crash narrow terminals if a custom widget emits over-wide lines.
• During transition, Inngest stays visible but must not dominate the runtime story when Restate + queue + Dkron are the real workload path.
• Report earned truth only. If the monitor compiles but hasn’t been dogfooded, say that.