background-jobs

Background jobs

The HTTP request does not wait for email, PDF, heavy outbound webhook or embed. Enqueue and respond. Outbox/domain events: event-driven-architecture. Redis broker: redis-cache (queue keys ≠ cache). Inbound HTTP from third parties: webhooks.

Preserve the repo's runner (Bull/BullMQ, Sidekiq, River, Oban, Hangfire, pg-boss). Do not swap the library.

Non-negotiable rules

• Payload small: IDs + type. The worker reads the canonical one in the DB. No 2 MB blob in the queue.
• Idempotency: the same job_id / idempotency_key does not apply the effect twice (unique in Postgres or SET NX).
• Retry with backoff + cap. Business error (4xx logical) does not retry infinitely.
• Timeout on the job. Cooperative cancellation if the runtime allows it.
• Dead letter / failed set visible; alarm. Do not hide the error.

When to enqueue

• I/O > ~200ms that the user does not need in the same response (email, thumbnail, sync).
• Fan-out, retry, schedule (cron / every).
• Outbound webhook (webhooks).

No: "async" to hide a race. Money invariant in the request + tx (postgres).

Job contract

name / queue:  email.send
idempotency:   email:send:{messageId}
payload:       { messageId }
attempts:      5, backoff exp
timeout:       30s

• Separate queues by urgency/IO (default vs critical vs low). Not a single queue for everything if the volume already hurt.
• Priority only if the driver already has it; otherwise queues.
• Schedule: the project's scheduler, not setInterval in the web process.

Worker

• SELECT … FOR UPDATE SKIP LOCKED if the queue is a table (postgres).
• One effect per job. Compensate in the next job, not a silent "half apply".
• Logs: job_id, attempts, duration — without payload PII.
• Limited concurrency (worker DB pool smaller than the API's).

Failures

Failure	Action
Network / partner 5xx	Retry backoff
4xx / validation	Fail, DLQ, no retry
Timeout	Retry or DLQ depending on the job
Duplicate deliver	No-op (idempotent)

Do not catch generic that marks success. Deploy: drain or interruptible job (production-readiness).

Anti-patterns

• await sendEmail() in the HTTP handler if a worker already exists
• Payload = entire stale entity
• Retry of "card already captured" until infinity
• Cron that relaunches the same work without unique
• Worker and API in the same process without isolation if the repo already separates them
• Redis queue without persistence for unrecoverable money jobs (use outbox + DB)

Integration with other skills

• Outbox / bus: event-driven-architecture
• Redis broker: redis-cache
• HTTP delivery: webhooks
• SQL: postgres
• Email UI: react-email (only the template)

Completion criteria

• Named job; payload ID; idempotency key
• Retry/backoff/timeout/DLQ defined
• User HTTP does not block on slow I/O
• Worker with timeout and logs without PII
• Unique against duplicate