webhooks

Webhooks

HTTP as a notification, not as a screen. Inbound: provenance + fast ack. Outbound: retries in the worker (background-jobs). Error/status contract: api-design. Route authz: authentication-authorization.

Inbound (them → you)

1. Verify signature (HMAC / provider key) on the raw body. Do not parse JSON before it if the provider signs the bytes.
2. Reject an old timestamp (replay). Documented tolerance (e.g. 5 min).
3. Idempotency: event_id unique in Postgres. Duplicate → 200 without reapplying.
4. Respond 2xx fast. Persist + background-jobs for the heavy work. The provider timeout is short.
5. Fail closed if the secret is not in the env.

• Dedicated route, HTTPS, secret per environment. Not the same key in staging/prod.
• Extra auth (mTLS, IP allowlist) only if the provider and the repo already use it — the signature is the minimum.
• Do not log the raw payload (PII, tokens). Log event_id, type, result.

Outbound (you → them)

• URL registered by the resource owner; HTTPS. Not an open relay.
• Sign the body (timestamp.payload) with a rotatable secret; documented header (X-*-Signature).
• Retry backoff on 5xx/network; do not retry 4xx (dead URL → DLQ / disable endpoint).
• Stable versioned payload (v1). IDs, not internal graph (event-driven-architecture for the fact; here only the HTTP).
• Short timeout on the client. Job with an attempt ceiling (background-jobs).

Idempotency and order

• Inbound can arrive duplicated and out of order. Apply by the event_id and aggregate state, not blind “last JSON wins”.
• Outbound: the consumer must be idempotent; even so send idempotency-key / event_id.
• At-least-once is the default. Do not promise exactly-once with HTTP alone.

Security

• Compare the signature in constant time.
• Secret only in env/secret manager (production-readiness).
• SSRF: when calling a user URL, block private IPs if the product allows arbitrary destinations.
• Rotation: two secrets (current + previous) in inbound verification.

Testing and ops

• Fixture with a valid/invalid signature and an expired timestamp. Replay of the same event_id → 200 no-op.
• Staging endpoint with its own secret. Do not point the prod provider at localhost.
• Metric: deliveries, HMAC failures, outbound DLQ. Alarm on a spike of 401/400 on inbound.

Anti-patterns

• 200 before validating HMAC
• Process for 30s in the Stripe request
• Log req.body in full
• Retry 400 until you kill the partner
• GET webhook “easier”
• A single route /webhook without discriminating provider/secret

Integration with other skills

• Queue: background-jobs
• Domain event: event-driven-architecture
• HTTP: api-design
• Authz: authentication-authorization
• Payments: store skill if it already exists in the repo (play-billing ≠ Stripe webhook)

Completion criteria

• HMAC + timestamp on inbound; fast 2xx
• event_id unique; duplicate is a no-op
• Signed outbound, retry only transient, job with cap
• No PII in logs
• Secret per environment; HTTPS