saas-rate-limiting-and-quotas

SaaS Rate Limiting and Quotas

Acknowledgement: Shared by Peter Bamuhigire, techguypeter.com, +256 784 464178.

Use When

Designing per-tenant rate limits for a pooled or mixed-mode SaaS to prevent noisy-neighbor degradation.
Adding plan-aware quotas (Pro gets 100k API calls/month; Enterprise gets unlimited).
Implementing fair-queueing so a single heavy tenant doesn't starve the shared worker pool.
Surfacing quota usage to the tenant (X-RateLimit-Remaining, in-product usage panel).
Coordinating limits across multiple enforcement points (edge, gateway, service, DB pool, queue).
Designing the response contract when limits are hit (status codes, headers, error body).

Do Not Use When

The task is general application performance — use frontend-performance / backend perf skills.
The task is the entitlements engine itself (what limits exist per plan) — use saas-entitlements-and-plan-gating. This skill is the enforcement runtime.
The task is queue-based fair scheduling alone — use queue-specific skills.

Required Inputs

Plan × limit catalogue from saas-entitlements-and-plan-gating.
Deployment model (pool vs mixed vs silo) from saas-deployment-models — affects where enforcement lives.
Tenant-count and request-volume estimates — drives algorithm + storage choice.
Existing edge/gateway/proxy in front of the app (Cloudflare, ALB, Nginx, Envoy, Kong).

Workflow

Read this SKILL.md.
Inventory what to limit (§2) — request-rate, concurrency, monthly-quota, per-resource quota.
Pick the algorithm per limit type (§3) — token bucket / sliding window / fixed window / leaky bucket.
Pick the enforcement layer (§4) — edge / gateway / app / DB / queue.
Pick the storage (§5) — Redis is default; vendor-managed for huge scale.
Design the response contract (§6) — status codes, headers, body, retry-after.
Add fair-queueing for shared resources (§7) — heavy tenants don't starve light ones.
Surface usage to the tenant (§8) — UI panel + API endpoint.
Apply anti-patterns (§9).

Quality Standards

Limits enforced before the resource is consumed (gate first, work second).
Atomic check-and-increment — no race condition that allows over-limit slip-through.
Limit decisions cached close to the request — typically Redis, sub-millisecond.
Response includes Retry-After, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset.
Per-tenant fair-queueing in shared async workers prevents one tenant starving others.
Every limit hit emits an event for analytics and lifecycle email (usage.approaching_limit / usage.limit_hit).

Anti-Patterns

Per-IP rate limiting only — multiple tenants behind a corporate NAT share IP; one bad tenant penalises others.
Per-API-key rate limiting only — tenant has multiple keys; aggregate limit isn't enforced.
SELECT count + INSERT race condition — between check and increment, multiple requests slip through.
App-only enforcement when edge could shed the load — wastes resources processing requests destined to be 429'd.
No fair-queueing — one big tenant's batch report dominates the worker pool and breaks SLA for everyone else.
No tenant-facing usage panel — customers have no idea why they're being limited.

Outputs

Limit inventory + algorithm + enforcement-layer per limit.
Storage architecture (Redis cluster / vendor).
Response contract (status, headers, body).
Fair-queueing design for async work.
Tenant-facing usage panel + API.

Evidence Produced

Category	Artifact	Format	Example
Architecture	Rate-limit + quota inventory	Markdown table	`docs/saas/rate-limits.md`
Release evidence	Response contract spec	Markdown doc with examples	`docs/saas/rate-limit-response.md`
Operability	Limit hit + denial dashboard	Dashboard link	`docs/saas/quota-dashboard.md`

References

references/algorithms.md — token bucket, sliding window, leaky bucket, fixed window — when each fits.
references/fair-queueing-patterns.md — per-tenant queues, weighted fair queueing, DRR.
references/redis-implementation.md — atomic Lua scripts for token bucket + sliding window.
Companion: saas-entitlements-and-plan-gating, saas-deployment-models, multi-tenant-saas-architecture, observability-monitoring, microservices-resilience.

§1 The Four Things to Limit

Type	Example	Time horizon
Request rate	100 req/sec per tenant	seconds
Concurrency	Max 50 in-flight requests per tenant	instantaneous
Period quota	100k API calls per month	rolling month
Per-resource quota	10 active workflows per tenant	always-on (entitlement)

Most SaaS need all four.

§2 The Limit Inventory

For every SaaS, list:

Limit	Type	Per-plan values	Enforcement layer	Storage
API requests	rate	free 10/s, pro 100/s, enterprise 1000/s	gateway	Redis
API requests	period	free 1k/mo, pro 100k/mo, enterprise unlimited	gateway	Redis + DB
Concurrent requests	concurrency	free 5, pro 50, enterprise 500	gateway	Redis
Webhook deliveries	rate	100/s per tenant	worker	Redis
DB connections	concurrency	per-tenant pool size	DB pool	PgBouncer / pg pooler
Heavy reports	concurrency	free 1, pro 5, enterprise 20	worker	Redis
AI token budget	period	per AI plan	service	DB
Login attempts	rate	5/min per IP+email	app	Redis

§3 Algorithm Choice

Token Bucket

Best for: request-rate with burst tolerance.
Refills at fixed rate; allows burst up to bucket size.
Use for: API request rate limits.

Sliding Window

Best for: strict rolling-window enforcement, smooths over burst boundaries.
Tracks request timestamps; counts in window.
Use for: per-month quota with rolling window.

Fixed Window

Best for: simple, low-cost; OK if boundary-burst is acceptable.
Reset at period boundary.
Use for: per-day / per-month counters where boundary burst is benign.

Leaky Bucket

Best for: smoothing — work drains at fixed rate.
Use for: queue-based work scheduling.

Concurrency Limit

Best for: in-flight request gate.
Atomic INCR on entry; DECR on exit; reject if > limit.
Use for: max concurrent requests, max concurrent reports.

Default recommendation: token bucket for rate, sliding window for period quotas, atomic counter for concurrency.

§4 Enforcement Layer

Edge / CDN (Cloudflare, Fastly, CloudFront)
  ↓
Gateway (Kong, Envoy, Tyk, AWS API Gateway, ALB)
  ↓
Application / service
  ↓
Database connection pool (PgBouncer / pgpool / RDS Proxy)
  ↓
Async worker (with per-tenant queues)

Enforce as close to the edge as possible. Reasons:

The closer to the edge, the less wasted work.
Edge can return 429 in < 5ms; app can't.
Edge integrates with WAF and DDoS protection.

Pattern:

Edge: coarse rate limits (per-IP DDoS protection, simple plan-tier global rate).
Gateway: per-tenant rate + concurrency limits (most enforcement).
App: per-feature-specific limits that need business logic.
DB pool: per-tenant connection cap.
Worker: per-tenant queue partitioning + fair-queueing.

§5 Storage

Storage	Latency	Capacity	Use case
Redis (single)	< 1ms	up to ~50k ops/sec	Default for most SaaS
Redis cluster	< 1ms	millions ops/sec	At scale
Cloudflare Rate Limiting Rules / WAF	< 1ms	unlimited (their infra)	DDoS + simple per-IP/per-route
AWS API Gateway throttling	< 5ms	scaled	If on API Gateway already
In-memory (per-process)	sub-µs	scoped to one process	NOT for multi-instance; only for local rate limits
DB (Postgres counter)	1-5ms	ops/sec	For period quotas reconciled periodically

Default: Redis (managed: ElastiCache / Upstash / Redis Cloud) with atomic Lua scripts.

Atomic token-bucket in Redis (Lua)

-- KEYS[1] = bucket key (e.g., "rl:tenant:123")
-- ARGV[1] = capacity, ARGV[2] = refill_per_second, ARGV[3] = now_ms, ARGV[4] = cost
local capacity = tonumber(ARGV[1])
local refill_rate = tonumber(ARGV[2])
local now_ms = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])

local b = redis.call('HMGET', KEYS[1], 'tokens', 'last_ms')
local tokens = tonumber(b[1]) or capacity
local last_ms = tonumber(b[2]) or now_ms

local elapsed = math.max(0, now_ms - last_ms)
tokens = math.min(capacity, tokens + (elapsed / 1000) * refill_rate)

local allowed = 0
if tokens >= cost then
    tokens = tokens - cost
    allowed = 1
end

redis.call('HMSET', KEYS[1], 'tokens', tokens, 'last_ms', now_ms)
redis.call('PEXPIRE', KEYS[1], math.ceil(capacity / refill_rate * 1000 * 2))
return {allowed, tokens}

§6 Response Contract

HTTP 429 Too Many Requests for rate / concurrency violations.

HTTP 402 Payment Required or HTTP 403 Forbidden for plan-quota violations (the user needs to upgrade, not just wait).

Headers (always include):

X-RateLimit-Limit: 100 — total in window.
X-RateLimit-Remaining: 0 — current remaining.
X-RateLimit-Reset: 1746576000 — unix timestamp when window resets.
Retry-After: 30 — seconds to wait (only on 429).

Body:

{
  "error": {
    "code": "RATE_LIMIT_EXCEEDED",
    "message": "You have exceeded 100 requests/minute. Retry in 30 seconds.",
    "retry_after_seconds": 30,
    "limit_type": "api_rate_per_minute",
    "tenant_id": "ten_123"
  }
}

For quota (period or plan):

{
  "error": {
    "code": "QUOTA_EXCEEDED",
    "message": "Monthly API call limit of 100,000 reached. Upgrade your plan to continue.",
    "limit_type": "api_calls_monthly",
    "current_usage": 100000,
    "limit_value": 100000,
    "upgrade_url": "https://app.example.com/billing/upgrade?context=api_quota"
  }
}

§7 Fair-Queueing for Shared Workers

Without fair-queueing, one tenant's 10,000-job batch saturates the worker pool. Patterns:

Per-tenant queues + round-robin dispatcher

Producers → push to per-tenant queue (e.g., Redis Lists keyed `q:tenant:123`)
Dispatcher → round-robins across active tenant queues; pushes one job to shared worker pool per turn
Workers → consume from shared pool

Weighted fair queueing

Each tenant has a weight (often weight = ceil(plan_seats / 10) or weight = tier_factor).
Dispatcher gives bigger tenants more turns per cycle; smaller tenants never starve.

Per-tenant concurrency cap

Per-tenant counter; max concurrent jobs = function of plan.
Heavy tenant's overflow waits in their own queue; doesn't enter shared pool.

Engines that help

Sidekiq Pro/Enterprise: concurrency groups, rate limits.
Temporal: per-namespace concurrency limits.
Celery: per-queue routing + concurrency.
Custom: Redis-backed per-tenant queues + a dispatcher process.

§8 Surface Usage to the Tenant

Tenants who don't know they're being limited are angry tenants.

In-product panel: "API calls this month: 87,432 / 100,000 (87%)".
API endpoint: GET /api/v1/me/quota returns current usage + limits per metered dimension.
Approaching-limit emails: 70% → soft, 90% → strong, 100% → block-message email (see saas-lifecycle-email-orchestration).
Per-request headers on every API call: X-RateLimit-*.
Webhook on threshold: for enterprise tenants, send a webhook when they cross 80%/90%/100% so their ops team knows.

§9 Anti-Patterns

Per-IP rate limit only. Corporate NAT means one bad employee blocks the whole company; or one bad cluster blocks one customer.
Synchronous quota check via SQL SELECT COUNT(*) per request. Slow + race-prone. Fix: Redis counter.
No fair-queueing in shared workers. One tenant's batch destroys SLA for the rest.
No 429 + Retry-After. Clients hammer the API blindly.
No quota panel for the tenant. Mystery throttling = support tickets + churn.
Quota counters never reset on period rollover. Accidental permanent block.
Burst capacity too low. Cron-driven clients submit 1k jobs at the top of the hour; legitimate burst is rate-limited.
No emission of usage.approaching_limit event. Lifecycle email can't trigger; expansion revenue lost.

§10 Read Next

saas-entitlements-and-plan-gating — defines what limits exist per plan.
saas-deployment-models — drives where enforcement lives.
multi-tenant-saas-architecture — tenant context drives the bucket keys.
microservices-resilience — circuit breakers, bulkheads (complementary to rate limits).
observability-monitoring — surface limits + denials in dashboards.
saas-lifecycle-email-orchestration — emit approaching-limit events for upgrade triggers.

AI Quotas Addendum

AI features introduce distinct quota dimensions beyond generic API rate limits:

AI dimension	Type	Per-plan example	Enforcement layer
AI tokens / minute	rate	Free 1k/m, Pro 100k/m, Enterprise 1M/m	LLM gateway (Redis token bucket)
AI generations / day	period	Free 50, Starter 500, Pro 5k	LLM gateway (sliding window)
AI tokens / month	period	Free 100k, Pro 20M	LLM gateway + DB
AI USD / month	period (currency)	Free $0, Pro $50, Enterprise custom	LLM gateway
Concurrent agent sessions	concurrency	Free 0, Business 5, Enterprise 50	gateway + worker
Agent steps per session	per-resource	Free 0, Business 20, Enterprise 50	agent runtime
KB pages ingested	period	Starter 500, Pro 10k	KB service
Embedding-jobs / hour per tenant	rate	tier-specific	ingestion dispatcher

Where enforced:

LLM gateway is the chokepoint for token, generation, USD caps.
Agent runtime enforces step caps and concurrent agent sessions.
KB service enforces ingestion-rate and total-pages caps.

Agent Quotas (Detailed)

Agentic features need a richer quota set than single-shot AI. The dimensions:

Quota	Algorithm	Default — Pro	Default — Enterprise	Enforcement
Concurrent agent sessions per tenant	concurrency counter (atomic INCR/DECR in Redis)	3	20	Runtime task-create handler
Agent steps per tenant per day	sliding window	500	10,000	Runtime step-enter handler
Agent tasks per user per hour	token bucket	20	100	Runtime task-create handler
Long-running tasks (wallclock > 5 min) in-flight per tenant	concurrency counter	1	10	Runtime task-create handler
Per-task wallclock cap	budget (`ai-agent-cost-and-step-budgets`)	5 min	30 min	Runtime budget check
Per-task cost cap (USD)	budget	$1	$10	Runtime budget check
Standing approvals invocations per tenant per day	counter	100	unlimited	Approval runtime

Concurrency Counter Pattern

def claim_agent_session(tenant_id: int, max_concurrent: int) -> bool:
    key = f"agent:concurrent:{tenant_id}"
    val = redis.incr(key)
    redis.expire(key, 86400, nx=True)
    if val > max_concurrent:
        redis.decr(key)
        return False
    return True

def release_agent_session(tenant_id: int):
    redis.decr(f"agent:concurrent:{tenant_id}")

release is called on every task terminal transition (COMPLETED, FAILED, KILLED, BUDGET_EXCEEDED, ABANDONED). A janitor reconciles drift by counting actual in-flight tasks vs counter every 5 minutes.

Response on Limit Hit

{
  "error": "agent_concurrency_limit",
  "code": "AGENT_CONCURRENCY_LIMIT",
  "current": 3,
  "limit": 3,
  "plan": "pro",
  "retry_after_seconds": 60,
  "upgrade_url": "/billing/upgrade?plan=enterprise&context=agent_concurrency"
}

The user sees: "You've reached your concurrent-agent limit (3). Wait for an active task to finish, or upgrade for 20 concurrent."

Cross-Plane Quotas

Quotas are declared in ai-entitlements-and-feature-gating (catalogue) and enforced in the agent runtime (this skill's runtime layer). The two skills hold the contract:

Entitlements: which keys exist, per-plan values, per-tenant overrides.
Rate limiting: which algorithms enforce them, which storage, which response shape.

Cross-references:

ai-agent-runtime-architecture — where agent quotas are enforced.
ai-agent-cost-and-step-budgets — per-task budgets (the inner quota layer).
ai-entitlements-and-feature-gating — agent quota catalogue.
ai-model-gateway — implements per-tenant token+USD caps.
ai-cost-per-tenant-attribution — soft/hard ceilings with comms.
ai-rag-multi-tenant — KB-side caps.
ai-on-saas-architecture — overall architecture.

Quota Breach → SLA Implications (Enhancement)

Not every quota breach is an SLA event. The platform commits to enforcement, not to unlimited capacity. The mapping below classifies each breach so the SLA-credit pipeline (ai-agent-sla-credit-automation) can route correctly.

Classification matrix

Breach pattern	Customer-observable?	SLA-impact?	Treatment
Tenant hits committed monthly quota (e.g., 50k tasks/Pro)	Yes (429 / "limit reached")	No — expected guard	UX: clear comms + upsell prompt; no credit.
Tenant hits a soft ceiling we set (cost protection)	Yes (slowed or refused)	No — platform self-protection	UX: notify tenant; document in account.
Tenant blocked because a neighbor exhausted shared pool (noisy-neighbor)	Yes (429 on tenant who didn't breach)	Yes — eligible breach	Pool isolation fix; credit the affected tenant.
Platform-wide quota mis-configuration blocking tenants under their committed cap	Yes	Yes — eligible breach	Mass-credit via incident pipeline.
Tenant blocked due to negotiated bespoke cap they signed	Yes	No — customer-caused	No credit; comms cite the contract.
Burst rate-limit fires during agent task → task aborts	Sometimes	Conditional — eligible if tier commits resolution rate AND breach traces to platform side
Burst rate-limit fires because tenant launched 10k concurrent tasks	Yes	No — customer-caused	Document in audit.

Emitted classification

When a quota breach terminates a billable event (task abort, refused request), the limiter emits:

{
  "event": "quota.breach.classified",
  "tenant_id": "...",
  "quota_key": "tasks_per_month | tokens_per_min | concurrent_tasks | ...",
  "limit": 50000,
  "consumed": 50001,
  "scope": "committed | soft | shared_pool",
  "cause": "tenant_traffic | neighbor_traffic | platform_misconfig | tenant_bespoke_cap",
  "sla_classification": "expected_guard | eligible_breach | customer_caused",
  "task_id": "...",
  "timestamp": "..."
}

The SLA-credit pipeline subscribes to events with sla_classification='eligible_breach'.

Pool isolation principle

Shared-pool quotas (rate-limit buckets that several tenants draw from) are the most common source of accidental eligible breaches. Pool design should prefer per-tenant buckets for any quota that gates a committed SLA dimension. Where shared pools must remain, the SLA-class table (ai-agent-sla-and-commitments) should explicitly call out the elevated breach probability.

Cross-links

ai-agent-sla-credit-automation/references/eligibility-rules.md — consumes the sla_classification field.
ai-agent-sla-and-commitments — defines which quotas map to committed dimensions.
ai-agent-cost-and-step-budgets — inner quota layer with its own SLA handoff.