Rate limiting

Rate Limiting

Instructions

Rate limiting protects the backend from accidental stampedes and intentional abuse. Mobile adds two wrinkles: many users share one IP (carrier NAT), and a single user runs multiple devices. Scope limits accordingly.

1. Algorithm Choice

• Token bucket: allows short bursts up to bucket size; refills at a steady rate. Best default.
• Leaky bucket: smooths to a constant rate; harsher on bursts.
• Fixed window: easy but lets 2x burst at boundary.
• Sliding window log: accurate but memory-heavy.

Token bucket is the right default for APIs consumed by mobile.

Parameters:

• capacity (burst): e.g., 60 tokens.
• refill_rate: e.g., 1 token/second.

2. Redis Token Bucket

Atomic Lua script avoids races:

-- KEYS[1] = bucket key
-- ARGV[1] = now_ms, ARGV[2] = capacity, ARGV[3] = refill_per_sec, ARGV[4] = cost
local now = tonumber(ARGV[1])
local cap = tonumber(ARGV[2])
local rate = tonumber(ARGV[3])
local cost = tonumber(ARGV[4])
local bucket = redis.call("HMGET", KEYS[1], "tokens", "ts")
local tokens = tonumber(bucket[1]) or cap
local ts     = tonumber(bucket[2]) or now
local delta  = (now - ts) / 1000.0
tokens = math.min(cap, tokens + delta * rate)
local allowed = 0
if tokens >= cost then
  tokens = tokens - cost
  allowed = 1
end
redis.call("HMSET", KEYS[1], "tokens", tokens, "ts", now)
redis.call("PEXPIRE", KEYS[1], 2 * 1000 * cap / rate)
return { allowed, tokens }

3. Scoping

Apply layered limits; the strictest wins.

Scope	Example
Global	100k req/s total
Per IP	60/min unauth, 600/min auth
Per user_id	60/min for writes, 300/min for reads
Per device_id	600/min total
Per endpoint	/auth/login 5/min per IP + per account

Never rate-limit solely on IP -- carrier NAT makes thousands of users look like one. Combine IP + device_id for public endpoints.

4. Response Shape

On block, return 429 RATE_LIMITED with the shared error envelope and the standard headers:

HTTP/1.1 429 Too Many Requests
Retry-After: 2
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1713522660

{
  "error": {
    "code": "RATE_LIMITED",
    "message": "Too many requests",
    "details": { "retry_after_ms": 1800 },
    "request_id": "01HX7..."
  }
}

Clients honor retry_after_ms with jitter before retrying; do not retry 429 without honoring it.

5. Per-Endpoint Budgets

Budgets per risk profile:

• Auth: strict. /token 10/min/IP + 5/min/account. Lockout after repeated failures.
• Writes: moderate. Enough to finish normal user work, not a scraping run.
• Reads: lenient. Let users paginate and refresh without grief.
• Expensive reads (search, aggregations): their own tighter bucket.

Document every budget so mobile teams can calibrate retry logic.

6. Client Cooperation

• Clients batch where possible (e.g., send 20 events per /analytics call, not 20 requests).
• On 429, apply full-jitter exponential backoff, not a fixed sleep.
• Surface an unobtrusive "too fast" state if the user hammers a button; disable re-taps for 500 ms.

7. Abuse Signals

Rate limiting is the floor; layer these:

• Velocity checks: N failed logins per account in 10 min -> temporary account lockout or step-up challenge.
• Reputation scoring: known-bad IPs / ASNs (VPN farms, datacenter ranges) get tighter limits for public endpoints.
• Device attestation: prefer traffic from App Attest / Play Integrity-verified devices; unverified traffic gets tighter limits.
• Anomaly detection: sudden 100x spike from one device -> throttle harder and alert.

8. Distributed Enforcement

• Centralized (Redis) counters are simplest; add a local L1 cache with 100–500 ms TTL to cut Redis load.
• For edge-enforced limits, use the CDN's native rate limiting (Cloudflare, Fastly, CloudFront + WAF). Fast, protects the origin, but lacks per-user keys unless you forward the user id via a signed header.
• Overload: if Redis is down, fail open for low-risk endpoints (serve requests) and fail closed for auth/payment (reject). Document the policy.

9. Example (Node/TS Express Middleware)

export function limit(scope: string, cost = 1) {
  return async (req, res, next) => {
    const key = `rl:${scope}:${req.user?.id ?? req.device?.id ?? req.ip}`;
    const [allowed, remaining] = await redis.evalsha(SCRIPT_SHA, 1, key,
      Date.now().toString(), "60", "1", String(cost));
    res.setHeader("X-RateLimit-Limit", "60");
    res.setHeader("X-RateLimit-Remaining", String(Math.floor(remaining)));
    if (!allowed) {
      res.setHeader("Retry-After", "2");
      return res.status(429).json({ error: { code: "RATE_LIMITED",
        message: "Too many requests", details: { retry_after_ms: 2000 },
        request_id: req.id } });
    }
    next();
  };
}

10. Observability

• Block rate per scope / endpoint.
• 99p token-bucket remaining (near zero means limits too tight for normal users).
• Repeat offenders (same key hitting 429 > 100 times/min) for investigation.

Checklist

• Token bucket chosen; capacity and refill documented per scope.
• Layered scopes: global, IP, user_id, device_id, endpoint.
• 429 responses include envelope + Retry-After + rate-limit headers.
• Client honors retry_after_ms with jitter.
• Auth endpoints carry stricter limits + lockouts.
• Reputation and attestation signals reduce limits for risky traffic.
• Fail-open / fail-closed policy on Redis outage documented.
• Block-rate and repeat-offender dashboards in place.