Rate Limiting: Buckets, Windows and the Keys That Actually Matter

Rate limiting is the seatbelt of API design: invisible until the crash, at which point it's the only thing that matters. And "the crash" takes more forms than people plan for — the credential-stuffing bot, the partner's runaway retry loop (their bug, your outage), the scraper, the customer's intern with a while-loop, and occasionally your own mobile app after a bad release. One discipline defends against all of them, and it's cheap. Let's design it properly: algorithms, keys, responses, and the operational layer nobody documents.

Algorithms: the honest three

• Fixed window — a counter per key per minute (INCR + EXPIRE). Trivial, memory-cheap, and famously allows 2× bursts straddling window edges (100 at 11:59:59 + 100 at 12:00:01). For most limits, honestly: fine. The boundary burst rarely matters at application scale.
• Sliding window — either the log version (a sorted set of timestamps, exact but memory-proportional-to-traffic) or the elegant counter approximation: weight the previous window's count by its remaining overlap. One extra GET, smooth limiting, no burst artifact — my default for public APIs.
• Token bucket — capacity + refill rate, allowing controlled bursts above the sustained rate ("120 burst, 60/min sustained"). This matches how legitimate clients actually behave — bursty page loads, batch syncs — which is why it's what I use where UX matters. Implement atomically in Redis with a few lines of Lua (read-modify-write on two keys must not race).

Framework note for the Laravel crowd: RateLimiter gives you fixed-window per named limiter out of the box, and it's the right 80% answer — reach for hand-rolled buckets only where burst-shaping genuinely matters.

Keys: what you limit matters more than how

The algorithm debate is fun; the key design is where limits succeed or fail. Real systems layer several:

Layer 1  Per IP, coarse       edge/WAF — bot-storm insurance; generous, because
                              NAT and CGNAT put thousands of humans behind one IP
Layer 2  Per credential       the workhorse: per API key / user / tenant —
                              aligned to whatever you bill or trust
Layer 3  Per credential+route expensive endpoints get their own budget:
                              search ≠ status-check; exports ≠ everything
Layer 4  Per resource         the sneaky one: 5 OTP requests per PHONE NUMBER,
                              3 resets per ACCOUNT — abuse limits keyed to the
                              victim, not the attacker (who rotates IPs freely)

That fourth layer is the one audits find missing: login attempts limited per-IP only (attacker: hello, botnet), OTP sends unlimited per target (attacker: SMS-bombs your own users at your expense). Abuse limits key on the resource being abused. Also deserving separate buckets: auth/reconnect endpoints (so a stampede degrades gracefully) and unauthenticated routes generally (strict by default — anonymous traffic has no reputation to lose).

The response: a contract, not a door slam

HTTP/1.1 429 Too Many Requests
Retry-After: 23
RateLimit-Limit: 100
RateLimit-Remaining: 0
RateLimit-Reset: 23

{ "type": ".../problems/rate-limited", "title": "Rate limit exceeded",
  "detail": "100 requests per minute per API key.", "retry_after": 23 }

Always 429 (never 403 — clients branch on this, per the API checklist), always Retry-After (well-behaved clients honor it; you're literally scheduling your own load), and rate headers on successful responses too, so clients can self-throttle before hitting the wall. Document limits publicly — secret limits don't prevent abuse, they prevent partners from building compliant clients.

Operational wisdom, bought at retail price

• Fail open, loudly. Redis down should mean "limits off + alert screaming", not "site down". The limiter protects availability; it must never become the availability problem. (Exception: abuse-critical limits like OTP may justifiably fail closed — decide per limit, on purpose.)
• Launch in shadow mode. Count and log would-be 429s for two weeks before enforcing. Every rollout skips this once, 429s their biggest customer's integration, and never skips it again.
• Monitor the limiter itself: 429 rate per key (top-ten list = your abuse dashboard and your "who needs a bigger plan" sales lead list), plus limiter latency — it's on every request's hot path.
• Tiers belong to config, not code: per-plan multipliers resolved at request time, so sales can sell a higher limit without a deploy.
• Degrade before denying where product allows: past the soft limit, serve cached/stale responses or queue the work; hard-deny only past the abuse threshold. The best rate limit is one legitimate users never feel.

The whole apparatus — buckets, layers, headers, shadow mode — fits in a few hundred lines against infrastructure you already run. What it buys is disproportionate: the difference between "one partner's retry loop" being a Tuesday log line versus a postmortem. Seatbelts, as noted, are cheap right up until they're priceless.

API getting hammered — or about to be launched into a world that hammers? Rate-limit design reviews pair beautifully with the API checklist; bring your traffic graphs.