Stop 20 · System DesignTopics · 13/15

Rate Limiting Algorithms

In short

Without rate limits, one buggy script or attacker can flood your API and starve everyone else. Each algorithm trades accuracy, memory and burst-friendliness differently, and interviewers love asking which.

In a system-design context: comparing fixed window, sliding window, token bucket and leaky bucket algorithms for their trade-offs in fairness, burst handling and implementation complexity.

The purple blob hands tokens from a slowly filling glass jar to people waiting in line.

Think of it like this

The token jar

A jar gets one new token every few seconds, up to a fixed amount. Each person in line must take a token to be served. Jar empty? You wait. Quiet for a while? Tokens pile up, so a small group can be served at once. That's the token bucket.

Watch it happen

Token bucket: capacity 5, refill 1 per second

1/5The bucket starts full with 5 tokens. It refills by 1 token every second, up to 5.

The big ideas

What you need to know

Token bucket

Refills at a steady rate and allows short bursts up to its capacity. Widely used in API gateways.

Leaky bucket

Requests queue up and leave at a fixed rate, like water dripping from a hole. Smooth output, no bursts.

Fixed window

Count requests per calendar minute. Simple, but allows double bursts around the minute boundary.

Sliding window

Weigh the current and previous windows, or keep timestamps, to avoid edge bursts. Costs a bit more memory.

See it in code

Walk through the code

1type Bucket = { tokens: number; last: number };2const buckets = new Map<string, Bucket>();3const CAPACITY = 5;4const REFILL_PER_SEC = 1;5 6export function allow(clientId: string): boolean {7  const now = Date.now() / 1000;8  const b = buckets.get(clientId) ?? { tokens: CAPACITY, last: now };9  b.tokens = Math.min(CAPACITY, b.tokens + (now - b.last) * REFILL_PER_SEC);10  b.last = now;11  buckets.set(clientId, b);12  if (b.tokens < 1) return false;13  b.tokens -= 1;14  return true;15}

1/3A new client starts with a full bucket.

No timers needed: tokens are topped up lazily from the time since the last request. For many servers, the same logic moves into Redis.

Save yourself the pain

Common mistakes

Oops

Rate limiting only by IP address.

Do this instead

Whole hostels and offices share one IP. Limit by API key or user ID where you can, with IP as a fallback.

Oops

Returning 500 or silently dropping limited requests.

Do this instead

Return 429 Too Many Requests with a Retry-After header, so well-behaved clients slow down.

Your mission

Compare two algorithms

~30 min

Quick check

A fixed window allows 100 requests per minute. A client sends 100 at 12:00:59 and 100 at 12:01:00. What happens?

Go deeper

1Implement both a fixed window and a token bucket rate limiter and compare their behavior.
2Learn why token bucket allows controlled bursts while fixed window can allow a burst right at window boundaries.
3Be ready to design one of these from memory — it's a common system design interview question.