Stop 20 · System DesignTopics · 3/15

Load Balancing

In short

In system design interviews, 'add a load balancer' is step one. The real questions come next: which layer, which algorithm, what about sessions, and what if the balancer itself dies?

In a system-design context: the broader strategy of distributing traffic across multiple servers as a prerequisite for horizontal scaling, plus the algorithms and health-checking involved.

The purple blob waves cars into open toll booths, with one booth closed behind a cone.

Think of it like this

The highway toll plaza

A toll plaza has many booths. Staff wave cars to the shortest lines, FASTag lanes get their own booths, and a broken booth gets a cone in front of it. The plaza also has backup power, because if it shuts, the whole highway stops.

Watch it happen

Balancing across regions and servers

GeoDNS

LB Mumbai

LB Singapore

App 1

App 2

App 3

1/4DNS returns the address of the nearest healthy region's balancer, so users in India go to Mumbai.

The big ideas

What you need to know

L4 vs L7

L4 forwards TCP by IP and port, very fast. L7 understands HTTP, so it can route /api and /images differently.

Sticky sessions, sparingly

Pinning a user to one server hides state problems and unbalances load. Prefer stateless servers.

No single point of failure

Run balancers in pairs or use a managed one spread across zones. One balancer is one big failure point.

Global balancing

DNS-based or anycast routing sends users to the nearest healthy region.

See it in code

Walk through the code

haproxy

1frontend web2  bind :443 ssl crt /etc/haproxy/certs/shop.pem3  acl is_api path_beg /api/4  use_backend api if is_api5  default_backend static6 7backend api8  balance leastconn9  option httpchk GET /health/ready10  server api1 10.0.1.11:3000 check11  server api2 10.0.1.12:3000 check12 13backend static14  balance roundrobin15  server web1 10.0.2.21:8080 check

1/4Terminate HTTPS at the balancer with the site's certificate.

HAProxy is a popular open-source L7 balancer. Managed options like AWS ALB or Google Cloud Load Balancing do the same with a UI.

Save yourself the pain

Common mistakes

Oops

Using sticky sessions to 'fix' users getting logged out.

Do this instead

It works until that server dies and everyone pinned to it is logged out anyway. Store sessions centrally.

Oops

Drawing one load balancer box in your design and moving on.

Do this instead

Interviewers will ask what happens when it fails. Mention redundant balancers across zones or a managed service.

Your mission

Route by path

~35 min

Quick check

You need /video requests to go to different servers than /api requests. Which balancer can do that?

Go deeper

1Diagram a load-balanced architecture with 3 backend instances behind one load balancer.
2Learn how a load balancer detects and routes around an unhealthy instance.
3Understand why load balancers themselves can become a single point of failure without redundancy.