Stop 02 · Networking Fundamentals for CloudTopics · 6/9

Load Balancers

In short

One server can only handle so many users. Put three behind a load balancer and users never notice which one answered, or that one of them just crashed.

Distributing incoming traffic across multiple resources to improve availability and handle scale — Layer 4 (transport) and Layer 7 (application) load balancers operate differently and serve different needs.

The mascot in a reflective vest waves cars towards three open toll booths, one booth closed with a cone.

Think of it like this

The toll plaza attendant

On a highway toll plaza, an attendant waves each car towards a free booth. If one booth's machine breaks, he stops sending cars there. Nobody waits behind a dead booth. A load balancer does this for web requests across servers.

Watch it happen

Round robin with a health check

Users

Load balancer

Server A

Server B

GET /products

1/6Every request goes to the load balancer's single public address, never straight to a server.

The big ideas

What you need to know

Layer 4 vs Layer 7

L4 balances by IP and port and is very fast. L7 reads HTTP, so it can route /api and /images to different servers.

Health checks

The balancer regularly calls something like /health. Servers that fail stop getting traffic automatically.

Pick an algorithm

Round robin takes turns, least connections picks the least busy server, and IP hash keeps a user on one server.

TLS often ends here

Many setups decrypt HTTPS at the load balancer, so certificates live in one place instead of on every server.

See it in code

Walk through the code

nginx

1upstream app_servers {2    least_conn;3    server 10.0.1.11:3000;4    server 10.0.1.12:3000;5    server 10.0.1.13:3000 backup;6}7 8server {9    listen 80;10    location / {11        proxy_pass http://app_servers;12    }13}

1/3Define a pool of servers and pick the 'least connections' algorithm.

Cloud providers offer managed load balancers too. The ideas are the same: a pool of targets, a balancing rule and health checks.

Save yourself the pain

Common mistakes

Oops

Storing user sessions in each server's memory, so users get logged out when the balancer switches servers.

Do this instead

Keep servers stateless. Store sessions in Redis or a database that every server can reach.

Oops

Pointing the health check at the home page, which passes even when the database is down.

Do this instead

Use a /health endpoint that checks what the app really needs, like its database connection.

Your mission

Balance two tiny servers

~30 min

Quick check

You need /api requests to go to one group of servers and /static to another. Which kind of load balancer do you need?

Go deeper

1Learn the difference between Layer 4 and Layer 7 load balancing.
2Explore your cloud provider's load balancer offering (e.g. AWS ALB/NLB) conceptually.
3Understand health checks — how a load balancer knows to stop routing traffic to an unhealthy resource.