Stop 20 · System DesignTopics · 1/15

Scalability Basics

In short

Your app works for 100 users. Will it survive 1 lakh during an IPL final? Scalability means handling more load by adding resources, without rewriting everything.

The general practice of designing a system so it keeps working well as load (users, data, traffic) grows — a core concern once you move beyond a single-server hobby project.

The purple blob pours chai into a long row of glasses for a big crowd of cricket fans.

Think of it like this

The chai stall on match day

A chai stall serves 50 cups an hour. On match day, 500 people show up. The owner can buy a bigger kettle, hire two helpers, or keep boiled milk ready. Each fix helps, but the slowest step, maybe the single cash counter, still decides how fast the queue moves.

Watch it happen

Where load piles up

Users

Balancer

App x3

Cache

Database

1/5Traffic grows tenfold. Every request hits the load balancer first.

The big ideas

What you need to know

Find the bottleneck

Measure first. Adding app servers does nothing if the database is the part stuck at 100 percent.

Stateless servers scale

Keep sessions and files out of app memory and disk. Then any server can handle any request.

Cache and queue

Cache repeated reads. Push slow work like emails and reports to a background queue.

Latency vs throughput

Throughput is requests per second. Latency is how long each takes. Good scaling grows throughput without hurting latency.

See it in code

Walk through the code

bash

1# Hit your API with 100 connections for 30 seconds2npx autocannon -c 100 -d 30 http://localhost:3000/products3 4# While it runs, watch where the pressure is (separate terminals):5top                                                 # app CPU6docker stats                                        # container CPU and memory7psql -c "SELECT count(*) FROM pg_stat_activity;"    # DB connections8 9# Then double the connections and compare latency10npx autocannon -c 200 -d 30 http://localhost:3000/products

1/3Simulate 100 users hammering one endpoint for 30 seconds.

autocannon reports requests per second and latency percentiles. If doubling connections doubles latency but not throughput, something is saturated.

Save yourself the pain

Common mistakes

Oops

Scaling before measuring anything.

Do this instead

You may add five servers when one slow query was the real problem. Load test and profile, then scale the bottleneck.

Oops

Keeping sessions in server memory, then adding more servers.

Do this instead

Users bounce between servers and get logged out. Move sessions to Redis or use stateless tokens.

Your mission

Find your bottleneck

~40 min

Quick check

App servers sit at 30% CPU while the database is at 100%. You add three more app servers. What happens?

Go deeper

1Identify the current bottleneck (CPU, memory, database, network) in a project you've built.
2Learn the difference between scaling for more users vs scaling for more data.
3Understand premature scaling is a real cost — most projects don't need this on day one.