Stop 11 · Cloud Architecture PatternsTopics · 8/9

Designing for Scalability

In short

Your app works for 100 users. Will it work for 100,000 when your reel goes viral? Scalable design means growth is mostly a matter of adding more machines, not rewriting code.

Building systems that can grow to handle increased load — horizontal scaling (more instances) is generally preferred over vertical scaling (bigger instances) for its higher ceiling and better fault tolerance.

The blob character sets up a second snack stall beside a crowded one at a fair.

Think of it like this

Another stall at the mela

The pani puri stall at the village fair has a crowd of fifty. You can't make one vendor's hands move faster forever. So you set up a second identical stall next to it, then a third. Same recipe, more counters.

Watch it happen

Scaling out as CPU climbs

web-1

web-2

1/5Two identical web servers behind a load balancer, averaging 30% CPU. Target is 50%.

The big ideas

What you need to know

Horizontal first

Adding machines (scale out) has a much higher ceiling than buying a bigger one (scale up), and survives failures.

Stateless servers

Keep sessions, uploads and caches outside the server so any copy can handle any request.

The database bottleneck

Apps usually scale until the DB doesn't. Use caching, read replicas and good indexes before sharding.

Queue slow work

Push emails, reports and image processing to a queue so web requests stay fast under load.

See it in code

Walk through the code

bash

1aws autoscaling put-scaling-policy \2  --auto-scaling-group-name web-asg \3  --policy-name cpu-50 \4  --policy-type TargetTrackingScaling \5  --target-tracking-configuration '{6    "PredefinedMetricSpecification": {"PredefinedMetricType": "ASGAverageCPUUtilization"},7    "TargetValue": 50.08  }'

1/3Attach a named policy to the existing Auto Scaling group.

Target tracking works like a thermostat: you pick the target, AWS adds or removes instances to stay near it.

Save yourself the pain

Common mistakes

Oops

Storing user sessions in server memory, then adding more servers.

Do this instead

Users get logged out when they hit another server. Move sessions to Redis or signed cookies.

Oops

Scaling web servers while every request still hammers one small database.

Do this instead

Find the real bottleneck first. Add caching and read replicas before adding more web servers.

Your mission

Make it scale on purpose

~45 min

Quick check

Why is horizontal scaling usually preferred over vertical scaling?

Go deeper

1Identify the bottleneck in a hypothetical architecture that would break first under 10x traffic.
2Learn why horizontal scaling requires stateless application design (no local session state).
3Practice designing a scaling plan for a hypothetical system as its user base grows.