Stop 08 · Technical Fluency for PMsTopics · 9/11

Latency, Scalability & Reliability Basics

In short

Users feel three things without naming them: is it fast, does it hold up when everyone shows up, and is it there when I need it? Those are latency, scalability and reliability.

Latency is how fast a response comes back, scalability is whether the system holds up under more load, and reliability is whether it stays up — three different technical concerns that get conflated in product conversations.

The purple blob serves pani puri quickly to a crowd of happy customers at a street stall.

Think of it like this

The pani puri stall

Latency is how fast each puri lands in your bowl. Scalability is whether the stall can still serve when a college crowd arrives at 6 pm. Reliability is whether it's open every evening, rain or shine.

Watch it happen

Reading latency percentiles

p50: 180ms

p90: 450ms

p99: 2.4s

1/5Teams describe speed with percentiles, not averages. Here is one API's response time.

The big ideas

What you need to know

Latency

Time from request to response. Users notice delays quickly, especially on slow mobile networks.

Scalability

Can the system handle more users by adding resources, without slowing down or breaking?

Reliability

How often it works as expected. Often tracked as uptime and error rate.

Set targets

Write numbers in specs: "p95 under 500 ms" is testable, "fast" is not.

Save yourself the pain

Common mistakes

Oops

Judging speed by testing on office Wi-Fi.

Do this instead

Test on a mid-range phone on a slow network, like your users in smaller towns.

Oops

Reporting average response time only.

Do this instead

Look at p95 or p99 too. Averages hide the slowest experiences.

Your mission

Feel slow networks

~20 min

Quick check

An API's p50 is 200 ms and its p99 is 4 seconds. What does that tell you?

Go deeper

1Learn to distinguish 'the app is slow' (latency) from 'the app is down' (reliability).
2Understand why a feature that works fine for 100 users can break at 100,000.
3Practice asking engineers which of these three concerns applies before proposing a fix.