Stop 10 · Performance TestingTopics · 8/8

Setting Performance Benchmarks

In short

'Is 800 ms fast?' has no answer until someone writes down what fast means for this app. Benchmarks turn opinions into a clear pass or fail.

Defining clear, agreed-upon performance targets (e.g. 'p95 response time under 500ms') before testing — without a benchmark, 'is this performance acceptable?' has no objective answer.

The purple blob holds up a stopwatch and a whistle at the finish line of a running track.

Think of it like this

The cut-off time at trials

At the cricket academy trials, the coach announces it first: 2 km in under 9 minutes to make the next round. Nobody argues after the race. Agree performance targets before you test, and results stop being a debate.

Watch it happen

From business goal to test threshold

Goal

Baseline

Target

Threshold

Review

1/5Start with the business: 'Search must feel instant during the sale, with 5,000 shoppers online.'

The big ideas

What you need to know

Specific and measurable

'Fast' is not a target. 'p95 under 500 ms with under 1% errors at 5,000 users' is.

Different pages, different goals

Search might need 300 ms. A monthly report export might be fine at 5 seconds.

Agree together

Product, developers and QA sign off on targets. Then a failed test is a decision, not an argument.

SLAs and SLOs

An SLO is an internal target, like p95 under 500 ms. An SLA is a promise to customers, often with penalties.

See it in code

Walk through the code

1export const options = {2  thresholds: {3    // 95% of requests under 500 ms, 99% under 1.5 s4    http_req_duration: ["p(95)<500", "p(99)<1500"],5    // fewer than 1 in 100 requests may fail6    http_req_failed: ["rate<0.01"],7    // checkout gets its own, looser target8    "http_req_duration{name:checkout}": ["p(95)<800"],9  },10};

1/3Two percentile targets: a typical-slow user and a very-slow user.

Benchmarks written as k6 thresholds. The last line applies only to requests tagged with name checkout.

Save yourself the pain

Common mistakes

Oops

Copying '2 seconds' from a blog post as the target for every page.

Do this instead

Base targets on your users, your baseline and what each page needs to do.

Oops

Setting targets after the test, to match whatever the results were.

Do this instead

Agree targets first. Otherwise every result 'passes' and the test proves nothing.

Your mission

Write a benchmark sheet

~30 min

Quick check

Which is a usable performance benchmark?

Go deeper

1Draft a performance benchmark for a hypothetical system's key user action.
2Learn why benchmarks should be based on real user expectations, not arbitrary round numbers.
3Practice justifying a specific benchmark threshold with reasoning, not just a guess.