Stop 11 · Monitoring & ObservabilityTopics · 4/9

Grafana Dashboards

In short

Numbers in a database don't help anyone at 3 am. A good Grafana dashboard shows in five seconds whether users are happy, and where to look if they're not.

The standard visualization layer paired with Prometheus (and many other data sources) — turning raw time-series metrics into readable dashboards for both real-time monitoring and historical analysis.

The mascot sits in a cricket stadium pointing at a giant scoreboard full of bar charts and line graphs.

Think of it like this

The stadium scoreboard

During a cricket match, the big screen shows run rate, required rate and a worm graph. A glance tells the whole crowd who's winning and whether the chase is slipping. A Grafana dashboard is that scoreboard for your system: a few key numbers anyone can read.

Watch it happen

From raw data to a useful panel

Data source

Query

Panel

Dashboard

1/5Connect Grafana to Prometheus, Loki, Elasticsearch, a SQL database or a cloud monitoring API.

The big ideas

What you need to know

Start with RED

For each service: Rate of requests, Errors, Duration. Three panels cover most of what users feel.

One audience per dashboard

An on-call overview, a deep-dive per service, a business view. Mixing them makes each worse.

Variables

Dropdowns for environment and service let one dashboard serve many apps instead of fifty copies.

Dashboards as code

Export JSON or use provisioning, and keep it in Git so a deleted dashboard isn't a lost one.

See it in code

Walk through the code

promql

1# Rate: requests per second2sum(rate(http_requests_total{app="shop"}[5m]))3 4# Errors: share of 5xx responses5sum(rate(http_requests_total{app="shop", status=~"5.."}[5m]))6  / sum(rate(http_requests_total{app="shop"}[5m]))7 8# Duration: 95th percentile latency9histogram_quantile(0.95, sum by (le) (rate(http_request_duration_seconds_bucket{app="shop"}[5m])))

1/3Total requests per second across all pods of the shop app.

Three panel queries for a RED dashboard. In Grafana you can replace app="shop" with a dashboard variable like app="$app".

Save yourself the pain

Common mistakes

Oops

Building a 40-panel dashboard with every metric you can find.

Do this instead

Start with a handful of panels that answer 'are users OK?'. Link to deeper dashboards for the rest.

Oops

Showing only average latency.

Do this instead

Show percentiles like p95 and p99. An average can look fine while one in twenty users waits five seconds.

Your mission

Build a RED dashboard

~35 min

Quick check

Average latency is 120 ms, but users complain about slowness. Which panel would reveal the problem?

Go deeper

1Connect Grafana to your Prometheus instance and build a simple dashboard.
2Practice building a panel that visualizes a specific PromQL query.
3Learn to design a dashboard around the most important 5-6 metrics, not every metric available.