Stop 14 · Build Your DevOps PortfolioProjects · 4/7

Monitoring Stack Setup (Prometheus + Grafana)

Part of the DevOps Roadmap

In short

Anyone can deploy an app. Being able to show how healthy it is, and get warned before users complain, is what makes you trustworthy on a team. This project proves that.

Deploy a full monitoring stack for a real or sample application — metrics collection, dashboards, and at least one meaningful alert rule, showing observability skills end to end.

The mascot drives a small open-top car and glances at its speedometer, fuel gauge and warning light.

Think of it like this

The car dashboard

You don't open the bonnet every kilometre. The speedometer, fuel gauge and warning lights tell you what's happening, and the orange engine light tells you when to stop. Prometheus collects the readings, Grafana is the dashboard, and alerts are the warning lights.

Watch it happen

How the stack talks

Your app
Prometheus
Grafana
Alertmanager

GET /metrics

1/6Every 15 seconds Prometheus pulls metrics from your app's /metrics endpoint. This is called scraping.

The big ideas

What you need to know

Must-have: golden signals

Graph latency, traffic, errors and saturation for your app. These four answer 'is it okay?' quickly.

Instrument your own app

Add a metrics library to your app and expose /metrics, not just node-level CPU graphs.

At least one real alert

One alert on a user-facing symptom with a runbook link beats twenty CPU alerts.

Dashboards as code

Export dashboards as JSON and provision them from the repo so the setup is reproducible.

See it in code

Walk through the code

yaml
1# alert-rules.yml (loaded by Prometheus)2groups:3  - name: shop4    rules:5      - alert: HighErrorRate6        expr: |7          sum(rate(http_requests_total{status=~"5.."}[5m]))8            / sum(rate(http_requests_total[5m])) > 0.059        for: 10m10        labels: { severity: page }11        annotations:12          summary: "More than 5% of requests are failing"13          runbook: "https://github.com/you/shop/blob/main/runbooks/errors.md"

1/4Rules live in groups. This one alert is named after what it means, not the metric.

rate() turns a counter that only goes up into a per-second rate. Dividing 5xx by all requests gives the error ratio.

Save yourself the pain

Common mistakes

Oops

Building a dashboard with 40 panels nobody can read.

Do this instead

Start with one overview row of the golden signals. Add detail panels below only when they answer a real question.

Oops

Alerting on high CPU alone.

Do this instead

Alert on what users feel, like errors and slow responses. High CPU with happy users is not an emergency.

Your mission

Milestone 1: your app on a graph

~45 min

Quick check

Why does the alert rule use 'for: 10m'?

Go deeper

  1. 1Instrument a sample application to expose custom metrics, not just infrastructure metrics.
  2. 2Build a Grafana dashboard focused on your app's key health signals.
  3. 3Set up at least one alert rule with a clear, documented response plan.

More projects at this stop

Stuck on this topic? Ask an Insider

Get 1:1 guidance from people who've walked this exact path — on the InsideEdge app.

Download
InsideEdge