Stop 19 · Monitoring & LoggingTopics · 8/9

Prometheus & Grafana (Overview)

In short

Logs tell you what happened to one request. Metrics tell you how the whole system is doing: requests per second, error rate, memory. Prometheus collects them and Grafana draws them.

A widely used open-source pairing for metrics — Prometheus collects and stores time-series metrics (request counts, latencies), and Grafana turns them into dashboards.

The purple blob notes a wall meter reading on a clipboard, with a big chart board behind.

Think of it like this

The electricity meter reader

The meter reader visits every house on a fixed schedule and notes the reading. Back at the office, those readings become monthly graphs, and a sudden spike gets flagged. Prometheus is the reader visiting your /metrics page. Grafana is the graph on the wall.

Watch it happen

Scrape, store, graph, alert

API /metrics

Prometheus

Grafana

Alertmanager

GET /metrics

1/6Prometheus scrapes your API's /metrics page on a schedule, often every 15 seconds. The app just exposes numbers.

The big ideas

What you need to know

Pull, not push

Prometheus scrapes its targets. If a target stops answering, that itself is a signal: up drops to 0.

Counters, gauges, histograms

Counters only go up (requests). Gauges go up and down (memory). Histograms bucket durations for percentiles.

Labels, carefully

Labels like route and status slice metrics. Never use user IDs as labels, or the number of series explodes.

Grafana for dashboards

Grafana reads from Prometheus and many other sources, turning queries into shared dashboards and alerts.

See it in code

Walk through the code

1import client from "prom-client";2client.collectDefaultMetrics(); // memory, CPU, event loop lag3const httpDuration = new client.Histogram({4  name: "http_request_duration_seconds",5  help: "HTTP request duration",6  labelNames: ["method", "route", "status"],7});8 9app.use((req, res, next) => {10  const end = httpDuration.startTimer({ method: req.method });11  res.on("finish", () => end({ route: req.route?.path ?? "unknown", status: res.statusCode }));12  next();13});14 15app.get("/metrics", async (_req, res) => res.type(client.register.contentType).send(await client.register.metrics()));

1/4Free metrics about the Node process itself, with no extra code.

In Grafana, p95 latency is histogram_quantile(0.95, sum by (le) (rate(http_request_duration_seconds_bucket[5m]))).

Save yourself the pain

Common mistakes

Oops

Using the full URL like /orders/981 as a label.

Do this instead

Every ID creates a new time series and Prometheus memory balloons. Use the route pattern /orders/:id.

Oops

Building dashboards nobody watches, with no alerts.

Do this instead

Start with a few alerts on what users feel: error rate, p95 latency and up. Dashboards come second.

Your mission

Your first dashboard

~45 min

Quick check

Why is userId a bad Prometheus label?

Go deeper

1Read how Prometheus 'scrapes' metrics from your application on an interval.
2Look at example Grafana dashboards to understand what's typically monitored (latency, error rate, throughput).
3Recognize this pairing is the de facto standard for infrastructure monitoring at most companies.