Stop 12 · Monitoring & ObservabilityTopics · 8/8

Distributed Tracing Basics

In short

In a single app, a slow request is a stack trace away. Across six services, logs alone leave you guessing. Tracing follows one request end to end and shows each hop's time.

Tracking a single request as it flows through multiple services, capturing timing at each hop — essential for diagnosing latency issues in microservice or serverless architectures where logs alone can't show the full picture.

The blob character scans a parcel with a handheld scanner at a courier hub.

Think of it like this

Tracking a courier parcel

Your parcel gets scanned at every hub: picked up in Jaipur, sorted in Delhi, out for delivery in Pune. One tracking number ties every scan together. If it's stuck, you know exactly which hub is holding it.

Watch it happen

One trace ID through four hops

Gateway

Orders svc

Payments svc

Database

traceparent: 00-4bf9…

1/5The gateway starts a trace and passes its ID in the W3C traceparent header.

The big ideas

What you need to know

Trace and spans

A trace is one request's full path. Spans are its steps, nested like a tree with parent and child.

Context propagation

Each service must pass the trace ID along in headers or message attributes, or the trace breaks.

Sampling

Tracing every request at scale is costly. Keep a percentage, plus all slow or failed ones if your tool supports it.

Tools

OpenTelemetry for instrumentation; AWS X-Ray, Google Cloud Trace, Jaeger or Grafana Tempo to view traces.

See it in code

Walk through the code

1import { trace } from "@opentelemetry/api";2 3const tracer = trace.getTracer("orders-service");4 5export async function placeOrder(order) {6  return tracer.startActiveSpan("placeOrder", async (span) => {7    span.setAttribute("order.id", order.id);8    try {9      await chargePayment(order); // child spans join this trace10      return { ok: true };11    } finally {12      span.end();13    }14  });15}

1/4Get a tracer named after your service. The OpenTelemetry SDK must be set up at startup.

Auto-instrumentation already traces HTTP and DB calls. Add manual spans like this around business steps you care about.

Save yourself the pain

Common mistakes

Oops

Instrumenting only some services, so traces stop halfway.

Do this instead

Propagate context everywhere, including queues and async jobs, or you'll see broken, partial traces.

Oops

Putting user emails or tokens into span attributes.

Do this instead

Traces get stored and shared widely. Use IDs, never personal data or secrets.

Your mission

See your first trace

~40 min

Quick check

Two services show up as separate, unrelated traces. What's the most likely cause?

Go deeper

1Read a conceptual explanation of how a trace ID propagates across service calls.
2Explore a tracing tool's UI (via a demo) to see a sample distributed trace.
3Learn why OpenTelemetry has become the standard, vendor-neutral way to instrument applications for tracing.