Stop 11 · Monitoring & ObservabilityTopics · 5/9

The ELK / EFK Stack

In short

With twenty pods across five servers, you can't SSH into each one and grep logs. ELK and EFK collect every log line into one place, so you can find the error that matters.

Elasticsearch (storage/search), Logstash or Fluentd (log collection/processing), and Kibana (visualization) — a common open-source stack for centralizing and searching logs across many services.

The mascot sorts a big sack of envelopes into a wall of wooden pigeonhole shelves.

Think of it like this

The post office sorting room

Letters arrive from post boxes all over the city. Collectors bring them in, sorters drop each into the right pigeonhole by area, and the counter clerk can find any letter fast. Fluentd or Logstash collects, Elasticsearch sorts and stores, Kibana is the counter where you search.

Watch it happen

A log line's trip

App stdout

Collector

Parse

Elastic

Kibana

1/5Your app writes a JSON line to stdout. The container runtime saves it to a log file on the node.

The big ideas

What you need to know

E, L or F, K

Elasticsearch stores, Logstash or Fluentd/Fluent Bit collect and transform, Kibana visualises.

Structured logs pay off

JSON logs become searchable fields automatically. Free text needs fragile parsing rules.

Retention costs money

Logs pile up fast. Set index lifecycle rules to delete or move old data after days or weeks.

Alternatives exist

OpenSearch is an open-source fork of Elasticsearch. Grafana Loki is a cheaper option that indexes labels only.

See it in code

Walk through the code

bash

1# Index one log event (local dev cluster with security turned off)2curl -X POST "localhost:9200/app-logs/_doc" -H "Content-Type: application/json" \3  -d '{"@timestamp":"2026-10-27T10:42:13Z","level":"error","msg":"payment failed"}'4 5# Search it back6curl "localhost:9200/app-logs/_search?q=level:error&pretty"

1/2Send one JSON document into an index called app-logs. Collectors do this for you in bulk.

Recent Elasticsearch versions turn security on by default. Only disable it on a local practice cluster, never on anything reachable from a network.

Save yourself the pain

Common mistakes

Oops

Logging passwords, tokens or full card numbers 'for debugging'.

Do this instead

Never log secrets or sensitive personal data. Mask fields in the app or in the collector before they're stored.

Oops

Keeping every log forever because storage seems cheap.

Do this instead

Set retention per index. Debug logs might need a week, audit logs might need much longer.

Your mission

Search your own logs

~40 min

Quick check

In an EFK stack on Kubernetes, what does Fluent Bit usually do?

Go deeper

1Explore a hosted or local ELK/EFK stack demo to see log aggregation in action.
2Learn why centralized logging matters once you have more than a handful of servers to check.
3Understand the basic flow: logs collected, shipped, indexed, then searched/visualized.