Stop 02 · Linux & Command LineTopics · 8/11

System Logs & journalctl

In short

When a service crashes at 3 a.m., the machine already wrote down why. Logs are the first place you look, and journalctl lets you search them like a pro.

Linux system logs record what's happening on a machine — journalctl (on systemd-based systems) is the modern way to query, filter and follow these logs when diagnosing an issue.

The mascot at a society gate flips through a thick visitor register next to a delivery parcel.

Think of it like this

The society gate register

Every visitor, delivery and cab at the society gate is written in the watchman's register with a time. When a parcel goes missing, you don't guess. You open the register and check who came at 4:15. Logs are your server's gate register.

Watch it happen

Chasing a crashed service

You

journald

nginx

bind() failed :80

1/5nginx tried to start, couldn't grab port 80, and logged the error before exiting.

The big ideas

What you need to know

journald collects logs

On systemd systems, journald stores logs from the kernel and every service in one searchable place.

/var/log still matters

Many apps also write plain files in /var/log, like /var/log/nginx/error.log. Check both.

Filter, don't scroll

Use -u for a service, --since for time and -p err for errors only. Thousands of lines shrink to five.

Logs eat disk

Old logs pile up. logrotate and journald size limits stop them from filling the disk.

See it in code

Walk through the code

bash

1journalctl -u nginx --since "1 hour ago"2journalctl -u nginx -p err -n 20      # last 20 errors3journalctl -f -u myapp                 # follow live, like tail -f4journalctl -b -1 -p err                # errors from the previous boot5journalctl --disk-usage6 7tail -f /var/log/nginx/error.log       # plain file logs

1/3Filter by service and time, then narrow to error level only.

-b -1 looks at the boot before this one. Useful when a server rebooted unexpectedly and you need to know why.

Save yourself the pain

Common mistakes

Oops

Restarting a crashed service again and again without reading its logs.

Do this instead

Read journalctl -u service first. The real reason is usually written in the last ten lines.

Oops

Ignoring log growth until the disk hits 100% and everything stops.

Do this instead

Set journald limits like SystemMaxUse and keep logrotate configured for app log files.

Your mission

Break it, then read why

~20 min

Quick check

You want only error-level logs from the myapp service in the last 30 minutes. Which command fits?

Go deeper

1Practice using journalctl to view logs for a specific service.
2Learn to filter logs by time range and follow them live (-f flag) during troubleshooting.
3Understand where traditional log files (/var/log) still matter alongside journalctl.