Stop 03 · Linux & Windows FundamentalsTopics · 9/10

Command Line Log Analysis

In short

When a server is acting strange, you don't always have a fancy dashboard. A few grep and awk commands can answer 'who hit us, how often, and when' in seconds.

Using grep, awk, and similar tools to search and parse log files quickly from the command line — often faster than a full SIEM for a quick, targeted investigation.

The blob sifts flour through a sieve and catches a few pebbles.

Think of it like this

Sifting atta for pebbles

You don't pick through a whole bag of flour grain by grain. You pour it through a sieve and the few pebbles stay behind. grep is the sieve, and each command after it sorts or counts what was caught.

Watch it happen

Finding the noisiest IP in a web log

cat log

grep

awk

sort | uniq

head

1/5Start with the raw access log: thousands of lines, one per request.

The big ideas

What you need to know

grep finds

grep -i ignores case, -v removes matches, -E allows regex. zgrep searches .gz rotated logs.

awk slices columns

awk '{print $1}' prints the first space-separated field. Know your log's format first.

sort | uniq -c counts

This combo is the quickest way to find top talkers, top paths or top user agents.

tail -f watches live

Follow a log as it grows, piped through grep, to watch an attack or fix as it happens.

See it in code

Walk through the code

bash

1# Top 10 IPs hitting the WordPress login page2grep "wp-login.php" /var/log/nginx/access.log \3  | awk '{print $1}' | sort | uniq -c | sort -rn | head4 5# Requests per hour, to spot a sudden spike6awk '{print substr($4, 2, 14)}' /var/log/nginx/access.log | uniq -c7 8# Watch failed SSH logins as they happen9sudo tail -f /var/log/auth.log | grep --line-buffered "Failed password"

1/3Filter login page hits, print the IP column, then count and rank them.

Field positions depend on the log format. Look at a few raw lines first so you know which field holds the IP or time.

Save yourself the pain

Common mistakes

Oops

Running grep on a huge log on a busy production server.

Do this instead

Copy the log to your analysis machine first, or use a SIEM. Don't slow down the server you're investigating.

Oops

Forgetting rotated logs like access.log.1 and .gz files.

Do this instead

The attack may have started days ago. Use zgrep or zcat to include older, compressed logs.

Your mission

Investigate a sample log

~30 min

Quick check

What does sort | uniq -c | sort -rn do to a list of IPs?

Go deeper

1Practice grep to search for a specific pattern (e.g. a failed login) across log files.
2Combine grep with awk to extract and count specific fields from structured log output.
3Learn to build a personal toolkit of common one-liners for recurring investigation tasks.