The Site Is Down. Stop Restarting and Read the Logs.

It's 2:14 on a Tuesday afternoon and the site is down. Someone from support has already pasted a screenshot into chat. Your first instinct is to refresh the page, then restart the service, then refresh again — and if that works, you'll never know what happened, which means it will happen again.

There's a better first move, and it's almost always the same one: go read the logs. Not skim them. Read them, with a specific question in mind and a couple of commands that narrow thousands of lines down to the handful that matter.

Most engineers know tail -f and stop there. That's like owning a library card and only ever reading the first page of whatever's on the front table. This guide covers the small set of commands that actually resolve incidents — where logs live, how to slice them by time and severity, and how to go from "something broke" to a line number in about five minutes.

Logs aren't a diary you read after the fact. They're a query interface to the past, and the only one you get.

Where your logs actually are

Linux logging is split between two systems, and knowing which one you're dealing with saves a lot of flailing.

The systemd journal holds logs from anything started as a systemd service — your web server, your database, your app if it runs under a unit file, plus the kernel and system events. It's a binary store, so you can't cat it. You query it with journalctl.

Plain text files in /var/log/ hold everything else, plus application-specific logs that programs write themselves. Nginx writes access.log and error.log there. Your app framework probably writes its own file somewhere under /var/log/ or inside its project directory.

Start by looking at what exists:

ls -lhtr /var/log/

The -t sorts by modification time and -r reverses it, so the most recently written files land at the bottom of your terminal — right where your cursor is. That's a small thing that saves scrolling every single time. The -h gives you human-readable sizes, which matters because a log file that grew to 4GB overnight is itself a finding.

If the service runs under systemd, skip the directory hunt entirely and ask the journal directly.

journalctl: five flags that do 90% of the work

journalctl with no arguments dumps everything since the beginning of time, which is useless. The value is in the filters, and they stack.

Filter by service with -u:

journalctl -u nginx

Filter by time with --since and --until. These accept both natural phrasing and absolute timestamps:

journalctl -u nginx --since "30 min ago"
journalctl -u nginx --since "2026-09-19 13:50" --until "2026-09-19 14:20"

Bracketing the incident window this way is the single highest-value habit in this guide. If support says the errors started around 2:10, look at 1:50 through 2:20 — the cause usually shows up slightly before the symptom.

Filter by severity with -p. Priority levels run from emerg (0) through debug (7), and -p err means "errors and anything worse":

journalctl -u nginx -p err --since today

Search text with -g (or --grep). Use this instead of piping into grep — the journal does the filtering internally rather than decompressing and formatting lines you're about to throw away:

journalctl -u sshd --since "2 hours ago" -g "Failed password"

One quirk worth knowing: if your pattern is entirely lowercase, matching is case-insensitive. Add a capital letter anywhere and it becomes case-sensitive. You can force the behavior with --case-sensitive=yes or no if you'd rather not think about it.

Follow live with -f, the journal's answer to tail -f:

journalctl -u myapp -f

Now combine them. This is what an actual incident command looks like:

journalctl -u myapp -p warning --since "1 hour ago" --no-pager

Errors and warnings, one service, last hour, printed straight to stdout so you can pipe it somewhere. That command turns a 40,000-line haystack into something you can read in a minute.

That last one deserves a note. If a machine rebooted unexpectedly, journalctl -b -1 -p err shows you the errors from the boot before the reboot — often the only surviving evidence of why it went down.

Text logs: grep, but with the parts people skip

For plain files, grep is still the workhorse. The flags most people never learn are the ones that make it useful during an incident.

grep -n -i "timeout" /var/log/myapp/app.log

-n prints line numbers, -i ignores case. But the real upgrade is context:

grep -n -C 5 "500 Internal" /var/log/nginx/error.log

-C 5 shows five lines before and after each match. An error message alone rarely tells you anything; the three lines before it usually do. You can also use -B (before) and -A (after) separately when you only need one side.

When you don't know what you're looking for yet, count instead of reading:

grep -c "ERROR" /var/log/myapp/app.log

A count answers "is this normal or new?" faster than any amount of scrolling. If yesterday's file has 12 errors and today's has 8,400, you've learned something before reading a single message.

For structured access logs, awk beats grep because it understands columns. Nginx's default combined format puts the status code in field 9:

awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

That pipeline gives you a frequency table of HTTP status codes — how many 200s, how many 500s, how many 404s — in one line. It's the fastest way to distinguish "the whole site is down" from "one endpoint is throwing errors."

sort | uniq -c | sort -rn is the most reusable four words in shell. Anything you can extract one-per-line, you can rank by frequency.

AD

Want the top offending URLs instead? Field 7 is the request path:

grep ' 500 ' /var/log/nginx/access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20

Rotated and compressed logs

Here's a trap that costs people twenty minutes. You grep app.log, find nothing, and conclude the error didn't happen. But logrotate moved yesterday's entries into app.log.1, and the day before into app.log.2.gz. Your search window and the file's window don't overlap.

Use zgrep, which handles both plain and gzipped files:

zgrep -h "OutOfMemory" /var/log/myapp/app.log*

The -h suppresses filename prefixes so the output reads as one continuous stream. If you want to know which file each hit came from, drop the -h.

Check the rotation policy too — /etc/logrotate.d/ holds a config file per service telling you how many days you actually retain. Discovering during an incident that you keep three days of logs and the customer is asking about last Thursday is a bad moment.

A five-minute triage routine

When something breaks, run these in order. Each one either answers the question or narrows it.

# 1. Is the machine itself fine? Disk full is the #1 mystery outage.
df -h
free -h

# 2. Is the service even running?
systemctl status myapp

# 3. What did it say in the last hour, errors only?
journalctl -u myapp -p err --since "1 hour ago" --no-pager

# 4. What was the error rate before vs. during?
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# 5. Full context around the first bad line.
grep -n -C 10 "<the message from step 3>" /var/log/myapp/app.log

Step 1 catches more outages than anyone likes to admit. A filesystem at 100% makes services fail in creative, misleading ways — databases refuse writes, logs stop being written (so the logs are empty, which is itself the clue), sessions vanish. Always check it first; it costs two seconds.

Step 3 is where you usually find it. And note the order of steps 3 and 5: get the exact error string from the journal, then use it to find full context in the application's own log. Searching for a string you actually saw beats guessing at keywords.

Make future-you's job easier

Two habits pay for themselves the first time you need them.

Log with enough context to search by. An error message reading Database error is unsearchable. One reading db_write_failed order_id=88412 retries=3 lets you find every instance in one grep, correlate it with a specific request, and count how often it happens. Include an identifier — request ID, user ID, job ID — on every line that could ever be interesting.

Log timestamps in UTC. When you have a web server in one region and a database in another, comparing local timestamps across machines during an incident is an unforced error. Pick UTC everywhere, convert at display time.

Reading logs isn't a talent. It's a short list of commands — journalctl -u ... -p err --since, grep -C, awk | sort | uniq -c | sort -rn — applied in a sensible order with a specific question in mind. The engineers who look fast at this aren't reading more; they're reading less, and they picked which lines to read before they started.

Next time something breaks, resist the restart. Spend five minutes in the logs first. You'll usually find the answer — and even when you don't, you'll have narrowed it enough that the restart becomes a decision instead of a reflex.