Server Log Monitoring Guide

Why log monitoring matters

Many server problems look like a sudden outage, but the real cause is often visible in the logs long before the incident becomes obvious. A site can start slowing down, an API can begin returning 500 errors, or an unexpected burst of login failures can appear. Logs are the first place to check because they show what happened, when it happened, and which part of the stack was affected.

A typical example is a website that suddenly slows at 2 a.m. If you can inspect the Nginx access log and the application error log quickly, you can often distinguish between database latency, backend exceptions, and a third-party timeout.

Key log files

Useful journalctl commands

journalctl -u nginx -f          # Watch nginx logs
journalctl --since "30 min ago"  # Recent logs
journalctl -p err                # Errors only

Real-time monitoring

tail -f /var/log/nginx/access.log
tail -f /var/log/nginx/error.log

Log analysis examples

# Count 5xx errors
grep 'HTTP/1.1" [5]..' /var/log/nginx/access.log | wc -l

# Top 10 IPs by request count
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -10

# Top 10 URLs by request count
awk '{print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -10

A simple alerting script

#!/bin/bash
ALERT_EMAIL="[email protected]"
ERROR_PATTERNS="(PHP Fatal error|ERROR|FATAL|Out of memory)"

tail -f /var/log/nginx/error.log | while read line; do
    if echo "$line" | grep -qE "$ERROR_PATTERNS"; then
        echo "$line" | mail -s "Server Error Alert" $ALERT_EMAIL
    fi
done

This is useful for small setups, but once you operate multiple servers, it is better to move to centralized monitoring tools such as Prometheus, Grafana, or an ELK-based workflow.

Log rotation basics

# /etc/logrotate.d/nginx
/var/log/nginx/*.log {
    daily
    rotate 14
    compress
    notifempty
    create 640 www-data adm
}

The next step: alerts and visualization

Looking at logs manually can solve an immediate issue, but a mature operations workflow also needs to turn visibility into timely alerts. A practical approach is to set thresholds for 5xx errors, timeout rates, and failed login attempts so the team is notified as soon as something abnormal starts to trend upward.

Once you run multiple servers or services, it becomes worthwhile to connect logs to a monitoring dashboard. Prometheus, Grafana, or an ELK-based workflow will not replace the need to inspect logs directly, but they make it much easier to connect a spike in errors to a sudden increase in resource pressure or a specific service failure.

Typical faults and the right log clues

Many beginners treat log monitoring as if it meant staring at a single terminal window, but a better system is to make each issue traceable. That means knowing which log file to inspect, which alert to trigger, and which pattern should be considered abnormal rather than just noisy.

Build a logging culture, not just a log folder

A mature team does not treat log monitoring as a last-minute emergency tool. It treats logs as part of the operating system: who checks alerts, who confirms recovery, which incidents should be documented, and how long logs should be retained for auditing. That turns incident handling from memory-based firefighting into a repeatable process. For small teams, this is especially valuable because one person may need to cover development, operations, and support at the same time.

Reference: systemd journalctl documentation https://www.freedesktop.org/software/systemd/man/journalctl.html; Nginx logging documentation https://nginx.org/en/docs/http/ngx_http_log_module.html

For long-term maintenance, log monitoring is not an extra task. It is part of the operating model for any production website.