Bash Log Analyzer Script

Nginx access logs already contain useful traffic data, but raw log lines do not immediately show which clients, paths, status codes, or user agents appear most often. This script turns a Combined Log Format access log into a repeatable text or CSV summary.

It reads the log once with gawk and calculates request totals, status-code counts, top client IPs, top paths, top user agents, the most common 4xx and 5xx paths, bytes sent, and requests by hour. A multi-gigabyte file is scanned once, not ten times through separate pipelines.

A one-off report script is useful when you need repeatable numbers from yesterday's access.log without standing up a full observability stack. You will also test the script against a small sample log with known results, so you can verify that later changes do not break the report. Bash handles command-line options, input validation, and optional gzip decompression. gawk parses the log and builds all of the report counts in one pass.

Define the Supported Log Format

Nginx ships a default log_format combined that expands to this field order:

$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"

A real line looks like this:

203.0.113.10 - - [12/Sep/2026:09:00:01 +0530] "GET / HTTP/1.1" 200 4521 "-" "curl/8.12.1"

The request, referrer, and user-agent fields are quoted and may contain spaces. That breaks naive cut -d' ' parsing: splitting on whitespace would tear "GET /api/search?q=nginx HTTP/1.1" into unrelated tokens. Field positions are reliable only when you parse the combined layout explicitly (regex or a state machine), and only while your server still uses the default combined format. A custom log_format may change the fields, their order, or their quoting. If its layout differs from the predefined combined format, this parser must be adjusted.

In the standard combined format, $body_bytes_sent is numeric and can be 0 when no response-body bytes were sent. This parser also accepts - defensively and treats it as zero rather than rejecting the entire line. The full-line regex therefore ends with two quoted fields and uses ([0-9]+|-) for bytes — not a generic [0-9-]+ class that would accept malformed tokens.

This script targets combined logs only. Point it at JSON, syslog, or a custom format and the counts will be wrong.

Create a Reproducible Sample Log

Create a small lab tree so examples stay copy-pasteable:

mkdir -p ~/log-analyzer-lab/{bin,fixtures,expected}

The fixture at ~/log-analyzer-lab/fixtures/access.log has 32 physical lines: 31 valid combined rows plus one deliberate malformed line on line 32. We'll use this sample file as our test fixture. Count lines before you trust request totals:

wc -l ~/log-analyzer-lab/fixtures/access.log

32 /root/log-analyzer-lab/fixtures/access.log

The fixture includes repeated IPs, query strings in paths, a - byte value to verify that the parser handles it safely, mixed 2xx/3xx/4xx/5xx codes, and the bad row this is not a valid nginx combined log line on line 32. Timestamps span three hour buckets on 12/Sep/2026 so the Requests by hour section groups on 12/Sep/2026:09, not bare hour digits. Expected output lives in ~/log-analyzer-lab/expected/report.txt for diff-based tests later.

Decide the Report Metrics

Define the report sections before writing parsing code. Each metric maps to one associative array increment inside a single gawk program:

Optional CLI knobs stay small: --top N for list length, --status to filter path sections, and --format csv for CSV output you can open in a spreadsheet or process with another script. HTTP status breakdown and IP totals are always global; only the optional filtered path section narrows when --status is not all.

Validate Input and Options

Start the script with strict mode, a require_gawk() startup check, and a documented exit contract. Configuration mistakes should exit before any log bytes are read — the same pattern as the Bash disk usage alert script uses for bad thresholds.

The Bash exit codes guide explains why monitoring scripts reserve 2 for "could not run" failures. Parse options with a while/case loop (manual parsing) or extend the same flags through Bash getopts if you prefer getopts for short options only. The shipped script rejects extra operands after -- so log-analyzer.sh -- --extra access.log exits 2 instead of treating --extra as a second path.

This script requires GNU awk (gawk), not just any awk implementation. It uses GNU awk features to capture regex groups and sort associative arrays by count. The startup check prints a clear error if gawk is missing:

gawk --version

GNU Awk 5.3.0, API 4.0, PMA Avon 8-g1, (GNU MPFR 4.2.1, GNU MP 6.2.1)

validate_log_path() runs in the main shell before the pipeline — not inside a subshell — so a missing file or failed gzip -t on a .gz path exits 2 immediately without printing an empty report. That keeps failure handling in Bash where exit 2 is reliable; the gawk program only counts and prints.

Reject a missing log file before streaming:

~/log-analyzer-lab/bin/log-analyzer.sh

ERROR: LOGFILE path is required
Usage: log-analyzer.sh [OPTIONS] LOGFILE
...

The script prints usage on stderr and exits 2 when LOGFILE is omitted.

Count Total Requests and Status Codes

Rather than splitting the line on spaces, the parser matches the entire record and captures each combined-log field separately. This keeps spaces inside the request, referrer, and user-agent fields from shifting later fields.

Extract those fields with gawk match() and a capture array. The regex spans the full line through both quoted trailing fields; bytes accept digits or - only:

/^([^ ]+) ([^ ]+) ([^ ]+) \[([^]]+)\] "([^"]*)" ([0-9]{3}) ([0-9]+|-) "([^"]*)" "([^"]*)"$/

In the capture array, m[1] is the client IP, m[4] is the timestamp, m[5] is the request, m[6] is the HTTP status, m[7] is the response-body bytes, m[8] is the referrer, and m[9] is the user agent.

Preview the first fixture line with gawk:

head -1 ~/log-analyzer-lab/fixtures/access.log | gawk '{
  if (match($0, /^([^ ]+) ([^ ]+) ([^ ]+) \[([^]]+)\] "([^"]*)" ([0-9]{3}) ([0-9]+|-) "([^"]*)" "([^"]*)"$/, m)) {
    print "ip=" m[1]
    print "status=" m[6]
    print "bytes=" m[7]
    print "ua=" m[9]
  }
}'

ip=203.0.113.10
status=200
bytes=4521
ua=curl/8.12.1

Inside the full program, increment total_requests on every successful match and bump status_counts[status] for the status table. Hourly buckets use a second match() on time_local to capture 12/Sep/2026:09 before the minute field — grouping by date and hour together prevents 12/Sep/2026:09 and 13/Sep/2026:09 from merging into one 09 bucket. The report sorts those display keys as strings with asorti(hour_bucket_counts, sorted_hour); that works well for short log snapshots but is not guaranteed to be chronological across months or years.

The awk command examples walkthrough covers match() capture arrays if this syntax is new.

Find Top Client IPs

Count each $remote_addr in an associative array: ip_counts[ip]++. After the read loop, sort keys by count descending with gawk's asorti() and the @val_num_desc control variable:

gawk 'BEGIN{ip_counts["203.0.113.10"]=7; ip_counts["203.0.113.30"]=6; ip_counts["198.51.100.5"]=3}
END{n=asorti(ip_counts,sorted,"@val_num_desc"); for(i=1;i<=n;i++) printf "%6d  %s\n", ip_counts[sorted[i]], sorted[i]}'

     7  203.0.113.10
     6  203.0.113.30
     3  198.51.100.5

@val_num_desc tells gawk to sort the array by its numeric values from highest to lowest. Here those values are request counts, so the busiest IPs appear first. Because the counts are already in associative arrays, the script does not need a separate sort | uniq -c pipeline for each report section.

Find Top Requested Paths

Split the captured request field on spaces. For "GET /api/search?q=nginx HTTP/1.1", the path is the second token:

head -1 ~/log-analyzer-lab/fixtures/access.log | gawk '{
  if (match($0, /^([^ ]+) ([^ ]+) ([^ ]+) \[([^]]+)\] "([^"]*)" ([0-9]{3}) ([0-9]+|-) "([^"]*)" "([^"]*)"$/, m)) {
    split(m[5], req, " ")
    print "path=" req[2]
  }
}'

path=/

Increment path_counts[path]++ for each valid line. Query strings stay attached (/api/search?q=nginx), which matches how operators usually want "top URL" reports from raw access logs. Paths that never receive traffic simply never appear in the table; there is no pre-seeded list of routes.

Find Top User Agents

The ninth capture group contains the user-agent value from the quoted $http_user_agent field. Increment ua_counts[user_agent]++ on every valid line and print Top user agents with the same asorti() sort as IPs and paths. In this fixture, the literal user-agent value Mozilla/5.0 appears most often; curl/8.12.1 and kube-probe/1.0 show automation and health-check traffic.

Long user-agent strings can dominate width in a terminal report; CSV mode emits a top_user_agent section so you can filter or chart in a spreadsheet without re-parsing the log.

Report 4xx and 5xx Activity

Track client and server errors in separate arrays so a spike in 404 does not mask 500 responses:

• err4xx_paths[path]++ when status is 400–499
• err5xx_paths[path]++ when status is 500–599

Against the fixture, /missing.css leads 4xx paths and /broken leads 5xx paths. Both sections print even when empty ((none)).

Sum Response Bytes

In the standard combined format, $body_bytes_sent is numeric and can be 0 when no response-body bytes were sent. This parser also accepts - defensively and treats it as zero rather than rejecting the entire line. The production program applies that rule inside add_bytes():

gawk 'BEGIN{total=0}
     function add_bytes(t,f){ if (f=="-") return t; return t+(f+0) }
     END{print add_bytes(add_bytes(add_bytes(0,"4521"),"-"),"0")}'

4521

The production program stores bytes in m[7] and applies the same rule inside add_bytes().

Generate a CSV Summary

CSV output uses four columns: section,rank,key,count. Wrap text keys with csv_escape() — double any embedded quote and surround the field with quotes — so paths or user agents containing commas do not break spreadsheet imports.

Preview CSV mode on the fixture:

~/log-analyzer-lab/bin/log-analyzer.sh --format csv --top 3 ~/log-analyzer-lab/fixtures/access.log | head -8

section,rank,key,count
summary,1,"total_requests",31
summary,2,"total_bytes",282045
summary,3,"malformed_lines",1
status,1,"HTTP 200",18
status,2,"HTTP 404",6
status,3,"HTTP 500",4

Every CSV row uses the same four columns, which makes the output easier to open in a spreadsheet or process with another script. When --status is not all, an extra filtered_path section appears in CSV output with paths that matched the filter.

Handle Malformed Lines

Lines that fail the combined regex increment malformed++ and are skipped. The summary prints Malformed/skipped: so a partial file or log rotation glitch is visible instead of silently lowering totals.

Inspect the bad row in the fixture:

grep -n 'not a valid' ~/log-analyzer-lab/fixtures/access.log

32:this is not a valid nginx combined log line

A full run reports 31 valid requests and 1 malformed line — matching the 32-line file minus the garbage row on line 32. Blank lines are skipped without incrementing either counter, so accidental empty rows in a rotated log do not inflate the malformed count.

Make Large-File Processing Efficient

Scan the log once. Feeding the same file through ten separate grep | sort | uniq -c pipelines re-reads disk and repeats parsing work. One gawk program holds every counter in associative arrays and prints all sections from its END block. Memory use can become large when the log contains huge numbers of unique values — for example, if every URL contains a unique request ID.

Open the input stream from a plain file, a .gz file, or stdin. The Bash layer uses cat for normal files, gzip -cd when the path ends in .gz, and cat on - for a pipe. Before decompressing, validate_log_path() runs gzip -t on .gz inputs so a truncated archive exits 2 instead of yielding a partial report.

Create a compressed copy of the fixture:

gzip -c ~/log-analyzer-lab/fixtures/access.log > /tmp/access.log.gz

Run the analyzer on the compressed copy — the script detects the .gz suffix and calls gzip -cd internally:

~/log-analyzer-lab/bin/log-analyzer.sh /tmp/access.log.gz | head -5

Log summary (Nginx Combined Log Format)
=====================================
Total requests:     31
Total bytes sent:   282045
Malformed/skipped:  1

A corrupt gzip file fails the integrity test before parsing starts. Create a truncated archive and confirm the script exits 2 without printing a report:

printf 'not gzip' > /tmp/bad.log.gz

Point the analyzer at the bad archive:

~/log-analyzer-lab/bin/log-analyzer.sh /tmp/bad.log.gz

ERROR: failed to read or decompress log input

You do not need a read file line by line in Bash loop here — gawk reads the stream incrementally.

Add --top and Status Filters

Limit list length with --top N (default 10). The value is validated in Bash before gawk starts so a typo cannot produce a partial report. Filter path-oriented sections with --status:

Show only the top three rows per section:

~/log-analyzer-lab/bin/log-analyzer.sh --top 3 ~/log-analyzer-lab/fixtures/access.log | sed -n '/Top client IPs/,/^$/p'

Top client IPs
       7  203.0.113.10
       6  203.0.113.30
       5  203.0.113.50

Filter to 404 responses when triaging missing assets:

~/log-analyzer-lab/bin/log-analyzer.sh --status 404 ~/log-analyzer-lab/fixtures/access.log | sed -n '/matching status filter/,/^$/p'

Top paths matching status filter (404)
       3  /missing.css
       2  /old-page
       1  /favicon.ico

Passing a non-integer to --top also exits 2:

~/log-analyzer-lab/bin/log-analyzer.sh --top abc ~/log-analyzer-lab/fixtures/access.log

ERROR: --top requires a positive integer (got: abc)

Test Against Known Fixture Output

Save expected output once, then diff after every change:

diff -u ~/log-analyzer-lab/expected/report.txt <(~/log-analyzer-lab/bin/log-analyzer.sh ~/log-analyzer-lab/fixtures/access.log)

No diff output means every section still matches the committed baseline: 31 requests, 282045 bytes, status counts, top lists, user-agent table, and the three 12/Sep/2026:HH hour buckets. Commit expected/report.txt beside the fixture so teammates and CI share the same expected results.

Complete Script and Sample Output

Create the lab directories if you have not already:

mkdir -p ~/log-analyzer-lab/{bin,fixtures,expected}

Save the full program as ~/log-analyzer-lab/bin/log-analyzer.sh:

#!/usr/bin/env bash
# log-analyzer.sh — summarize Nginx Combined Log Format access logs
# Supported format (default log_format combined):
#   $remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"
# Requires GNU awk (gawk) for match(..., arr) and asorti(..., "@val_num_desc").
# Exit contract (this script only):
#   0 = report generated
#   2 = usage, missing file, unreadable input, or input processing failure
set -euo pipefail

TOP_N=10
STATUS_FILTER="all"
OUTPUT_FORMAT="text"
LOG_PATH=""

usage() {
  cat <<'EOF'
Usage: log-analyzer.sh [OPTIONS] LOGFILE

Summarize an Nginx access log in Combined Log Format.

Options:
  --top N          Show top N rows per section (default: 10)
  --status FILTER  Limit path sections to statuses: all, 4xx, 5xx, or an exact code (404)
  --format FMT     Output format: text (default) or csv
  --help           Show this help

Examples:
  log-analyzer.sh /var/log/nginx/access.log
  log-analyzer.sh --top 5 --status 404 fixtures/access.log
  gzip -cd access.log.gz | log-analyzer.sh -
EOF
}

require_gawk() {
  if ! command -v gawk >/dev/null 2>&1; then
    echo "ERROR: GNU awk (gawk) is required" >&2
    exit 2
  fi
}

validate_top() {
  if ! [[ "$TOP_N" =~ ^[0-9]+$ ]] || (( TOP_N < 1 )); then
    echo "ERROR: --top requires a positive integer (got: ${TOP_N})" >&2
    exit 2
  fi
}

validate_status_filter() {
  case "$STATUS_FILTER" in
    all|4xx|5xx) return 0 ;;
    [0-9][0-9][0-9]) return 0 ;;
    *)
      echo "ERROR: --status must be all, 4xx, 5xx, or a three-digit code (got: ${STATUS_FILTER})" >&2
      exit 2
      ;;
  esac
}

validate_format() {
  case "$OUTPUT_FORMAT" in
    text|csv) return 0 ;;
    *)
      echo "ERROR: --format must be text or csv (got: ${OUTPUT_FORMAT})" >&2
      exit 2
      ;;
  esac
}

parse_args() {
  while [[ $# -gt 0 ]]; do
    case "$1" in
      --top)
        [[ $# -ge 2 ]] || { echo "ERROR: --top requires a value" >&2; exit 2; }
        TOP_N=$2
        shift 2
        ;;
      --status)
        [[ $# -ge 2 ]] || { echo "ERROR: --status requires a value" >&2; exit 2; }
        STATUS_FILTER=$2
        shift 2
        ;;
      --format)
        [[ $# -ge 2 ]] || { echo "ERROR: --format requires a value" >&2; exit 2; }
        OUTPUT_FORMAT=$2
        shift 2
        ;;
      --help|-h)
        usage
        exit 0
        ;;
      --)
        shift
        break
        ;;
      -*)
        echo "ERROR: unknown option: $1" >&2
        usage >&2
        exit 2
        ;;
      *)
        if [[ -n "$LOG_PATH" ]]; then
          echo "ERROR: only one log file path is allowed" >&2
          exit 2
        fi
        LOG_PATH=$1
        shift
        ;;
    esac
  done

  if [[ $# -gt 0 ]]; then
    if [[ -n "$LOG_PATH" ]]; then
      echo "ERROR: only one log file path is allowed" >&2
      exit 2
    fi
    LOG_PATH=$1
    shift
  fi

  if [[ $# -gt 0 ]]; then
    echo "ERROR: only one log file path is allowed" >&2
    exit 2
  fi

  if [[ -z "$LOG_PATH" ]]; then
    echo "ERROR: LOGFILE path is required" >&2
    usage >&2
    exit 2
  fi

  validate_top
  validate_status_filter
  validate_format
}

validate_log_path() {
  if [[ "$LOG_PATH" == "-" ]]; then
    return 0
  fi
  if [[ ! -e "$LOG_PATH" ]]; then
    echo "ERROR: log file not found: ${LOG_PATH}" >&2
    exit 2
  fi
  if [[ ! -r "$LOG_PATH" ]]; then
    echo "ERROR: log file not readable: ${LOG_PATH}" >&2
    exit 2
  fi
  if [[ "$LOG_PATH" == *.gz ]]; then
    if ! command -v gzip >/dev/null 2>&1; then
      echo "ERROR: gzip is required for .gz input" >&2
      exit 2
    fi
    if ! gzip -t "$LOG_PATH" 2>/dev/null; then
      echo "ERROR: failed to read or decompress log input" >&2
      exit 2
    fi
  fi
}

open_log_stream() {
  if [[ "$LOG_PATH" == "-" ]]; then
    cat
  elif [[ "$LOG_PATH" == *.gz ]]; then
    gzip -cd "$LOG_PATH"
  else
    cat "$LOG_PATH"
  fi
}

run_report() {
  validate_log_path
  if ! open_log_stream | gawk -v top_n="$TOP_N" -v status_filter="$STATUS_FILTER" -v out_fmt="$OUTPUT_FORMAT" '
function csv_escape(s,    t) {
  t = s
  gsub(/"/, "\"\"", t)
  return "\"" t "\""
}

function status_matches(code,    c) {
  c = code + 0
  if (status_filter == "all") return 1
  if (status_filter == "4xx") return (c >= 400 && c < 500)
  if (status_filter == "5xx") return (c >= 500 && c < 600)
  return (code == status_filter)
}

function add_bytes(total, field) {
  if (field == "-") return total
  return total + (field + 0)
}

function print_text_top(title, arr,    n, i, key, sorted, limit) {
  print title
  n = asorti(arr, sorted, "@val_num_desc")
  limit = (n < top_n ? n : top_n)
  if (limit == 0) {
    print "  (none)"
    print ""
    return
  }
  for (i = 1; i <= limit; i++) {
    key = sorted[i]
    printf "  %6d  %s\n", arr[key], key
  }
  print ""
}

function print_csv_top(section, arr,    n, i, key, sorted, limit) {
  n = asorti(arr, sorted, "@val_num_desc")
  limit = (n < top_n ? n : top_n)
  for (i = 1; i <= limit; i++) {
    key = sorted[i]
    print section "," i "," csv_escape(key) "," arr[key]
  }
}

{
  line = $0
  if (line ~ /^[[:space:]]*$/) next

  if (!match(line, /^([^ ]+) ([^ ]+) ([^ ]+) \[([^]]+)\] "([^"]*)" ([0-9]{3}) ([0-9]+|-) "([^"]*)" "([^"]*)"$/, m)) {
    malformed++
    next
  }

  ip = m[1]
  time_local = m[4]
  request = m[5]
  status = m[6]
  bytes_field = m[7]
  user_agent = m[9]

  split(request, req_parts, " ")
  if (length(req_parts) < 2 || req_parts[2] == "") {
    malformed++
    next
  }
  path = req_parts[2]

  if (!match(time_local, /^([^:]+:[0-9]{2}):/, hm)) {
    malformed++
    next
  }

  total_requests++
  ip_counts[ip]++
  path_counts[path]++
  status_counts[status]++
  ua_counts[user_agent]++
  total_bytes = add_bytes(total_bytes, bytes_field)
  hour_bucket_counts[hm[1]]++

  code = status + 0
  if (code >= 400 && code < 500) {
    err4xx_paths[path]++
  }
  if (code >= 500 && code < 600) {
    err5xx_paths[path]++
  }

  if (status_matches(status)) {
    filtered_paths[path]++
  }
}

END {
  if (out_fmt == "csv") {
    print "section,rank,key,count"
    print "summary,1," csv_escape("total_requests") "," (total_requests + 0)
    print "summary,2," csv_escape("total_bytes") "," (total_bytes + 0)
    print "summary,3," csv_escape("malformed_lines") "," (malformed + 0)

    n = asorti(status_counts, sorted_status, "@val_num_desc")
    for (i = 1; i <= n; i++) {
      code = sorted_status[i]
      print "status," i "," csv_escape("HTTP " code) "," status_counts[code]
    }

    print_csv_top("top_ip", ip_counts)
    print_csv_top("top_path", path_counts)
    print_csv_top("top_user_agent", ua_counts)
    print_csv_top("top_4xx_path", err4xx_paths)
    print_csv_top("top_5xx_path", err5xx_paths)
    if (status_filter != "all") {
      print_csv_top("filtered_path", filtered_paths)
    }

    n = asorti(hour_bucket_counts, sorted_hour)
    for (i = 1; i <= n; i++) {
      hour = sorted_hour[i]
      print "hour," i "," csv_escape(hour) "," hour_bucket_counts[hour]
    }
    exit 0
  }

  print "Log summary (Nginx Combined Log Format)"
  print "====================================="
  printf "Total requests:     %d\n", total_requests + 0
  printf "Total bytes sent:   %d\n", total_bytes + 0
  printf "Malformed/skipped:  %d\n\n", malformed + 0

  print "Status code distribution"
  n = asorti(status_counts, sorted_status, "@val_num_desc")
  for (i = 1; i <= n; i++) {
    code = sorted_status[i]
    printf "  %6d  HTTP %s\n", status_counts[code], code
  }
  print ""

  print_text_top("Top client IPs", ip_counts)
  print_text_top("Top requested paths", path_counts)
  print_text_top("Top user agents", ua_counts)
  print_text_top("Top 4xx paths", err4xx_paths)
  print_text_top("Top 5xx paths", err5xx_paths)

  if (status_filter != "all") {
    print_text_top("Top paths matching status filter (" status_filter ")", filtered_paths)
  }

  print "Requests by hour"
  n = asorti(hour_bucket_counts, sorted_hour)
  if (n == 0) {
    print "  (none)"
    print ""
  } else {
    for (i = 1; i <= n; i++) {
      hour = sorted_hour[i]
      printf "  %6d  %s\n", hour_bucket_counts[hour], hour
    }
    print ""
  }
}
'; then
    echo "ERROR: failed to read or decompress log input" >&2
    exit 2
  fi
}

require_gawk
parse_args "$@"
run_report

Mark it executable after the file is written:

chmod +x ~/log-analyzer-lab/bin/log-analyzer.sh

Run against the fixture:

~/log-analyzer-lab/bin/log-analyzer.sh ~/log-analyzer-lab/fixtures/access.log

Log summary (Nginx Combined Log Format)
=====================================
Total requests:     31
Total bytes sent:   282045
Malformed/skipped:  1

Status code distribution
      18  HTTP 200
       6  HTTP 404
       4  HTTP 500
       2  HTTP 403
       1  HTTP 304

Top client IPs
       7  203.0.113.10
       6  203.0.113.30
       5  203.0.113.50
       5  203.0.113.40
       5  203.0.113.20
       3  198.51.100.5

Top requested paths
       8  /
       3  /missing.css
       3  /health
       2  /static/app.js
       2  /report.csv
       2  /old-page
       2  /broken
       2  /admin
       1  /login
       1  /favicon.ico

Top user agents
      24  Mozilla/5.0
       4  curl/8.12.1
       3  kube-probe/1.0

Top 4xx paths
       3  /missing.css
       2  /old-page
       2  /admin
       1  /favicon.ico

Top 5xx paths
       2  /broken
       1  /login
       1  /crash

Requests by hour
      20  12/Sep/2026:09
       7  12/Sep/2026:10
       4  12/Sep/2026:11

Pipe from decompressor when the file is already on stdout:

gzip -cd access.log.gz | ~/log-analyzer-lab/bin/log-analyzer.sh -

When you pass -, the analyzer cannot validate the command producing stdin. Check the exit status of the upstream command, or pass the .gz filename directly when you want the script to run gzip -t before analysis. From another Bash script, enable pipefail so a failed decompressor does not hide behind a successful report:

set -o pipefail
gzip -cd access.log.gz | ~/log-analyzer-lab/bin/log-analyzer.sh -

Limits of a Bash Log Analyzer

This script works well when you need a quick report from one Nginx access log on one server. It does not tail logs, deduplicate by request ID, or join across multiple files. Reach for dedicated tools when requirements grow:

Common upgrades:

• GoAccess — interactive terminal or HTML reports from access logs with minimal setup.
• Elasticsearch / OpenSearch (ELK) — full-text search, aggregations, and Kibana dashboards at fleet scale.
• Grafana Loki — label-based log storage when you already run Prometheus-style metrics.

Keep the Bash reporter for jump boxes, CI smoke checks, or air-gapped hosts where installing GoAccess or shipping logs to a cluster is not practical. Move to a dedicated log platform when you need fast searches across historical logs, long-term retention, or shared dashboards.

Summary

The log analyzer parses Nginx Combined Log Format with one gawk pass: a regex extracts IP, timestamp, request, status, bytes, referer, and user agent; associative arrays accumulate counts; asorti(..., "@val_num_desc") prints top-N sections. Malformed lines increment a visible counter instead of disappearing from totals. Hourly buckets include both the date and hour so requests from different dates are not merged; the report sorts those keys as strings and is intended mainly for short log snapshots rather than long date ranges.

CLI validation exits 2 on bad options, missing files, or corrupt gzip before the pipeline runs. The gawk startup check refuses hosts where only a non-GNU awk is installed. validate_log_path() and gzip -t keep unreadable inputs from producing a misleading empty summary. CSV mode escapes quoted fields; pass a .gz path directly when you want integrity checks, or use - when stdin is already decompressed.

Use this script for repeatable combined-log snapshots on jump boxes or in CI. Move to GoAccess, ELK, or Loki when you need live tailing, cross-host search, alerting, or formats beyond the default combined layout.

References

• Nginx — Logging and monitoring (log_format)
• GNU Awk manual