How to parse Nginx access logs with awk

How to parse Nginx access logs with awk

Extract IP addresses, status codes, and response times from Nginx access log files using awk.

Prerequisites

• awk installed.
• Nginx access log in the default combined log format.

Step-by-Step: Parse Nginx Logs with awk

1. Print unique IP addresses with request counts. awk tallies the first field (client IP) and sorts by frequency:

   awk '{ips[$1]++} END {for (ip in ips) print ips[ip], ip}' /var/log/nginx/access.log | sort -rn | head -20

2. Filter requests by HTTP status code. Print all requests that returned a 500 error (status code in field 9):

   awk '$9 == 500 {print $1, $7, $9}' /var/log/nginx/access.log

3. Calculate the average response size (field 10 in combined log format):

   awk '{sum += $10; count++} END {print "Avg bytes:", sum/count}' /var/log/nginx/access.log

4. Show the busiest hours by extracting the hour from the timestamp:

   awk -F'[/: ]' '{hours[$5]++} END {for (h in hours) print h":00", hours[h]}' /var/log/nginx/access.log | sort -t: -k1 -n