2.20.2.3. Log analysis with console commands

You can analyze logs with console commands on a hosting or on a local PC:

  1. Download logs on your PC.
  2. Through filemanager or any FTPclient upload the downloaded logs to the hosting to the root directory of the hosting account.
  3. If the logs are in an archive file, unzip them using the file manager.
  4. Connect to hosting via SSH.
  5. Use in terminal console commandspresented below.
  1. Download logs on your PC.
  2. If the logs are in an archive file, unzip them.
  3. Start the terminal on your PC:
    • On Windows — Requires WSL or Cygwin installed. If there are difficulties with their installation, use the simpler analysis on hosting.
    • On Linux, you can use a standard terminal.
    • On macOS — you can use standard terminal.
  4. Use in terminal console commandspresented below.

Commands

In all commands, instead of access.log specify the name of the downloaded log file or the full path to it.

Server response codes in descending order of their number: 

awk '{print $9}' access.log | sort | uniq -c | sort -r

25 most active IPs: 

cat access.log | awk '{ print $1 }' | sort | uniq -c | sort -rn | head -n 25

The number of requests from each IP in descending order: 

cat access.log | awk '{print "requests from " $1}' | sort | uniq -c | sort -r

10 Most Popular Referer: 

cat access.log | awk -F \" ' { print $4 } ' | grep -v '-' | sort | uniq -c | sort -rn | head -n 10

Top 10 User-Agents: 

cat access.log | awk -F \" ' { print $6 } ' | sort | uniq -c | sort -rn | head -n 10

Total number of requests per day: 

awk '{print $4}' access.log | cut -d: -f1 | uniq -c

Hourly number of requests per day:

  • If the log contains information for only one day:
    cat access.log | cut -d [ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c
  • If the log contains information for several days (instead of DD/Mon substitute the desired day of the month and the first three letters of the month name in English):
    grep "DD/Mon" access.log | cut -d [ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c

Per-minute number of requests for the specified hour of the specified day (instead of DD/Mon/YEAR:HH substitute the desired day of the month, the first three letters of the month name in English, year and hour): 

grep "DD/Mon/YEAR:HH" access.log | cut -d [ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c | awk '{ if ($1 > 10) print $0}'

Number of unique visitors: 

cat access.log | awk '{print $1}' | sort | uniq -c | wc -l

Top 25 URIs: 

cat access.log | awk '{ print $7 }' | sort | uniq -c | sort -rn | head -n 25

List of unique IPs: 

cat access.log | awk '{print $1}' | sort | uniq

List of unique IPs with date and time for each request from them: 

cat access.log | awk '{print $1 " " $4}' | sort | uniq

List of unique IPs with date, time and method for each request from them: 

cat access.log | awk '{print $1 " " $4 " " $6}' | sort | uniq

List of unique IPs with date, time and URI for each request from them: 

cat access.log | awk '{print $1 " " $4 " " $7}' | sort | uniq
Content