Inside your server request logs: Apache and IIS

Justin James takes a closer look at the data in your Apache and IIS server logs. Here's how to read them so you'll be able to troubleshoot more efficiently.

Recently, I wrote about things to look for in your server's logs. Today I want to take you on a walk-through of a typical server log file to show you what they look like and what data is inside of them. We are going to be looking at both Apache and IIS logs.

Apache error logs

Apache's error logs are very straight forward. They simply list a timestamp of the error, the error's importance, the client that had the error, and error text from the application that had the problem. For example:

[Sat Sep 10 06:07:24 2011] [error] [client] File does not exist: /usr/home/tcrowbar/public_html/stats/usage_201109.html

[Sat Sep 10 06:25:46 2011] [error] [client] File does not exist: /usr/home/tcrowbar/public_html/robots.txt

[Sat Sep 10 08:47:25 2011] [error] [client] File does not exist: /usr/home/tcrowbar/public_html/robots.txt

It is up to each application to write errors to the log and provide the necessary troubleshooting information.

Apache access logs

Here is where the real fun begins: the logs showing what requests are being made and the nature of those requests. In your Apache configuration you can specify what data the logs retain. You can use the LogFormat directive or the CustomLog directive. LogFormat sets the default log format and optionally allows you to give that format a nickname to be referred to by CustomLog. TransferLog accepts a file or pipe name to send the log data to, and the format used will be the most recent LogFormat entry. CustomLog allows you to specify a log format directly or use a named format created with LogFormat, and a destination for the log.

The log formats themselves are created with a string that you put different "placeholders" in that get replaced with the actual values. Some of the most useful placeholders are:

  • %a - IP address of the client
  • %h - Hostname of the client
  • %A - The local IP (useful when combining logs from different servers into one)
  • %B - Size of the HTTP response (in bytes), not including the HTTP headers themselves
  • %D - How long the request took to service (useful for discovering slow applications), in microseconds
  • %T - How long the request took to service, in seconds
  • %f - Requested filename
  • %U - The requested URL path (excluding query string)
  • %m - The HTTP method
  • %q - The query string of the request
  • %s - The HTTP status of the response
  • %t - The timestamp
  • %u - The username if the user was authenticated
  • %{XYZ}I - The XYZ header of the request (%{Referer}I for the Referer or %{User-agent}I for the User Agent, for example)

There are others as well, but these are the ones that you see the most often. The two most common, standard log formats are the common log format (CLF) and NCSA extended/combined.

CLF: "%h %l %u %t \"%r\" %>s %b"

NCSA extended/combined: "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""

I've found that NCSA extended/combined gives me everything I need in most scenarios.

IIS error logs

IIS does not have error logs per se, but ASP.NET will dump unhandled exceptions to the Event Log, in the "Application" log. IIS itself will report any issues there as well. In addition to the Event Log, you can use the request tracing functionality to see what is going on. For me, request tracing has not been terribly fantastic. Usually, it will point me in the right direction, such as letting me know the exact step of the process that is failing, but in true Microsoft fashion, the error codes tend either to be undocumented or documented in an unhelpful manner. All the same, the information in them, while it is rarely of the "this is your problem right here!" variety, is usually enough for a few searches to get your problem solved.

IIS access logs

Microsoft usually takes a beating on the topic of "following standards" but IIS' error logs are a refreshing change of pace. IIS supports a number of different log formats: IIS, NCSA common log, W3C extended log format, and Custom (which go to an ODBC database). W3C and Custom allow you to select the fields that you want to use with a simple list of checkboxes. The IIS format and NCSA common log are good choices because they are well known and many tools use them. W3C Custom has the advantage of listing the field names at the beginning of each change to the logs (such as when the fields selected change, or the server is restarted), so tools can adapt to the contents automatically, with no configuration. And logging to an ODBC data source has the advantages that you would expect, such as being searchable and usable with tools you already have and know how to use.

The IIS log format records the following:

  • IP address of the client
  • The username if the user was authenticated
  • The date of the request
  • The time of the request
  • Service and instance
  • Hostname of the server
  • IP address of the server
  • How long the request took to service, in milliseconds
  • Size of the entire HTTP request, in bytes
  • Size of the entire HTTP response, in bytes
  • The HTTP status of the response
  • Windows status code ("0" meaning "OK")
  • The HTTP method
  • The requested URL path (excluding the query string)
  • The query string of the request