Implement Web form tracking with ASP.NET

IIS can log a huge amount of Web usage data, but keeping robust logs without dragging down performance requires a skilled developer's help. Use ASP.NET to generate rich Web form tracking logs without unacceptably impeding performance.

Microsoft Internet Information Services (IIS) can record all Web site requests to a log file. These logs files have many uses, including security, performance, and load analysis by system administrators and analysis of visitor browsing patterns by marketing departments—but making this log information useful to Web clients requires some work on the part of the developer.

Recently, a client wanted to allow Web site users to view usage statistics in real time. Our solution was to write code that enabled every ASP.NET Web form to log every request to a database. Here's a look at why we took this approach and the details of our solution.

Logging setup
IIS can be configured to specify what information is going to be logged. Of course, the more data that is logged, the more hard drive space is consumed, so it is best to log only what will be used. Table A shows a full listing of the data eligible for logging, which is an extract from Microsoft IIS documentation. As you can see, there is a staggering amount of data that each server transaction can generate, and much of it would be of little use (and often incomprehensible) to an end user. The date, time, size, authenticated identity, and perhaps IP address or referrer are typically all a nontechnical user will want or need to see.

Table A
Field Appears As Description
Date Date The date that the activity occurred.
Time Time The time that the activity occurred.
Client IP Address c-ip The IP address of the client that accessed your server.
User Name cs-username The name of the authenticated user who accessed your server. This does not include anonymous users, who are represented by a hyphen (-).
Service Name s-sitename The Internet service and instance number that was accessed by a client.
Server Name s-computername The name of the server on which the log entry was generated.
Server IP Address s-ip The IP address of the server on which the log entry was generated.
Server Port s-port The port number the client is connected to.
Method cs-method The action the client was trying to perform (for example, a GET method).
URI Stem cs-uri-stem The resource accessed; for example, Default.asp.
URI Query cs-uri-query The query, if any, the client was trying to perform.
Protocol Status sc-status The status of the action, in HTTP or FTP terms.
Win32 Status sc-win32-status The status of the action, in terms used by Microsoft Windows.
Bytes Sent sc-bytes The number of bytes sent by the server.
Bytes Received cs-bytes The number of bytes received by the server.
Time Taken time-taken The duration of time, in milliseconds, that the action consumed.
Protocol Version cs-version The protocol (HTTP, FTP) version used by the client. For HTTP this will be either HTTP 1.0 or HTTP 1.1.
Host cs-host Displays the content of the host header.
User Agent cs(User-Agent) The browser used on the client.
Cookie cs(Cookie) The content of the cookie sent or received, if any.
Referrer cs(Referer) The previous site visited by the user. This site provided a link to the current site

IIS log accessibility
Although the use of IIS logs is highly encouraged, there are times when developers do not have access to them. This may happen when a developer is hosting a Web site on a shared Web server (shared hosting). Shared hosting is fairly commonplace, especially among fledgling Internet sites. One IIS server can host multiple sites.

System administrators typically do not, and should not give developers access to log files. The reason for this should be obvious—the log files will contain the logging information of other sites on the server. The log files have to be filtered before the information is given to a developer. This presents a problem if the developer needs real-time access to the files.

Here's another case where IIS logs are not readily accessible. This time the configuration is the exact opposite. Instead of hosting multiple sites on a Web server, a configuration may exist that hosts one site on multiple servers. This is known as a Web farm and is used for high volume sites. In this scenario, each IIS server may have its own log files. To access log information, the IIS logs must be consolidated into one comprehensive log.

In both situations, it may be desirable to log requests to the database, since this will provide a more timely, consolidated data view. That was the case with the client who wanted to allow Web site users to view usage statistics in real time. To meet this requirement, we developed generic code that would enable every ASP.NET Web form to log every request to a database. Needless to say, such a solution would obviously entail some degree of performance degradation, since every page request would issue a database insert. However, we optimized the code as much as we could to try to mitigate the performance hit. It's also important to note that logging requests to a database will result in duplicate copies of the information if IIS is also logging the data to a file. When comparing the IIS logs to the database logs, it's important to watch for activity overlap and, above all, log discrepancies.

Building a solution
There were several possible ways to approach this project. One method was to write a custom Web request module. This would perhaps be the most complex approach and was overkill for this scenario. ASP developers would more than likely resort to an include file to accomplish this. But ASP.NET has no support for include files. An alternate approach uses the event that fires whenever a Web form is requested. There are four events, which fire in the following order:
  • Init
  • Load
  • Prerender
  • Unload

The Init event fires first, so we could easily inject code into this event to log to a file or, in this case, a database. The event code may resemble the following:
Private Sub Page_Init(ByVal sender As System.Object, ByVal e As System.EventArgs)
Handles MyBase.Init
'Code to log to file or database could go here but
'the problem with this approach is that you would have to insert
'the code into EVERY single Web form.

End Sub

The comments above describe the problem with this approach: It is cumbersome. A more elegant solution would be to leverage inheritance.

Every Web form inherits from the System.Web.UI.Page class by default. All we needed to do was place an extra layer between the derived Web form and its base class. We created a class called PageTrack that inherits from System.Web.UI.Page. The code for logging was located in PageTrack. Every Web form could then be modified to inherit from PageTrack instead of System.Web.UI.Page. Through inheritance, the logging code would automatically be included. Here's the PageTrack code:
Public Class PageTrack
Inherits System.Web.UI.Page
Private Sub Page_Init(ByVal sender As System.Object, ByVal e As System.EventArgs)
Handles MyBase.Init
'Code to Log to file or database could go here
End Sub
End Class

Remember that this class should be placed in a .NET class library project to be reusable across Web applications. Furthermore, through .NET interlanguage compatibility, you can create PageTrack in VB.NET and use it in a C# Web application.

Log those requests
Logging straight to a database can be an effective technique for making log information instantly accessible to applications. However, if you decide to use this technique, you should make every effort possible to optimize the logging code to minimize the inevitable performance degradation. Furthermore, you should remember that you may have duplicate copies of the logging information: The IIS copy and the database copy.

Editor's Picks