ISA Server represents Microsoft’s first attempt at a true enterprise-level firewall product. Most of what you read about ISA Server focuses on the new (and cool) firewall features. You don’t hear much about the Web caching component of ISA Server because most people are more familiar with the Web caching features provided by Microsoft’s Proxy Server 2.0. But Proxy Server 2.0’s caching capabilities are old school next to the ISA Server’s new caching utility.

In this Daily Drill Down, I’ll go over some of the basics of how Web caching works and look at ISA Server Web cache configuration and management. By the time you finish this article, you’ll be ready to make the ISA Server Web caching facility do what you want it to do.

How Web caching works
Web caching improves end users’ Web browsing experience by storing Web information locally, allowing users to access information at the full speed of your LAN rather than having to share a much narrower Internet connection pipe.

Web caching works by storing Web objects requested by users in either a memory or disk cache. When subsequent users visit the same site or request the same Web objects, ISA Server intercepts the request and determines if the requested Web object exists in the Web cache. Each time ISA Server sends a requested object to the user from it’s local cache rather than having to pull the object across the organization’s WAN, your company saves money, because the cached object represents bandwidth that can be used for other business functions.

If the Web object is in cache, and the Time To Live (TTL) on the Web object has not expired, the object will be returned to the user from cache. If the object is missing from cache, or the object has expired, then ISA Server will obtain the object from the Web server on the Internet. From there, ISA Server places the object in the memory or disk cache and returns the object from the cache to the user. Note that the object is always returned from the cache.

For example, let’s say a user on your network visits a Web page that has text and a single graphic. The text is considered one Web object, and the graphic is considered a second object. When a second user on the network requests the same Web page, ISA Server checks its cache and returns the text page and graphic to the second user. The same events take place when subsequent users attempt to access the same page.

Now imagine that the Webmaster has set an expiration date on the text page so that it expires in seven days and has set a separate expiration date on the graphic so that it expires in 30 days. When a user requests the Web page eight days from now, ISA Server first checks the cache to see if the Web objects have expired. ISA Server will only pull the text portion from the Web site because the graphic has not yet expired. ISA sends the graphic from its cache to the user and then sends the test after retrieving it from the Web site.

ISA Server retains most of the caching functionality that was available in Proxy Server 2.0, although the Web caching feature has been redesigned to make room for improvements. So if you have experience with Proxy Server 2.0, you’ll have a leg up on how the ISA Server Web caching feature works, and you’ll probably be more aware of the upgrades as well.

Some people believe that Web caching does not help much for small networks. But I’ve found from log evaluations and analysis that even the smallest shops with just a handful of users can save dozens of megabytes of traffic a day on their Internet connections by taking advantage of ISA Server’s cache. Performance enhancements are quite noticeable among the end users.

ISA Server cache configuration
When you install ISA Server in cache or integrated mode, ISA Server’s Setup program automatically sets the cache configuration for you. During installation, you choose the drive letter(s) and size(s) of the cache file(s). In contrast to Proxy Server 2.0, which saved the contents of the cache as discrete files on the cache disk, ISA Server stores all of the cached information as a single database file that resides on the drives that are configured for use by the cache.

A single cache file can be up to 10 GB in size. If you configured a larger cache size on a particular drive, a second cache file will be created. However, there are few situations where you would require a cache file larger than 10 GB.

You can also configure the size of the Web cache after installation. Cache configuration options available after installing ISA Server include:

  • Cache size and location.
  • HTTP caching options.
  • FTP caching options.
  • Active caching options.
  • Advanced caching options.

Let’s look at each of these in more detail.

Cache size and location
To configure cache size and location, open the ISA Management console by clicking Start | Programs | Microsoft ISA Server | ISA Management. Expand your server or array name and expand the Cache Configuration node. Click on the Drives node in the left pane.

In the right pane of the console, you’ll see your server name and some stats on the current cache configuration. Double-click on the server name. This opens the server Properties dialog box and the Cache Drives tab. You can select any of the NTFS drives that appear in this dialog box.

You’ll see information regarding the total disk size, as well as the amount of free space available for the cache. Note that you can only use NTFS partitions to store the cache file. To set a cache size, select a drive, enter a value expressed in megabytes in the Maximum Cache Size (MB) text box, and click Set, as shown in Figure A.

Figure A
By using this text box, you can control cache drive locations and cache sizes.

Click Apply, and you’ll see the dialog box shown in Figure B. If you’re in a hurry, you can choose the default and restart the Web proxy service manually. Otherwise, select the second option to automatically restart the Web proxy service. It may take a few seconds to a few minutes before the service is restarted, depending on how busy ISA Server happens to be at the time.

Figure B
ISA Server warns you that you must restart the Web proxy service after making a change.

You can see the cache file using Windows Explorer on your server. Navigate to the drive where you stored the cache file, and you’ll see a screen similar to the one in Figure C. All the Web objects are stored in the ISA Server .cdat file. This database file format makes the disk-based cache more efficient and faster than Proxy Server 2.0’s flat-file method of storing Web objects.

Figure C
You can view the cache file in Windows Explorer.

HTTP caching options
You can view HTTP caching options by right-clicking on the Cache Configuration node and clicking Properties. The first tab you’ll see is the General tab, which contains basic information about the current cache size and location, as shown in Figure D.

Figure D
The General tab contains the size and location of the cache.

You can configure the HTTP caching options by clicking on the HTTP tab, as shown in Figure E. The Enable HTTP Caching option must be selected in order for ISA Server to perform Web caching.

Figure E
Make sure the Enable HTTP Caching check box is selected.

The options under Unless Source Specifies Expiration, Update The Object In Cache are often misunderstood. ISA Server only refers to these options when the Web server does not return an expiration date for the Web object. If the Webmaster was kind enough to configure his or her Web page in such a way as to tell your proxy server when it should fetch a fresh version of the Web page, these options are ignored. However, not all Webmasters are so considerate.

If a Web object doesn’t contain an expiration date, it usually includes a creation or modification date. This information can be used to determine how long the object remains valid in cache.

For example, I have selected the Less Frequently option. This keeps objects with no expiration date in cache for 40 percent of the content age. If the object was created 10 days ago, ISA Server could continue to return the object from cache for four days, after which time it will return to the Web server to see if there is a more current version of the Web object.

Note that even though the 40 percent in this example represents four days, the fact is that the object will only be returned from cache for two days because the No More Than setting is set to its default setting of two days. This is the hard-coded value for the Less Frequently option, as you can see in the grayed out areas of the Time To Live setting.

You can configure the objects without expiration dates to expire more frequently or normally. The more frequently you update objects in the cache, the more bandwidth will be consumed on the external interface. On the other hand, you’ll get more recent versions of Web pages that don’t contain expiration information in their HTTP headers.

FTP caching options
Click on the FTP tab to get the FTP caching options shown in Figure F. FTP caching allows you to determine how long you want files downloaded via FTP to be stored in cache before they expire. FTP objects, unlike HTTP objects, do not contain expiration dates or times in their headers. You must always manually configure an expiration date for these objects.

Figure F
ISA Server can cache FTP requests as well as HTTP requests.

Note that not all FTP downloads are placed in the Web proxy cache. Only files that have been downloaded by the Web proxy service will be stored in cache. Only Web proxy clients can send FTP requests to the Web proxy service. If you use the command-line FTP, or a dedicated FTP application, you will not access the objects through the Web proxy service, and the objects will not be cached.

Active caching
If you click the Active Caching tab, you’ll see the screen shown in Figure G. Active caching is an extension of the HTTP Web cache service. The feature has a single purpose: to improve the perceived Web browsing experience for end users.

Figure G
Active caching can increase performance for end users.

Active caching takes more bandwidth than normal caching, because the active caching feature allows ISA Server to download updates to Web pages that are considered “popular” before that Web object expires. The risk you take with active caching is that it will update Web objects in the cache that will never be viewed again by users.

Active caching consists of two processes: placing objects in an update list and updating those options in the list. The options have the following frequency values:

  • Frequently = 1
  • Normally = 2
  • Less Frequently = 3

When you enable active caching, ISA Server will respond in a specific way when a user requests an object. ISA Server will keep and cache the object in its update list, therefore continually updating it, for a period of time equal to the number of its frequency value (listed above) multiplied with the object’s TTL. For example, in the example above, the Normally option is selected. If the Web object has a TTL of 24 hours, then the object will remain in the update list as long as someone requests the object at least once within the next 48 hours (2 x 24).

Objects on the update list are updated based on the amount of system load:

  • System load 0-25 percent: Updates all objects that have passed 50 percent of their TTL
  • System load 25-75 percent: Updates all objects that have passed at least 75 percent of their TTL and are scheduled to expire in the next 7.2 minutes
  • System load 75-100 percent: Updates objects that have passed at least 95 percent of their TTL and are scheduled to expire in the next 3.6 minutes

You should use active caching very judiciously. If you choose an active caching scheme that is too aggressive, you might end up getting a call from your ISP about the large amount of activity coming from your ISA Server.

If a large number of objects being requested have a low TTL (such as news, sports, and financial sites), you can end up wasting a lot of bandwidth on the external interface. If you choose to enable active caching, you might want to review your Web proxy logs to see what types of sites are frequently access by your users and use the settings on the HTTP tab to manually set a longer TTL on objects without an explicit expiration date/time.

Advanced caching options
Click the Advanced tab to access the advanced caching options, as shown in Figure H. This tab is the most complex and the easiest to get lost on.

Figure H
The Advanced Cache Configuration Properties tab gives you a lot of caching flexibility.

The Do Not Cache Objects Larger Than option allows you to set a limit on the size of objects placed in the cache. This allows you to store more objects in the cache and is especially helpful if your system has a limited amount of RAM.

I was not able to find any documentation on the positive or negative effects the Cache Objects That Have An Unspecific Last Modification Time option has on cache performance. I would avoid choosing it for precisely that reason.

The Cache Objects Even If They Do Not Have An HTTP Status Code Of 200 allows the ISA Server Web cache to perform negative caching. This allows ISA Server to cache a negative response, such as HTTP errors 203, 300, 301, 410. This negatively cached entry will remain in cache for the period of time configured in the If Web Site Of Expired Object Cannot Be Reached options.

The Cache Dynamic Content option allows you to cache the content obtained via database searches. These sites have question marks (?) in the URL. Generally, it isn’t a good idea to cache these objects because the contents of a database change either frequently or in an unpredictable manner.

However, in certain circumstances, you might want to enable this option if you are limiting users to a small number of sites and these sites have relatively static databases. If you enable this option, the other cache configuration options still apply, and most sites will set a short expiration on the page or the modification time will be such that the page will expire within a few minutes.

The If Web Site Of Expired Object Cannot Be Reached options allow you to configure how ISA Server should respond when objects in the cache have expired and the Web server cannot be contacted for an update. If you select Do Not Return The Expired Object, ISA Server will return an error page to the user. If you choose Return The Expired Object Only If Expiration Was option, you can set the period of time that an expired page can be returned to the user. In the example above, I have set the At Less Than This Percentage Of Original Time To Live option to 50 percent and the But No More Than option to 60 minutes. So, if the original TTL was one hour, ISA Server would return the expired object in cache for 30 minutes. If the Web server couldn’t be contacted after 30 minutes, ISA Server would issue an error to the user.

Users can override the cache settings on the client side by manually refreshing the page by clicking the Refresh button in the browser, or by pressing the F5 key on the keyboard. In fact, users should be encouraged to refresh pages if they suspect that they don’t have the most current version of the page.

While the ISA Server caching feature doesn’t get a lot of attention, it does offer a major improvement over the caching functionality provided by Proxy Server 2.0 Web caching. It also significantly improves the Web browsing experience for internal network users and can also reduce the amount of bandwidth used on the external interface of ISA Server. Those are a lot of good features for a cost-reducing tool.