SolutionBase: Improving Windows Server 2003 memory usage

RAM may be getting cheaper all the time, but managing it in servers is still an important issue. Brien Posey shows you how streamline memory usage in Windows Server 2003, including how to make virtual memory usage more efficient.

Does anyone other than me think that the concept of virtual memory (using hard disk space to make up for what the system lacks in RAM) is archaic? Ten years ago, when memory was $100 per megabyte, virtual memory was a true necessity. Today though, you can walk into any electronics store in America and get a PC with over a gigabyte of RAM. Physical memory is no longer in short supply, and yet the Windows operating system still uses virtual memory.

Technically speaking, I guess virtual memory isn't a true requirement since you can disable it. However, Windows systems tend not to run very well without the aide of virtual memory. In fact, if you configure the machine's virtual memory incorrectly, you can encounter some serious stability problems. In this article, I will talk about how Windows uses virtual memory and how you can go about configuring Windows Server 2003 to make the best use of your machine's physical and virtual memory.

A crash course in virtual memory

Virtual memory was invented at a time when physical memory was very expensive. The idea was to use hard disk space to supplement the amount of memory that's available to the system. Although this process works, using virtual memory is extremely inefficient for several reasons.

For starters, Windows can not process data that's residing in virtual memory. If Windows needs to do something with data that's in virtual memory, it must move the page containing the needed data from virtual memory into physical memory. The problem is that the physical memory may already be full. Therefore, Windows has to use a technique called paging. Paging is the process of locating a page of data in physical memory that is not currently being used and transferring it to virtual memory to make room for the needed page to be moved from physical memory to virtual memory.

Paging really slows down a system. Windows must use CPU cycles and even a portion of RAM just to manage the paging process. Furthermore, hard disk access times are measured in milliseconds, as opposed to RAM access times which are measured in nanoseconds. To put it simply, the hard disk is a whole lot slower than RAM, and the system wastes a lot of resources and takes a lot of time to move pages of data back and fourth between memory and virtual memory.

If a system becomes too low on RAM then thrashing can occur. Thrashing is a term for nearly constant paging. You have probably seen at least one situation in which a system's hard disk was running constantly with no visible results and a very sluggish response time. Chances are that this system was probably thrashing.

As much as virtual memory usage tends to slow down your system though, Windows is designed to use it. Windows uses a file named pagefile.sys for virtual memory. By default, this file (and therefore the system's virtual memory) is set to 1.5 times the size of the system's physical memory. For example, if a server has a gigabyte of RAM, then the initial pagefile size will be one and a half gigabytes.

Notice in that paragraph that I said that the initial size of the page file will be a gig and a half. If the system has insufficient memory, or if your server is running demanding or leaky applications, then Windows may expand the size of the page file to prevent the system from running out of memory. This is where things get tricky. There is a very fine balance between not giving Windows enough space for paging, and giving the operating system too big of a page file.

Setting the page file size

As you read the remainder of this article, you may find yourself wanting to make adjustments to the Windows pagefile. Therefore, I want to show you up front how to change the pagefile size. For the purposes of this article, I am assuming that you are running the 32-bit version of Windows Server 2003.

To set the size of the Windows page file, select the System option from the Windows Control Panel. When the System Properties sheet appears, select the Advanced tab. At this point, locate the tab's Performance section and click the Settings button. This will cause Windows to open the Performance Options properties sheet. Select the properties sheet's Advanced tab and click the Change button. You will now see the dialog box shown in Figure A.

Figure A

The Virtual Memory dialog box allows you to set the size of the system's page file.

If you look at the figure, you will notice that all of my system's drives are listed, but that the page file exists on the C: volume. If you are really interested in boosting your system's performance, then C: is usually not the best place for the page file. You want to put your page file in the place where Windows will be able to read and write data with the greatest throughput. There are a few things to consider when choosing a location. For example, you should consider the speed of the hard disk, but also how heavily that disk is used. Having a fast drive does you no good if it is already in constant use. Keep in mind that there is a big difference between hard drives and volumes on a hard drive. It will do you little good to put a page file on a seldom used volume on a high speed drive if the drive contains another volume that's heavily used.

Windows will actually allow you to get creative with virtual memory usage. You could for example, place three small page files on three different volumes rather than having one large pagefile. Windows generally tends to perform better if you use a single pagefile though. I have also known people to span a pagefile across a stripe set for better performance. This will work, but you need to make sure that the stripe set is accessible during the boot phase.

Just below the listing of the system's volumes is an area for you to set the minimum and maximum page file sizes. The minimum size should be 1.5 times the size of the physical memory. The maximum size can usually be set to about double the physical RAM, but demanding or leaky applications can cause you to need a larger pagefile. Don't forget to adjust the minimum page file size if you add memory to your server.

Calculating virtual memory needs

Windows includes a tool called the Performance Monitor that you can use to determine how well various parts of the system are performing. There are lots of ways that you can use the Performance Monitor to determine how well the currently allocated virtual memory is meeting the system's needs. Performance Monitor offers hundreds if not thousands of different performance counters, and many of these can be used in some way to gauge memory and virtual memory performance. However, I am a big believer in keeping things simple when you can. It is possible to determine how well the currently allocated virtual memory is meeting the system's needs by watching just a few different Performance Monitor counters.

I recommend beginning by opening the Performance Monitor (it's on the Administrative Tools menu) and using the X icon to remove the default counters. After doing so, click the + icon to access the Add Counters dialog box, shown in Figure B.

Figure B

You can use the Add Counters dialog box to control which counters Performance Monitor watches.

The Add Counters dialog box is used to select the Performance Monitor counters that you want to watch. To select a counter, start by selecting the appropriate Performance Object. The performance object is basically just a category of counters. Once you select the correct performance object, select the desired counter and click Add. Once you have added the necessary counters to the Performance Monitor, click close.

The first two counters that you should monitor are the Available Bytes counter (in the Memory performance object), and the % Usage counter (in the Paging File performance object) counter. Once you add these counters, the Performance Monitor will look something like what you see in Figure C.

Figure C

You should start by monitoring the Memory / Available Bytes and the Paging File / % Usage counters.

The Available Bytes counter tells you how much RAM the system presently has to work with. The number that the Performance Monitor gives you is in bytes. Therefore, you will have to divide the number by 1024 to get the number of available kilobytes, and divide by 1024 again to get the number of available megabytes. Your system should never, under any circumstances, have less than four megabytes of available memory. Four megabytes or less indicates a serious problem and you need to add memory to the server or free up some memory by shutting down some services.

The % usage counter indicates the percentage of your pagefile that is currently being used. Around 40% to 50% is usually a good number. If eighty percent or more of the pagefile is in use then you probably need to add more memory to the server and / or increase the size of the page file. On the flip side, if only about 20% or less of the page file is in use then you are probably wasting system resources with an unnecessarily large page file.

If the server that you are working with happens to be running Microsoft Exchange Server 2003, then there are a few other counters that I also recommend watching. The reason for this is because Exchange Server is a particularly demanding application and is notorious for fragmenting the pagefile. In an Exchange environment, it doesn't matter how much space within the pagefile is free if the pagefile is fragmented and does not have large enough blocks of free space to accommodate the necessary data. Exchange Server 2003 is less prone to pagefile fragmentation than Exchange 2000 was, but these counters are still worth keeping an eye on.

The counters that you will want to focus on are all found in the MSExchangeIS performance object. The counters themselves are VM Largest Block Size, VM Total 16 MB Free Blocks, VM Total Large Free Block Bytes.

The VM Largest Block Size counter indicates the size of the largest free block of virtual memory. This value should never fall below 32 MB. The VM Total 16 MB Free Blocks counter tells you how many blocks of space within the page file are at least 16 MB in size. This value should never fall below three. The VM Total Large Free Block Bytes counter provides you with the sum of the available bytes found in free blocks of space that are 16 MB or larger in size. This value should never fall below 50 MB. If any of these counters exceed their threshold values then you should add some memory to the server and increase the size of the pagefile.

Using the /3GB switch

So far I have talked mostly about the way that Windows uses virtual memory. However, the way that Windows uses virtual memory is a direct reflection of the way that Windows is using physical memory. By default, Windows uses something called the 2 GB memory model. As you probably know, 32-bit operating systems can address up to 4 GB of RAM. In Windows, the 4 GB address space is split evenly into two 2 GB address spaces. 2 GB of address spacer is used by the Windows operating system and 2 GB is used for user mode processes (applications).

You might have seen some Windows Server deployments in which there was a /3GB switch used in the server's BOOT.INI file. What the /3GB switch does is to allocate 1 GB of address space to the Windows operating system and 3 GB of address space to user mode processes. This allows Windows to better accommodate demanding applications such as Exchange Server.

Before I go on, I want to point out that the /3GB switch should be used sparingly. After all, Microsoft implemented the 2 GB memory model for a reason. There are consequences to depriving Windows of a gig of address space. Windows relies on a mechanism called Page Table Entries (PTEs) for allocating memory to the operating system and to the applications running on it. If you use the /3GB switch, Windows forfeits most of its PTE space. This isn't usually a problem if you are running a single, high demand application (such as an Exchange Server that is hosting 50 or more mailboxes). However, if you are running multiple applications or if your applications are not extremely demanding then you should stick to using the 2 GB memory model. Otherwise, there is a good chance that you could run the system out of PTEs. When available PTEs start getting low, Windows starts becoming instable. Microsoft recommends that you never use the /3GB switch on a domain controller or on a Small Business Server.

In case you are wondering, there is a way to find out how many PTEs your system has available. You can use the Performance Monitor's Free System Page Table Entries counter (found in the Memory performance object). The PTEs should never drop below 7,000. As the number of available PTEs approaches 7,000 the system becomes less stable.

So what happens if you need the /3GB switch, but you are running low on PTEs? Well, there is a trick that you can use to allocate more PTEs to the system, but this technique is not supported by Microsoft. To increase the number of PTEs, you can add the /USERVA=3030 switch to the BOOT.INI file. This will allocate an additional 42 MB of space to the page table entries.

You can experiment with the number assigned to the USERVA switch. The lower the number, the more space that gets allocated to the page table entries. There are five fundamental rules that you must abide by when using the USERVA switch:

  • Don't use the USERVA switch unless you have to
  • Don't expect Microsoft (or me) to assist you with the USERVA switch
  • Never use a value higher than 3030
  • Never use a value lower than 2800
  • Expect the unexpected until you find the value that seems to work for your system