Networking

SolutionBase: Working with Windows Server 2003's IP Security Monitor

Use this free built-in utility to monitor IPSec on your network.


When Microsoft originally released Windows 2000, it included a new protocol called IPSec. The idea was that network security could be greatly increased by encrypting traffic as it flowed across the wire. Although IPSec was a good idea, the problem was that far too many administrators would enable it and never really do anything to confirm that IPSec was actually encrypting traffic. This was unfortunate because Windows 2000 included a utility called IPSECMON.EXE that made it easy to check if IPSec was working properly.

Over the years, I've grown quite fond of IPSECMON.EXE. I was really surprised to see that it didn’t exist in Windows Server 2003. Although IPSECMON.EXE is gone, Microsoft replaced it with a new tool called the IP Security Monitor Console. In creating this tool, Microsoft has basically rewritten IPSECMON.EXE to make it work within a console. It then added support for all of the new IPSec features that exist in Windows 2003. In this article, I'll introduce you to this new tool and show you how to use it to verify that IPSec is working as intended.

What’s new
Before you can really appreciate the IP Security Monitor Console, you need to see what its predecessor looked like. Figure A shows the IPSECMON.EXE utility that’s included with Windows 2000 Server. Notice that the utility contains no menu options and only two buttons: Options and Minimize. The only thing that you can do with the Options button is to change the refresh rate. My point is that although IPSECMON.EXE does a decent job of counting confidential and authenticated bytes, there really isn’t much more that you can do with it.

Figure A
The IPSECMON.EXE tool allowed you to confirm that IPSec was working, but was crude, to say the least.


Now that you've seen what the IPSECMON.EXE utility looks like, let’s take a look at the IP Security Monitor Console. As you can see in Figure B, the console looks a lot different than IPSECMON.EXE, but the differences aren’t just cosmetic.

Figure B
The IP Security Monitor Console has been extended to take advantage of all of the new IPSec features.


Earlier, I mentioned that the IP Security Monitor Console had been extended to support all of IPSec’s new features. There are too many new IPSec features to discuss here, but I wanted to at least take a moment and talk about some of the things that the IP Security Monitor Console does that IPSECMON.EXE didn’t.

As you look at the console, one of the first things you might notice is that just below the IP Security Monitor container is a server listed by name. The reason for this is that the IP Security Monitor Console can monitor the IPSec statistics for multiple computers rather than just for the local computer, as was the case with IPSECMON.EXE.

Another nice new addition to the console is the ability to view individual IPSec policies. You can use the console to view things such as the policy name, description, modification date, store, path, OU, and even the name of the group policy that it is being called from.

Yet another new feature is the ability to use DNS name resolution for filter and security association output. At first, this might not seem like a big deal. After all, if you look at Figure A, you'll notice that most of the host names are being resolved already. Therefore, DNS name resolutions might not even seem like a new feature. The difference, though, is that all of the host names you see in Figure A belong to hosts on my local network. The new IP Security Monitor Console allows you to resolve host names from across the Internet. This is important because Windows Server 2003 supports using IPSec over a NAT firewall.

One last new feature that I want to talk about is the filter search. As you probably know, IPSec policies are based on filters and rules. For example, a filter rule might dictate that traffic flowing from and addressed to a specific address must be encrypted. Although filters and rules tend to work well, the very nature of the Active Directory allows conflicts to occur. When conflicts occur between filters, one filter will take priority over another, and the lower-priority filter will be ignored. This can cause IPSec to behave unexpectedly.

If you notice IPSec acting strangely, you can use the new console to search for specific filters or rules. This allows you to locate the filter that is causing the unexpected behavior.

Accessing the IP Security Configuration Console
Now that I've shown you some of the differences between IPSECMON.EXE and the IP Security Monitor Console, I want to demonstrate how you can use the console to help monitor and enforce IP security within your own organization.

Begin by entering the MMC command at the Run prompt. When you do, Windows will load an empty Microsoft Management Console. Now, select the Add / Remove Snap In command from the console’s File menu. This will cause Windows to display the Add / Remove Snap-in properties sheet. Click the Add button found on the properties sheet’s Standalone tab to reveal a list of the available snap-ins. Select IP Security Monitor from the list of available snap-ins and click Add, followed by Close and OK. The IP Security Monitor should now appear within the console.

Adding a computer
Since the IP Security Monitor allows you to monitor multiple computers, the first thing you need to know how to do is add another computer to the list of systems to be monitored. To do so, right-click on the IP Security Monitor container and select the Add Computer command from the resulting shortcut menu. When you do, you'll see the Add Computer dialog box. It's important to note that the IP Security Monitor can only monitor computers running Windows Server 2003.

Monitoring a computer
After you finish adding the computers to be monitored to the list, it’s time to begin the actual monitoring process. Expand the computer container and, after a brief delay, you'll see three subcontainers:
  • Active Policy
  • Main Mode
  • Quick Mode

Active Policy
As you're no doubt aware, IPSec policies are applied as a part of a group policy. Furthermore, group policies are hierarchical in nature. Group policies can be applied at the local computer level, the site level, the domain level, and the Organizational Unit level. This means that several policies can be applied to a user or to a computer. As you might expect, it's possible for policies to contradict with one another. When group policies contradict, Windows uses an algorithm to determine which group policy is in effect.

What this means is that IPSec policies can be applied at many different levels as well. If you need to know what IPSec policy is currently active, you can work through the algorithm and determine the resultant set of policy, or you can just open the IP Security Monitor console and look at the Active Policy section.

The Active Policy section, shown in Figure C, contains all of the pertinent details regarding the policy that is presently in effect. As you can see in the figure, the Active Policy container lists the policy name and provides a description of what the policy is designed to do. You can also see the date of the most recent modification to the policy. Although they're not applicable to my current configuration, you can also see in Figure C that when appropriate, the Active Policy container also displays information regarding the policy path, organizational unit, and Group Policy Object Name.

Figure C
The Active Policy container shows which policy is in effect.


Main Mode
The next section is the Main Mode container. The Main Mode container’s job is to display various Internet Key Exchange (IKE) statistics. The Main Mode container is divided into five separate subcontainers: Generic Filters, Specific Filters, IKE Policies, Statistics, and Security Associations.

The Generic filters, shown in Figure D, contain a generic representation of the current IKE policy. For example, on my test machine, the generic filter is configured to use my computer as the source and any computer as the destination. The authentication method is Kerberos, and the filter applies to all connections. The policy that the filter links to is policy number 1, which is the machine’s default policy. I’ll show you the policies a little later.

Figure D
The Generic filter applies policy number 1 to all traffic coming from my computer.


The next container in the Main Mode area is the Specific Filters container. This is really just an expansion of the information found in the Generic Filters container. As you can see in Figure E, the specific filters actually list the machine’s IP address rather than just saying Me as the source. Furthermore, the Specific Filters container also shows the direction of the traffic. As you can see in the figure, there are actually two filters, one to handle inbound traffic and one to handle outbound traffic.

Figure E
The Specific Filters container shows which filters apply to the machine, but in more detail than the Generic Filters container shows.


Earlier, I mentioned that the generic filter (as well as the specific filter) was linked to the default policy, or policy number 1. That policy is contained within the IKE Policies container. The default view simply shows the policy’s number and the fact that it has four security methods associated with it. However, if you right-click on the policy and select the Properties command from the resulting shortcut menu, you can see exactly what the four methods consist of, as shown in Figure F.

Figure F
Each IKE policy consists of one or more methods.


I want to momentarily skip over the Statistics container and look at the Security Associations container. This container, shown in Figure G, shows the peer machines that the IKE policy is being used with. Basically, this screen means that communications between my machine (147.100.100.99) and the machines Relevant (147.100.100.58) and Homer (147.100.100.52) are encrypted by the current IKE policy.

Figure G
The Security Associations container shows which computers are using the IKE policy to secure communications.


The last Main Mode feature that I want to talk about is the Statistics container, shown in Figure H. As you can see, there are quite a few different statistics.

Figure H
The IP Security Monitor collects a number of IKE statistics.


The statistics you'll find include:
  • Active Acquire: This reflects the number of queued requests to initiate IKE negotiation in an effort to establish a secure connection. Under heavy loads, it’s normal for this number to be one higher than the actual number of queued requests.
  • Active Receive: This is the number of IKE messages that have been received and are queued for processing.
  • Acquire Failures: This reflects the total number of outbound acquire requests that have failed since the last time the IPSec service was started.
  • Receive Failures: This is the total number of error occurring while receiving IKE messages since the last time the IPSec service was started.
  • Send Failures: This is the total number of errors that have occurred while transmitting IKE messages since the last time the IPSec service was started. It is normal for this to be a high number if a dial-up connection is in use.
  • Acquire Heap Size: This is the number of successful acquires.
  • Receive Heap Size: This is the number of currently buffered inbound IKE messages.
  • Authentication Failures: These are the total number of times that authentication has failed since the IPSec service was last started.
  • Negotiation Failures: The total number of negotiation failures that have occurred since the IPSec service was last started. This counter tracks negotiation failures in Main Mode and in Quick Mode.
  • Invalid Cookies Received: This is the total number of cookies that couldn’t be matched to a Security Authority since the IPSec service was last started. In this case, cookies refer to an identifying value in an inbound IKE message.
  • Total Acquire: This is the total number of requests issued to IKE since the last time the IPSec service was started.
  • Total Get SPI: This is the total number of requests for a Security Parameter Index (SPI) made since the last time the IPSec Service was started.
  • Key Additions: These are the total number of outbound Quick Mode security authorities that have been added to the IPSec driver by IKE since the last time the IPSec Service was started.
  • Key Updates: These are the total number of inbound Quick Mode security authorities that have been added to the IPSec driver by IKE since the last time the IPSec Service was started.
  • Get SPI Failures: These are the total number of failed requests that have been sent to the IPSec driver by IKE since the last time the IPSec Service was started.
  • Key Addition Failures: These are the total number of failed outbound Quick Mode Security Authority addition requests sent to the IPSec driver by IKE since the last time the IPSec Service was started.
  • Key Update Failures: These are the total number of failed inbound Quick Mode Security Authority addition requests sent to the IPSec driver by IKE since the last time the IPSec Service was started.
  • ISADB List Size: This displays the total number of Main Mode negotiations (successful or not).
  • Connection List Size: This is the current number of Quick Mode negotiations that are in progress.
  • IKE Main Mode: This shows the total number of successful SAs that have been negotiated during Main Mode negotiations since the last time the IPSec Service was started.
  • IKE Quick Mode: This is the total number of successful SAs that have been negotiated during Quick Mode negotiations since the last time the IPSec Service was started.
  • Soft Associations: These are the total number of security associations formed with computers that failed to respond to Main Mode negation attempts.
  • Invalid Packets Received: This shows the total number of invalid or corrupt IKE messages that have been received since the last time the IPSec Service was started.

Quick Mode
Quick Mode works just like Main Mode except that Quick Mode deals with IPSec instead of IKE. You might have noticed that Main Mode has a container named IKE Policies, while Quick Mode has a container named Negation Policies. Although the policy types are different, the actual method for viewing the policies remains the same whether you’re in Main Mode or in Quick Mode.

The only real difference between Main Mode and Quick Mode (aside from the obvious) is the statistics that are displayed. You can see the Quick Mode statistics in Figure I.

Figure I
Quick Mode offers some statistics of its own.


The statistics you'll find include:
  • Active Security Associations: This is the number of currently active Quick Mode security associations.
  • Offloaded Security Associations: This shows the current number of Quick Mode security associations that have been offloaded to an IPSec-compatible NIC.
  • Pending Key Operations: This is the number of IPSec key exchange operations that are currently going on, but that have not yet completed.
  • Key Additions: These are the total number of successful key additions for Quick Mode security association negotiations that have been successfully added since the computer was last rebooted.
  • Key Deletions: These are the total number of successful key deletions for Quick Mode security association negotiations that have been successfully deleted since the computer was last rebooted.
  • Rekeys: These are the number of successful Quick Mode rekey operations since the last reboot.
  • Active Tunnels: This shows the total number of currently active IPSec tunnels.
  • Bad SPI Packets: These are the number of packets that have had an incorrect SPI since the last time the computer was rebooted.
  • Packets Not Decrypted: This shows the number of packets that could not be decrypted since the last reboot.
  • Packets Not Authenticated: This is the total number of packets that have failed integrity verification since the last reboot.
  • Packets With Replay Detection: This is the total number of packets with invalid sequence numbers since the computer was last rebooted. If this number increases steadily, it could be an indication of an attempted replay hack.
  • Confidential Bytes Sent: This shows the total number of sent packets encrypted by ESP since the last reboot.
  • Confidential Bytes Received: This is the total number of received packets encrypted by ESP since the last reboot.
  • Authenticated Bytes Sent: This is the total number of transmitted bytes encrypted by the AH or the ESP protocol.
  • Authenticated Bytes Received: This is the total number of received bytes encrypted by the AH or the ESP protocol.
  • Transport Bytes Sent: This shows the number of bytes sent using IPSec transport mode since the last reboot.
  • Transport Bytes Received: This is the number of bytes received using IPSec transport mode since the last reboot.
  • Bytes Sent in Tunnels:�This shows the number of bytes sent in IPSec tunnels since the last reboot.
  • Bytes Received in Tunnels: This is the number of bytes received in IPSec tunnels since the last reboot.
  • Offloaded Bytes Sent: This is the total number of bytes transmitted using IPSec hardware offloading since the last reboot.
  • Offloaded Bytes Received: This is the total number of bytes received using IPSec hardware offloading since the last reboot.

Editor's Picks

Free Newsletters, In your Inbox