Windows XP SP2: What to do when the Windows break

Learn about the problems that have been cropping up with Windows XP Service Pack 2, and get some solutions.

Windows XP Service Pack 2 (SP2) is a complex update with many ramifications for IT pros. TechRepublic's Windows XP Service Pack 2 Quick Guide drills down on critical SP2 need-to-know areas, with sections on fundamentals, changes that occur after installation, deployment procedures, problem areas, and removal.

Surprise, surprise. Windows XP Service Pack 2 has caused a rash of problems for a variety of users and organizations that have deployed it. For months before the release of XP SP2, Microsoft had been preparing people for the fact that SP2 was going to potentially cause some problems because of its new, tighter security restrictions. In my June 7 article, "Windows XP SP2 is big step forward in security—but it can break things," I alerted TechRepublic readers to many of the potential problems that SP2 was going to cause.

Nevertheless, shock and dismay have accompanied the daily barrage of reports of incompatibilities and software issues resulting from installations of XP SP2 since its release at the beginning of August. Let's take a look at some of the problems that XP SP2 is reportedly causing.

Known problems

Soon after Microsoft shipped SP2, it published Knowledge Base Article 842242, "Some programs seem to stop working after you install Windows XP Service Pack 2." This article includes a list of prominent applications that won't work correctly until the administrator tweaks either the application or the default XP SP2 settings.

Because of the gigantic size of this update, you may want to consider turning off automatic update features (or at least use the setting Notify Me Before Downloading Any Updates) simply because networks may experience a serious degradation of service while downloading the update. The Windows XP Home automatic update for SP2 is 80 MB and started automatically downloading on Aug. 18. However, Microsoft has delayed the release of the automatic update version of SP2 for Windows XP Professional. Users will probably have time to disable the automatic update feature if they haven't already.

Some of the problems that have cropped up for those who have installed XP SP2 include:

  • Some FTP clients will fail.
  • Streaming multimedia applications don't always work.
  • Some e-mail software won't properly update and show new mail.
  • There can be server-related problems (when running server functions), including a failure to recognize or reply to client requests. Look for problems with IIS and file sharing as well as some Remote Desktop functions.
  • One problem that is known to require an actual patch is Microsoft Business Solutions CRM Sales for Outlook 1.2.
  • There's a problem with Microsoft L2TP clients connecting to servers that use network address translation (NAT).
  • There are general problems that involve multiplayer games and instant messaging, but those shouldn't affect most business users.
  • German security firm Heise Security has discovered flaws in XP SP2, and it believes these flaws could lead to viruses and worms that might cause new havoc for Windows.

XP firewall issues

Many of the known application problems are related to the default activation of the Windows Firewall (also known as the Internet Connection Firewall) and simply require you to reconfigure the ICF to accept the application or manually open specific ports if ICF can't deal with the new application directly. There's a separate Knowledge Base Article (875357) that addresses ICF-related problems with XP SP2 and how to deal with them.

Many administrators may simply turn off ICF. In most corporate settings, there's already a network firewall, so there's no need for ICF. However, remote users, branch offices, and small businesses that don't already have a well-configured firewall should consider working with ICF, or else they'll simply toss away most of the security improvements included with SP2.

If you're lucky, ICF will present an error message when you try to run a program that isn't already configured to operate with a stateful firewall. This is the Windows Firewall Security Alert (FSA) giving you the option of quickly unblocking the application. Doing so may eliminate any future problems.

If you don't see the FSA dialog, you'll need to determine which ports should be open and reconfigure ICF to manually recognize your program. Microsoft provides the following instructions for doing so through ICF:

  • Click on Start, Run, and enter wscui.cpl.
  • Click Windows Firewall.
  • Go to the Exceptions tab and then to Add Program.
  • Select the program from the list if it appears there, click OK, and then confirm that the box next to the program is checked in the Exceptions dialog.

I suggest you make a list of the programs you've manually added so you can go back and uncheck them if you encounter problems with other applications. If you're able to fix a program this way, you don't need to know any additional technical details, such as port numbers used by the application. The ICF will automatically manage opening and closing the port, thereby increasing security.

If either the FSA dialog fix or the manual program configuration doesn't solve the problem, or if the program name doesn't appear in the Exceptions list, you'll need to manually configure the firewall. To do this, you'll need to know which ports the application uses.

For manual port configuration:

  • Run wscui.cpl to open Windows Firewall.
  • Go to Add Port on the Exceptions tab, key in the port number, identify whether it's TCP or UDP, and give it a name.
  • Click on the Exceptions tab to see whether the new service has been added. You'll still need to enable the port by checking the box next to the service.

If you don't know the port numbers and can't get them from the documentation or directly from the vendor, you'll have to monitor the program's activities when the program tries to operate normally.

Microsoft recommends that you use the command netstat –ano > netstat.txt to monitor the application. The a switch displays all listening ports and connections; the n switch shows the port numbers; the o will identify the program that's using the ports; and netstat.txt will be the file that all of this information is captured in. The Tasklist will show the process identifier; use tasklist /svc for services.

According to the Microsoft KBA 875357, the following programs are likely to require you to reconfigure ICF port permissions in order to run properly. Please note that this is not a complete list. I've included only the applications you're most likely to encounter:

  • Microsoft Visual Studio .NET
  • Microsoft SQL Server 2000a (ports 1433 and 1434)
  • Microsoft SMS 2003 Server (TCP 2701)
  • Microsoft Operations Manager 2000 SP1
  • Microsoft SNA 4.0 SP3
  • Attachmate KEA! 340 5.1
  • Attachmate Extra! Personal Client 6.5 and 6.7 (port 23)
  • Attachmate Extra! Enterprise 2000 (port 23)
  • Attachmate Extra! Bundle for TCP.IP 6.6 (port 23)
  • Autodesk AutoCAD 2000 (port 21)
  • Autodesk AutoCAD 2002 (port 21)
  • Autodesk AutoCAD 2004 (port 21)
  • Computer Associates ARCserve
  • Computer Associates eTrust 6.0.100 and 7.0
  • Macromedia ColdFusion MX SE 6 (port 8500)
  • NetManage ViewNow 1.0 and 1.05
  • Veritas Backup Exec 9 (port 10000), Exec 9.1.4691 (see documentation), and Volume Manager 3.1 (port 2148)
  • Symantec's Ghost Server Corporate Edition 7.5 and AntiVirus Corporate Edition 8.0 and 9.0

Final word

As of August 24, 2004, an online survey by the SANS Institute showed that 46 percent of respondents haven't had a problem with SP2 yet; 27 percent have had small problems; and 8 percent have had big problems that they could fix. Another 8 percent reported major problems they hadn't corrected, and 7 percent had to rebuild from scratch. Most troublesome to me are those who couldn't even revert to Safe Mode to fix a problem and had to completely rebuild their systems—at 7 percent, we're talking about a lot of systems worldwide.

It's normal to expect that the installation of a new software firewall will trigger problems with applications, which must respond to client queries or client software, which must get data from servers. This should be relatively easy for most administrators to deal with—they can simply turn off ICF, since they probably already have a network firewall. Also, administrators can look at their current firewall configuration and use those port settings to configure ICF for any workstations or laptops that are outside the corporate firewall.

A major security enhancement in XP SP2 (and the one that will directly affect administrators) is the way the update will block most worms infecting through buffer overruns. But hold on to your applause. That's a great advance but one that relies on the No eXecute (NX) feature, which will prevent any code from executing in protected memory areas. This means the buffer overrun will still occur, but the malware code will be pushed into a memory area where it can't do any damage.

The problem is that the vast majority of CPUs don't have the NX command. In fact, it's found only on fairly new AMD chips and some Intel Itanium server chips. For the moment, adding NX protection to XP is more of a theoretical help than a real improvement in security, but it could have a big impact the next time you upgrade your systems if NX gets implemented in more chips.

Another thing to remember is that ICF basically filters only the traffic coming into a system. You won't get any protection from keystroke-logging malware, which will still be free to send out reports from your system. A report on ZDNet (UK) also makes the interesting point that, since this is a Microsoft coding project, it may not be long before crackers discover a way to turn off ICF, modify its settings, or simply fake its error messages. Only time will tell, but major firewall vendors are already producing XP SP2-compatible firewalls that kill ICF when their commercial-grade firewall is installed.

If you've installed or experimented with Windows XP SP2 and have questions or would like to share what you've found, you can join this discussion in the TechRepublic forums.

Additional resources

Editor's Picks

Free Newsletters, In your Inbox