SolutionBase: Protect your workstation with Windows XP SP2's Data Execution Prevention technology

If the computer isn't capable of providing hardware-enforced DEP, you'll see a notice to that effect at the bottom of the Data E

Hardware-enforced DEP

As its name implies, hardware-enforced DEP is based on logic built into the processor and is designed to monitor all application code as it is being loaded into memory. More specifically, hardware-enforced DEP keeps track of each memory location that an application uses and marks those locations that do not specifically contain executable code with a special attribute or flag. With those flags set, the application is free to perform its tasks and the processor goes about its business as it normally would.

However, if malicious code tries to sneak an unauthorized executable operation into an available, but flagged memory location, the processor raises an exception. When this occurs, the malicious code is intercepted and rejected and the potential attack is averted.

This same series of events is also triggered if an authorized program goes awry and inadvertently attempts access a memory location not available for executable code. In addition to a program going awry, there are some legitimate programs, typically legacy or poorly written programs, that may attempt use a memory location not available for executable code. In both of these cases, the program would be prevented from running any further and would fail--usually resulting in the Blue Screen of Death.

As I mentioned in the introduction, both the newer AMD and Intel processors have hardware-enforced DEP built into them. More specifically, the hardware-enforced DEP in AMD processors is called the no-execute page-protection (NX) processor feature while the hardware-enforced DEP in Intel processors is called the Execute Disable bit feature. Regardless of their different names, both architectures are Windows compatible and can work with the software-enforced DEP features built into Windows XP SP2.

Software-enforced DEP

Windows XP SP2's software-enforced DEP works similarly to the hardware-enforced DEP, in that it monitors memory locations for unauthorized access. However, SP2's software-enforced DEP is only configured to monitor the memory locations used by crucial operating system executables and services. By default, the hardware-enforced DEP component, which is accessible from Windows XP SP2, is not enabled.

This intended hobbling of the DEP technology was done in the name of compatibility in order to allow those legitimate legacy programs that don't adhere to the DEP code of behavior to run unimpeded by errors and failures. In other words, disabling hardware-enforced DEP helps to dispel the misguided belief that Windows XP SP2 breaks applications.

Checking hardware-enforced DEP status

If you're not sure whether the processor in a computer running Windows XP SP2 is capable of providing hardware-enforced DEP, you can easily find out by accessing Data Execution Prevention configuration tool in Windows XP SP2. To do so, press [Windows] [Break] to bring up the System Properties dialog box. Next, select the Advanced tab and then click the Settings button in the Performance section. When the Performance Options dialog box appears, select the Data Execution Prevention tab.

If the computer is not capable of providing hardware-enforced DEP, you'll see a notice at the bottom of the dialog box like the one shown here. In this case, the computer is only using SP2's software-enforced DEP.

By Greg Shultz

Greg Shultz is a freelance Technical Writer. Previously, he has worked as Documentation Specialist in the software industry, a Technical Support Specialist in educational industry, and a Technical Journalist in the computer publishing industry.