Windows Remote Desktop Protocol (RDP) hasn't always had the best reputation for security. But since FIPS (Federal Information Processing Standard) grade security was added to Windows Server 2003 SP1 (Service Pack 1), Windows Remote Desktop security has improved immensely. We'll step through the process of implementing FIPS-grade security whenever you Remote Desktop to a Windows Vista computer from a Windows XP or Vista client machine.
Editing the Group Policy Object
The first thing you need to do is edit the Group Policy Object by running gpedit.msc.
Accessing Security settings
Once inside the Group Policy Editor, navigate to Computer Configuration, Administrative Templates, Windows Components, Terminal Services, and then Security.
Setting the Encryption Level
Set the Encryption Level to High Level.
Enabling Secure RPC Communication
Set Require Secure RPC Communication to Enabled.
Setting Specific Security Layer For Remote (RDP) Connections
Set Require Use Of Specific Security Layer For Remote (RDP) Connections to SSL (TLS 1.0)
Accessing Security Options
Move to a different GPO section, at Computer Configuration, Windows Settings, Security Settings, Local Policies, and then Security Options.
Enabling FIPS mode
Select Enabled on the Local Security Setting tab.
Enabling Remote Desktop
Enable Remote Desktop from the System Properties Window. Note that you're setting it to allow any RDP 6.0 client rather than locking it down to permit only Vista clients.
Refreshing the Group Policy
When you've finished the configuration, refresh the Group Policy to implement the new settings without a reboot using a forced GPUpdate.
The update was successful.
Launching the RDP client
Launch the RDP client using the MSTSC command. Windows 2003 and XP users must download and install RDP 6.0 clients, whereas Vista comes with the correct client. On XP, you also need to launch the Run command before you can issue the MSTSC command.
Entering the server name
Enter the name of the server, noting that this initial process should happen on the LAN first. For this example, we're going to an RDP host machine called "msi-p965." This is not a fully qualified name, so it will work only on the same subnet LAN for now. (In a future gallery, we'll look at how to enter a redirect entry into the local host file pointing to an IP or dynamic DNS address, so that you can access "msi-p965" or whatever you call your machine from the public Internet.)
Now it's time to set the Remote Desktop Connections options.
The Display tab
Set the display to your liking.
The Local Resources tab
Specify whether you want sound, printers, or the Clipboard to work.
The Programs tab
Specify any programs you want to launch upon connection.
The Experience tab
Specify how you want the remote desktop to look using the settings shown here. The more features you add, the more bandwidth it takes.
The Advanced tab
Set the RDP client to warn you if the RDP server fails to prove its authenticity. You don't want to accidentally hand over your user credentials to a hacker who might be intercepting your connection.
Gateway Server Settings
Click Settings and configure the options as shown here. In this example, we're telling it not to use a TS Gateway server.
After you click OK, be sure you go back to the General tab and click Save As to save your entire profile. Otherwise, you'll have to do this whole procedure again next time. You can save it to the desktop for easy access.
Click Connect and you'll be prompted for your username and password.
The first time you connect, you'll see this authentication warning telling you that the server's certificate is not trusted (yet). To rectify this situation and force it to be trusted in the future, click the View Certificate button.
This self-signed cert generated by the Vista RDP host machine is valid for the next six months. Click on the Install Certificate button to add it to the CTL (Certificate Trust List).
Certificate Import Wizard
When the Certificate Import Wizard launches, click Next.
Certificate Store screen
Choose Place All Certificates In The Following Store and click the Browse button.
Selecting the certificate store
Select Show Physical Stores and highlight Local Computer.
The Certificate Store screen
Back in the Certificate Store screen, click Next.
Finishing the process
To complete the import, just click the Finish button.
When you see the success message, click OK. At this point, you'll be securely connected to the Vista RDP host, but more important, future connections to msi-p965 won't result in any warning signs or even password prompts. It will simply connect in a secure manner, and any warning signs must be viewed with a critical eye.
Trying to connect via IP address or a dynamic DNS entry from the public Internet
If you try to connect by any name other than the one you used to generate the certificate (in this example, it's "msi-p965"), you will see a warning. You can tell it to connect anyway and choose Don't Prompt Me Again For Connections To This Computer.
Proceeding with the connection
If you connect anyway, you'll see a warning like this one, but it's not a bad thing. You can view the certificate and it will say it's for "msi-p965" and that it's trusted.
Name mismatch and certificate errors
If a hacker poses as your server with a made-up certificate, you'll see this warning. Not only does the name not match, the certificate isn't even from a trusted certifying authority. If you see this kind of error when you've already gone through the certificate installation procedure, you know someone is trying to dupe you. You should click No and not connect to the server. If you attempt to make the connection anyway, you'll reveal enough of your credentials for the hacker to quickly run a dictionary attack to find your password.