SolutionBase: Diagnosing File Replication Service problems using Ultrasound

Fix File Replication Service problems with Ultrasound.

One of the more essential components of an Active Directory domain is the File Replication Service (FRS). When information is updated on a domain controller, the FRS makes sure that the information is replicated to other domain controllers within the domain. The File Replication Service is also used to keep replicas within a Distributed File System Tree in synch.

The FRS usually works quietly, behind the scenes. However, as with any other complex operating system component, things can and do occasionally go wrong. One tool that you can use to help to diagnose FRS-related problems is Ultrasound.

Acquiring Ultrasound
Ultrasound is available free from the Microsoft Web site. The Ultrasound setup file is a 3-MB, self-extracting, executable file.

Before I explain how to install Ultrasound, I want to take a moment to discuss the hardware and software requirements. As you probably know, Ultrasound is designed to monitor the various FRS replicas. At a minimum, each replica must be running Windows 2000 SP2 with the fix described in Microsoft Knowledge Base article number 322141 installed. This particular operating system / service pack / hot fix combination is barely adequate for running Ultrasound, however. Later operating systems and service packs correct a number of bugs and performance issues that will otherwise plague Ultrasound. The tool runs best under Windows 2000 SP4 or under Windows Server 2003.

As you consider the software requirements, you need to consider that one of your servers will have to be running the Ultrasound console. The Ultrasound console has some fairly hefty requirements. For starters, the machine must have a minimum screen resolution of 1024 x 768.

Additionally, the server must be running the .NET framework and a database engine. The .NET framework is simple to install, but installation can take a long time to complete.

As for a database engine, you can use either the Microsoft Database Engine (MSDE) or SQL Server 2000 with Service Pack 3A. MSDE is completely free, but has some limitations that SQL Server doesn't have. MSDE is acceptable if only a few people will be using Ultrasound. If Ultrasound will be heavily used, though, you're better off using SQL Server. For the purposes of this article, I'll use SQL Server 2000. If you want to use MSDE instead, you can download it from Microsoft.

You must also consider the amount of disk space that will be used on your server. The initial database size is 5 MB. However, this database will grow at a rate determined by the number of servers that you're monitoring and the level of activity on those servers. If you've decided to use MSDE, then keep in mind that MSDE has a 2-GB maximum database size. Additionally, the server will need 2.5 MB for the Ultrasound console, 1.5 MB for the Ultrasound controller, and 400 KB for the provider.

Installing Ultrasound
Assuming that you have already installed SQL Server (or MSDE) and the .NET Framework, begin the installation process by double-clicking on the Setup_Ultrasound file. When you do, Windows will launch the Setup wizard. Click Next to bypass the wizard's Welcome screen and you will be asked to accept the end user license agreement. Accept the license agreement, click Next, enter your name and your organization's name, click Next, and then tell Setup that you want to perform a complete installation.

Click Next and Setup will ask you to enter the name of your database server and the database. Enter either the name of your SQL Server or the name of an instance on your SQL Server. After doing so, click Next and Setup will create the necessary databases. Click Finish to complete Setup.

Using Ultrasound
When you launch FRS for the first time, you'll see a message asking if you want to add FRS replica sets to Ultrasound. Remember that since Ultrasound has never been used before, the database is completely empty. Therefore, be sure to click Yes. When you do, Ultrasound will display a screen that asks you which FRS replica sets you want to add. This screen contains a drop-down list that you can use to select the various domains.

When you select a domain, the FRS replica sets within the domain are displayed below. For the purposes of this article, I'll be monitoring the Active Directory replication for a domain called Therefore, I'll add the SYSVOL share, as shown in Figure A. Click OK to close the Add / Remove Replica Sets dialog box.

Figure A
Select the SYSVOL option to monitor Active Directory replication within a domain.

After you click OK, Ultrasound will collect the schema information for the FRS replica set that you have selected. After Ultrasound finishes reading the Schema, you will see a message indicating that FRS needs to install Ultrasound WMI providers onto member servers. As you can see in Figure B, you have the choice of installing WMI providers on all servers, hub servers, or no servers. There is also an option to automatically collect the initial set of data from the WMI providers after the installation process completes.

Figure B
You must install WMI providers onto replicas within the FRS replica set.

Ultrasound uses WMI providers as agents on the servers containing replicas. The providers gather information from the individual replicas. This information can then be polled by the Ultrasound controller (the machine running the Ultrasound console). Normally, you would want to choose the All Servers option when asked which FRS member servers to install the WMI provider onto. You would also want to leave the Collect Initial Data From WMI Providers After Installation check box selected.

Click OK and you'll see the screen shown in Figure C. This screen allows you to monitor the WMI provider deployment process. It also allows you to add and remove FRS replica sets by using the Add / Remove Replica sets button. Another thing that you can do on this screen is select individual servers from the Ultrasound WMI Provider Deployment Status section and install, uninstall, or update the installation of the provider by using the buttons on the right. You can even schedule the installation of the provider, if you like.

Figure C
The Configure Ultrasound dialog box allows you to control the installation of the WMI provider on an individual server basis.

Now that I've shown you how to select which servers you want to install the WMI provider to, let's look at how to actually use Ultrasound. To give you a better feel for how to use the tool, I've added the domain controllers from my domain to the list of servers to be monitored.

The Ultrasound console is divided into five tabs:
  • Health
  • Details
  • Alert History
  • Summary
  • Advanced

The Health tab, shown in Figure D, is Ultrasound's default selection. In addition, all tabs display the Alerts dialog box shown to the right in Figure D. The top portion of the Alerts dialog box displays the number of current errors, warnings, and informational messages. The lower portion of this dialog box displays the specific events. You can double-click on an event for more information.

Figure D
The Health tab displays each domain or replica set's health.

If you look to the left side of Figure D, you'll notice that both of my domains are displayed. Each domain has an icon that looks like a green circle with a check mark in it. This icon indicates that the replica set is in good health. It doesn't mean, however, that the replica set is entirely free from problems. Note that just beneath the SYSVOL[TEST] container, there is a warning event displayed for server Brien.

Double-click on the warning to display a summary of the server similar to that shown in Figure D. Double-clicking on that summary will show details of the alert. For example, there may be a service patch that updates FRS to a new version.

The Details tab, shown in Figure E, allows you to examine the FRS in greater detail. What you see in Figure E is the connector topology for FRS members in the domain.

Figure E
The Details tab allows you to see each server's inbound and outbound connectors.

The column on the far left allows you to select the domain or replica set that you want to view. The column in the middle displays the members of the domain or the replica set. This chart also shows the number of inbound and outbound connectors for each member server. The server currently selected,, is displayed in green. All of the information in the column to the right pertains to the currently selected server. As you can see in the figure, the column to the right displays details of the inbound and the outbound connectors on the currently selected server.

As handy as it is to be able to see the individual connectors, you can do more with the Details tab. Just above the pane that allows you to choose a domain or a replica set, there is a drop-down list called Show. From this drop-down list, you can display things like topology changes, propagation tests, and files. Figure F shows the Files selection. You can see how many files need to be replicated, the number of backlogged files and sharing violations, and the names of the files themselves.

Figure F
The Files selection on the Details tab allows you to see which files need to be replicated.

Alert History
The Alert History tab, shown in Figure G, is very useful. At first glance, this tab doesn't really look like it does anything other than display event information. As you take a closer look at the tab, though, you will notice that the section to the left allows you to display all messages or specific types of messages. For example, you could display all warnings, or all warnings pertaining to the Win2K3 FRS release.

Figure G
The Alert History tab allows you to collectively view all FRS-related messages.

If you look at the main pane on this tab, you will see all of the messages that have been generated within the last 24 hours. You can change the time frame by adjusting the Date Filter just above the pane. Also, you can right-click on an individual message to change its status. For example, in Figure G, you'll notice that the Assigned To field of the selected message says Brien.

It is actually possible to assign FRS messages to various technicians. When a technician has looked into the message, they can right-click on it and change its severity or status. Changing the severity means that the technician can control whether the message should be displayed as informational, a warning, or an error. Changing the status allows the technician to change the message from being an active issue to being a resolved issue.

The Summary tab, shown in Figure H, is divided into five main sections. The top section is for all replica sets. This is basically just an overview of the health of the individual replica sets as a whole. This section will also display the current number of errors and warnings related to individual members or connectors.

Figure H
The Summary screen gives you an overview of FRS health as a whole.

The next section is Members (Unhealthy / Recently Updated). In the figure, you can see an example of both. Server Bart was recently updated, and Server Brien is considered to be unhealthy because its software is out of date.

The section to the right of the Members section is the Connections (Unhealthy / Recently Updated) section. This section shows all of the connections for the selected domain or FRS replica set that either have problems or have recently been updated.

The bottom two sections refer to Recently Completed Propagation Tests and Active Notifications. A function on Ultrasound's Tools menu allows you to run a propagation test. This allows you to tell for sure if replication is working or not, without having to worry about trying to replicate real data.

The Advanced tab, shown in Figure I, is where you can get the most detailed information on FRS. The column on the right allows you to look at general statistics or at statistics related to a specific FRS replica set. Within the General container or beneath an individual FRS replica set are a number of different filters. For example, you can look at failed notifications, FRS Event Logs, and AD Connection changes, just to name a few.

Figure I
Here you can find out the most detailed information about FRS.

When you select a filter, information related to that filter is shown in the pane to the right. It's important to note that this information scrolls way beyond the confines of the window, so there is a lot of information available that is not shown in the screen capture. Depending on the filter that you select, you may have some different options for which columns should be displayed. For example, if you select the Connections container, you have the option of displaying columns related to inbound connections, outbound connections, or all available columns.

Safe and sound
As you can see, Ultrasound can be invaluable in alerting you to FRS-related problems. Even if FRS is working perfectly, it is still a good idea to install Ultrasound and allow it to run in the background. After all, Ultrasound is free and if you ever do have a problem, Ultrasound will have lots of data on hand to help you immediately diagnose it.