Microsoft suggests using the Aggregation Application Block when gathering information from disparate sources, such as files, Web Services, databases, and internal objects, to present them as a cohesive dataset. But implementing the block correctly (to say nothing of wading through the dense documentation) is another matter entirely.
You can download the Aggregation Application Block for free. You'll have to load the project after installing the package and actually build the blocks using Visual Studio .NET 2002 or 2003 or the command line compiler of your choice. This block utilizes the Exception Management Application Block, which is included in the project file. You must add a reference to the dll file in the /bin/ directory in the References folder of the project in which you intend to use the blocks. Finally, you'll need an assembly reference on each file or form:
Putting it to work
Let's work through an example to see how the block works. In this scenario, I have to get the weather for the cities in a salesperson's territory. I will use the sample Northwind database. I want to gather information that looks something like this:
I can get this information from no fewer than two sources: first, the Northwind database, with a query like this:
Employees.FirstName + ' ' + Employees.LastName AS FullName,
Customers ON Orders.CustomerID = Customers.CustomerID
Employees ON Orders.EmployeeID = Employees.EmployeeID
Then, for the weather, I need a Web service that facilitates searching by city and country, like GlobalWeather from Cape Clear.com. This service is currently available.
Now, I have access to the weather reports for the cities. I could look up each report using a foreach block, but it is more fun to have a dataset with the weather reports in a table with the name and the city.
Using the Aggregation block
So now I have two data sources, and I would like to use them as if they were one. It's time to bring in the Aggregation Application Block. In a nutshell, I need to define the data sources in a special configuration file and make an asynchronous request to gather the data. The block provides all piping code. When the request returns, I'll have the dataset I need.
The first step in the process involves ServiceAgents, which are important pieces of information that relate to our data sources. These ServiceAgents are custom bits of code compiled into classes that implement IServiceAgent and therefore know how to access the data sources. The ServiceAgent class would include code to access the SQL Server and call the Web service.
The AggregationConfig file will store the location of that assembly, the name of the class, and perhaps some security information. This is fantastic for scalability issues, as the assembly can be easily distributed. Once the client application knows where the ServiceAgents are located, I use the AddRequest method to make an asynchronous call.
It is a simple process to conceive, but it is a significant amount of code. The ServiceAgent code needs to be designed as part of the application middleware, the configuration file must be set up for each client, and the asynchronous calls must be designed in the system. Once the middle tier is developed, the implementation is fairly straightforward.
Implications of using aggregation
Aggregation has all of the problems of XML Web services, database access, file systems, and everything else you want to throw in—along with the problems of integration. Web services just aren't reliable, and if you aggregate them, your system is going to be sick when the services are sick. There is ample error handling, but the potential pitfalls are tremendous.
Still, aggregation can be a major boon in a couple of situations. The first is when you have no other choice. For example, the data you need is on the mainframe, and it is not moving, and there is nothing you can do. Now, at least, there is no need to copy it every day—just aggregate. The second situation where aggregation can be handy is when you are using it for a non-mission-critical app and failure is an option, like our weather report above. If the system has a problem, the sales staff just has to call "time and temperature" today.