Protocol analyzers and packet monitors are nothing new, but the problem with them is that, usually, you have to have a very in-depth knowledge of networking at the packet level before such a tool will be of any benefit to you. Fortunately, there are exceptions. A company called Sandstorm Enterprises has created a packet monitor called NetIntercept that you don't have to be a rocket scientist to use. In fact, this tool translates captured packets into readable text, Web pages, and graphics, allowing you to see exactly what is flowing across the wire.
How does NetIntercept work?
NetIntercept begins by capturing network traffic. As packets flow across the network, they are automatically captured through a NIC that's running in promiscuous mode. Although a 10/100 MBPS Ethernet NIC will get the job done, Sandstorm recommends using a gigabit Ethernet NIC for best results.
As packets are captured, they are stored on the server's hard disk in TCPDUMP format. In case you aren't familiar with the TCPDUMP format, TCPDUMP is a UNIX utility. This means that TCPDUMP is a widely accepted format. Therefore, you can export data to network shares or to removable storage devices and then use other third-party utilities to examine the data if you want. Of course, NetIntercept is fully capable of analyzing the data, but it's nice to know that you can use other utilities too if you want.
Because of the extremely high quantity of data being stored, and because the data is coming into the server at such a high rate, Sandstorm recommends storing the captured packets within a dedicated RAID array.
To make data storage more efficient, NetIntercept relies on circular logging. What this means is that as the hard disk begins to fill up, the software begins overwriting the oldest data.
As you can see, NetIntercept is capturing data at all times. In fact, Sandstorm claims that NetIntercept will capture 99.9 percent of the packets flowing across a TCP/IP network with bandwidth that is completely maxed out. For networks with light traffic, Sandstorm claims a 99.99 percent capture rate.
Because data is always being captured and almost no data is missed, it is easy to go back and look at data from a specific time frame. If an administrator wants to examine the data from a specific time frame, the administrator begins by selecting the time period that he or she wishes to examine.
The packets from that time period are sorted by connection and are then saved to a different hard disk. This makes the data easier to examine and prevents the packets from being overwritten. Technically, though, the original copy of the data may be overwritten, but the copy that the administrator will be working with is completely safe.
After all of the packets have been sorted by connection, the packets are run through a series of parse modules. These parse modules are responsible for detecting protocols and content types. There are quite a few different parse modules, each responsible for detecting a particular protocol. Because NetIntercept takes a modular approach to parsing data, the software is easily extensible by simply adding additional modules as new protocols are introduced.
NetIntercept takes a hierarchical approach to parsing. For example, the TCP/IP protocol has many other protocols contained within it. Therefore, if a high-level module determined that a packet was a TCP/IP packet, then a series of child modules would be called until the exact protocol has been determined.
The hierarchical approach works well because it saves the CPU from having to do so much work. After all, if a packet is determined to be an IPX packet, then you can safely assume that it is not HTTP, FTP, or any of the other numerous TCP/IP protocols. By determining up front that the protocol is not TCP/IP based, you can save the CPU from having to run any TCP/IP-related parse modules.
Another way that NetIntercept preserves CPU and hard disk time is by using memory mapping. Rather than copying session data unnecessarily, the data's address within the system's memory is passed from module to module.
Once the parse is complete, the results are stored in a database. I say the results are stored in a database because the actual objects are not included within the database. For example, suppose that a particular data stream made up a Web page. The Web page itself would be written to disk. NetIntercept would then use the MD5 Digest algorithm to calculate a signature for the object. The signature and a pointer to the object would be stored within the SQL database.
Saving the actual objects to disk reduces database complexity because the object does not need to be stored within the database. NetIntercept also takes steps to preserve system resources by not storing duplicate objects. For example, my Web site features a logo that is made up of a file named HEADER.GIF. If a hundred people in your company visited the Web site, there would be a hundred copies of HEADER.GIF going across the wire and being captured by the database. To preserve disk space, the hundred different references to HEADER.GIF would be stored within the SQL database. However, the actual HEADER.GIF file would only be written to disk once. Each of the hundred references to the file would point to the same file on the server's disk.
After the packets and the results of the analysis have been stored, the administrator can use the software's GUI to browse the results. The administrator can view both connections and content. The administrator can also generate a wide variety of reports regarding system usage. All of these functions can be performed either directly from the NetIntercept console or by using an X Windows session on another system. If you choose to use another system, you must tunnel to the NetIntercept server by using SSH.
Browsing and discovering
The NetIntercept user interface is divided into two main windows: the main window and the connection examination window. You can access the user interface either from the server console or through a remote connection. If you choose to use a remote connection, NetIntercept uses an encrypted X Windows connection for security reasons.
The main window is constructed in such a way that it allows you to get a general feel for the overall traffic flow on your network. You can then drill down through the packet stream and analyze individual connections. To facilitate this functionality, the main window is divided into six different tabs.
The first tab is the Traffic tab. This tab displays network traffic volume over time. Although this tab is interactive, you can also use it to spot network usage trends at a glance.
The Summary tab is just that—a summary. It displays an overview of and statistics for the current NetIntercept database.
The Forensics tab is where all of the action is, and needs a bit more explaining. The Forensics tab allows you to enter specific search criteria that you want to look for within the database. This can be a port number, a protocol type, or even a user name. You can search on just about anything. Once you have entered your search criteria, NetIntercept displays the results through the Connection Examination window that I mentioned earlier. I will discuss this window in more detail a bit later.
Another handy tab is the Alerts tab. The Alerts tab displays anything within the packet flow that might be unusual or suspicious. This is a great way of finding attacks against your network or potential security violations. You can take any information found within the Alerts tab and examine it through the Connection Examination window.
The View tab is one of my favorite components to the Main window. The idea behind the View tab is that as the packet stream is saved to the database, packets are reassembled and various objects are detected and then saved to the server. The View tab is where you would go to see these objects. For example, if you suspected that one of your employees had been surfing the Web for porn, you could go to the View tab. Not only would this tab display any images that have been downloaded from the Web, it would even show fully assembled Web pages.
The main window's final tab is the Configuration tab. As the name implies, this tab is used to control NetIntercept's configuration. Using this tab, you can tweak all of the various data capturing and analysis settings.
Now that I have shown you how the main window works, let's take a look at how the Connection Examination window works. As I explained earlier, the Connection Examination window provides more detailed information on a specific network connection. Like the main window, the Connection Examination window is divided into a series of tabs.
The first of these tabs is the Traffic Session tab. This tab allows you to see the actual data stream for the connection that you have selected. You also have the option of seeing the packet headers for the individual packets composing the stream.
Like the main window, the Connection Examination window also has a Summary tab. This Summary tab pertains to the selected connection rather than to the database as a whole.
The next part of the Connection Examination window is the Conclusions tab. This tab shows the results of any content analysis that you might have run against the connection. This tab also provides links to any files that have been associated with the connection (such as images and Web pages).
The final tab is the Criteria tab. As you may recall, earlier I said that the Connection Examination window was accessible through the main window's Forensics tab, and that the Forensics tab allowed you to enter search criteria. The Connection Examination window's Criteria tab displays the search criteria entered within the Forensics tab that caused this particular connection to be analyzed.
NetIntercept offers a variety of reporting options. Each report is template-based and is, therefore, fully customizable. Additionally, reports can be generated in either plain text or HTML and can also be displayed through the NetIntercept console or through a remote NetIntercept connection. There are five different types of reports that NetIntercept can create.
The first type is a traffic report. As the name suggests, a traffic report is a detailed summary of traffic volume on a machine-by-machine basis.
The next report type is a content report. A content report allows you to see what type of content makes up your traffic flow. For example, you could tell what percentage of your overall traffic consisted of Web pages or e-mail messages.
The third type of report is a Network Description Report. This report provides a summary of your network's configuration and performance.
NetIntercept also allows you to create a Focus report. A Focus report is a detailed traffic report that's focused on a specific user, host, or IP address.
Finally, you can generate a Security and Protocol Hygiene Report. As the name implies, this report displays security breaches and abnormal uses of protocols.
NetIntercept runs under X Windows (a GUI shell for UNIX). The software comes bundled with a server, so there's no need to worry about whether your current servers can handle the workload.
You can purchase NetIntercept by credit card or corporate purchase order. You may reach Sandstorm by calling (617) 426-5056, faxing (617) 357-6042, or by e-mailing email@example.com.
NetIntercept's pricing scheme includes server hardware. There are three basic systems advertised on the Sandstorm Web site. The first is a dual Intel Pentium III 1.4 GHz machine. This machine includes a 960/770 GB RAID array with an 8-way RAID controller. The server also features an 80-GB internal hard disk, 3 GB of SDRAM, a gigabit Ethernet NIC, a 10/100 Ethernet NIC, and a CD-RW drive. The price of the server with the NetIntercept software is $29,500.
A less expensive alternative is a dual Pentium III, 1.26 GHz machine with a 480/300 GB RAID array and a 4-way RAID controller. This machine ships with 1 GB of SDRAM, two 10/100 NICs, and a CD-RW drive. The price for this server with the NetIntercept software is $18,750.
NetIntercept's bottom-of-the-barrel server has a single 2.0 GHz Intel Pentium 4 processor, a 120/95 GB hard disk, 512 MB of DDR-RAM, two 10/100 NICs, and a CD-RW drive. This server includes the NetIntercept software and sells for $8,900.
If you require a higher end server than what I have listed here, you can contact Sandstorm. Sandstorm offers terabyte systems, but they are not listed on the Web site.
Powerful, but not inexpensive
As you can see, NetIntercept is a very powerful application. There are several advantages to using NetIntercept. First, NetIntercept captures data at all times. This means that if there were to be a security breach or a network problem, the forensic data is readily available because the problem packets were captured as they occurred.
Another advantage to using NetIntercept is that it can reassemble packets into their original data streams. Most protocol analyzers display packets based on the time that they were received, thus mixing packets from lots of different data streams together.
Because of the way that NetIntercept reconstructs data, you can do a better job of detecting spoofing. You can actually look at the contents of a complete data stream rather than being limited to examining packet headers and port numbers. To put it simply, Sandstorm claims that NetIntercept is the first tool to make it practical to analyze hundreds of thousands of simultaneous data streams.