Data Centers

Big Brother is watching your network

A good network monitor is worth its weight in gold, and Scott Lowe has an open source network monitor that weighs in at a hefty sum. Read on as Scott shows you how to install and begin monitoring with Big Brother.

Big Brother from BB4 technologies is an open-source system and network-monitoring solution that covers a range of devices. From a user’s desktop to the sshd process running on your UNIX server, almost anything that can go wrong on your network falls under Big Brother’s watchful eye.

For this Daily Drill Down, I’ll install the latest UNIX-based version of Big Brother server; BB4 also offers the toolkit for the Windows platform. I’m going to walk through an installation and configuration of Big Brother on a Red Hat Linux 7.1 server and use it to monitor a RH 7.1 server, as well as a Windows 2000 server running the Big Brother client.

It slices, it dices…
What sorts of features does the Big Brother package include? A better question would be what does it not include. The basic Big Brother package includes a lot of functionality, and over the years hundreds of users have tweaked the software or written modules to give it new capabilities. For example, one third-party module adds support for monitoring Netapp RAID arrays and network interfaces, while another module determines the status of an APC UPS unit.

The basic package as downloaded from the Big Brother Web site includes support for network and network services checking, server checks, notification, redundancy, and history logs. And that’s just for starters.

Let’s set things up
You should not run the Big Brother processes as root, so our first step in preparing for an install is to create a user for the processes. For this article, we will create a user called bb and a group called bb. On my Red Hat Linux 7.1 server, I’ll type /usr/sbin/groupadd bb and then /usr/sbin/adduser bb -g bb at the command line to accomplish these two tasks. Of course, I need to be logged in as the root user to do so.

Get it!
To get the Big Brother software, point your favorite Web browser to the BB4 Web site and download the latest release. When I wrote this article, the current UNIX-based release was version 1.8d1. Save this download into the /usr/local directory.

Expand it
You should still be logged in as a root user to expand and install the Big Brother distribution.

First, let’s expand the distribution. Switch to the /usr/local directory with the command cd /usr/local and type gunzip -dc bb-1.8d1.tar.gz | tar xvf –. You will end up with two files: a README.FIRST file and a .tar file. The README.FIRST file contains installation notes and so forth, and the .tar file contains the guts of the Big Brother install. Type tar xvf bb18d1.tar to expand the archive. The extracted files will be located in /usr/local/bb18d1 and /usr/local/bbvar.

At this point, create a symbolic link to the /usr/local/bb18d1 directory. While still in /usr/local, type ln –s bb18d1 bb. Instead of having to type cd /usr/local/bb18d1, you now can type cd /usr/local/bb. This shortcut also will aid in version control when you decide to upgrade Big Brother later on.

Getting the installer up to speed
First, type cd /usr/local/bb/install to switch to the installation directory. Next, type ./bbconfig to begin the installation process. Please note that the bbconfig program can take optional parameters to make some of your installation choices, such as on which OS Big Brother is being installed. However, in our example we are going to let the bbconfig installation program figure it all out.

When you launch the install routine, you’ll be asked to acknowledge a license/information agreement. Once you have complied, the installation program will ask you a series of questions.

Let’s run through the installer setup section by section. We’ll depict the program’s on-screen messages through text in gray boxes.

I am running Red Hat 7.1 on my test system, so I chose redhat (as shown in Figure A) from the list and pressed Enter.

Figure A
The installer needs to know on which OS you are running Big Brother.

For security reasons, I have chosen to prevent bb from being executed as the root user (in Figure B), and I have chosen the bb user ID that we created earlier as my Big Brother user ID.

Figure B
This section is the user section, as it defines user parameters.

Big Brother then tests to see whether the installation directory is writeable, which it is. The next step will have you decide if you want to keep the old-style directory structure, as shown in Figure C. Since we are going to install a brand new Big Brother setup and not use any older third-party add-ons, we will use the new directory structure.

Figure C
The new directory structure is usually preferable in a clean Big Brother setup.

Do not say no to DNS (as in Figure D) unless you absolutely cannot use DNS. If you have a DNS server and everything is registered, you most certainly should use DNS with Big Brother.

Figure D
Since my test lab does not have its own DNS server, I won’t use fully qualified domain names.

Since I have installed the software onto a lab Linux server, I am going to use the host server for Big Brother’s Web-based BBDISPLAY and BBPAGER services (Figure E).

Figure E
We’ll use the Linux Big Brother server to run Web page and alert services.

A BBDISPLAY is a server that displays statistics gathered by the Big Brother processes, while a BBPAGER server sends out notifications when something is amiss. These can be the same machine.

As Figure F shows, you will want to configure someone (preferably the servers administrator) to receive alert mails.

Figure F
You need to identify the user that gets urgent messages from the server.

To take advantage of Big Brother, you need a Web server with CGI capability. Figure G shows that I use Apache and that I have told the Big Brother installer where to find my Apache installation.

Figure G
Depending upon your distribution, the cgi-bin directory may also be located in /var/www/.

After you’ve chosen your cgi-bin, the installer will update, check, and verify paths, showing you some info along the way. Next, you will need to adjust some parameters, based on how you have your server configured. Figure H shows some parameters you will need to adjust.

Figure H
This section is specific to your Web server installation.

Lastly, you’ll see a section, depicted in Figure I, that gives you final instructions on building the application. We’ll closely follow these instructions as we move on. If you are using Red Hat 6.1, please pay attention to the note.

Figure I
This is the final section.

On to actually performing the installation
At this point, we’ve told the Big Brother installation program how to set itself up. Now we need to run through a few more steps to actually perform the installation.

Type cd /usr/local/bb/src at the command line to switch to the source file directory. Compile the source files by entering make at the command prompt and then enter make install to place the binary files where they belong.

Now, let’s add a network resource to be monitored and see what happens. Type cd/usr/local/bb/etc and edit the file bb-hosts using your favorite editor (mine is pico). Before you continue, I highly recommend saving a copy of the original bb-hosts file for later reference.

Remove the contents of the bb-hosts file and put in the following entry all on one line. (We’ll talk about the format of the file in a future article). In this entry, I am using my own lab network information. You will want to make sure to change the IP address and machine name to reflect your environment.

Now let’s make sure that your configuration has no errors. To check the bb-hosts file for errors, type /usr/local/bb/etc/ at the command prompt. If errors occur, they will be displayed at the prompt.

Next, we’ll assign ownership of the Big Brother files to the bb user. Switch to the bb directory by typing cd /usr/local and then type chown –R bb bbvar bb18d1.

Now we need to create a link from the bb directory to the path where we told the installer to store our Web pages. In this example, I typed
ln -s /usr/local/bb/www /usr/local/apache/htdocs/bb

to create the symbolic link. The /usr/local/apache/htdocs directory is my docroot for Apache. If yours is different, make the appropriate changes.

Start the software
Everything should be set up at this point. Let’s try to start Big Brother and see what happens. Type /usr/local/bb/ start to try it out.

With your favorite browser, navigate to (or wherever you told BBDISPLAY to present information when we were asked to enter the base URL for the server—during installation), and you’ll find yourself at the Big Brother login screen.

Each of the headings specifies a service that Big Brother is monitoring: connection (a ping check), CPU utilization, disk usage, Web services, processes running, and the sshd daemon. If you click on one of the green dots, you get more information about that particular service. When you click on the disk dot, you are presented with a list of all disks and their current space usage.

Monitoring other servers
You now have a working Big Brother system that monitors the server where Big Brother is installed. But the program would be pretty useless if it were limited to just that. On remote machines, such as another UNIX/Linux machine or a Windows 2000 Server, you can install a Big Brother client that sends its statistics to the Big Brother server’s listener port, port 1984. Yes, 1984. You can add network devices to the bb-hosts file to make sure that they are alive via periodic ping checks. This allows you to monitor your entire infrastructure.

We will go over adding a Windows 2000 machine to the monitor list and watch its CPU utilization to make sure that a particular process is running at all times.

To fully monitor a Windows 2000 machine, you need to download a specific client from the Web site. As of this writing, the Windows client filename was bbnt1-08b.exe. Download this file to your Windows 2000 machine, double-click its icon, and follow the defaults for the installation. When the install is done, the client configuration window will open. For this example, I am going to use the parameters shown in Figure J.

Figure J
Windows NT/2000 client configuration is simple with Big Brother.

The BBDISPLAY and BBPAGER hosts are the Linux box where I installed Big Brother. I am also going to watch the process listed under the Processes List section. This will alert me when this process fails or when one of the services listed fails. I will also be alerted when the C: drive usage goes beyond 90 percent.

Once I choose Stop Service, Save, and Quit, this Big Brother client will begin to periodically send this information to the BBDISPLAY and BBPAGER hosts.

On the Big Brother server, add the scott-2kp entry to the bb-hosts file in /usr/local/bb/etc/bb-hosts with the following line:

Your Big Brother status screen should now show the scott-2kp machine.

The scott-2kp machine was added to the list and is being monitored by the Linux server. In the bb-hosts file, we just had to add the machine and its IP address, and the Big Brother daemon listening on port 1984 does the rest. It receives the statistics sent from the Windows 2000 machine, determines what is being monitored, and populates the proper columns.

If you click on the cpu button next to scott-2kp, you will see the screen shown in Figure K.

Figure K

The page shows you more than just CPU utilization; it gives you a snapshot of your overall memory utilization. For me, this has come in handy more than once in diagnosing problems with a Windows 2000 server.

You might notice a button labeled History at the bottom of the page. It appears at the bottom of most of Big Brother’s statistical readouts. Once you have Big Brother up and running, you will find this button invaluable for statistics reporting.

Clicking the History button on the connection detail screen in my test lab results in the image shown in Figure L below.

Figure L
See it? Built-in statistics!

The page shows me that this machine has been 98 percent available in the reporting period and tells me whether it was a hard error (failed a ping check) or something else that caused the outage. This info can be very valuable!

Wrap up
As you can probably see, Big Brother is definitely worth the download. Although it can be a difficult setup, it can be easy to use and provides a slew of statistics, depending on your needs.

The next time we visit Big Brother, we will discuss ways to add redundancy to avoid a single point of failure in monitoring, as well as how to configure it to send alerts on problems it detects.

Editor's Picks