Linux

Track network connections with LSOF on Linux

Use the LiSt Open Files (LSOF) utility to track data flow related to ports, users, and applications.

LiSt Open Files (LSOF) is a Linux utility that allows you to view current network connections and the files associated with them. LSOF provides verbose output and is useful in tracking down all sorts of information. For example, it allows you to see what program is operating on an open port, which daemons have established connections, and what ports are open on your server.

While similar in many ways to utilities like netstat and fuser, LSOF has many unique options that let you find specific information on ports, users, processes, and files.

Getting and installing LSOF
On many Linux systems, LSOF will be installed by default. Try running lsof -v to see whether the program exists on your system. If it doesn't, you will need to download one of the many packages available and install it yourself. Source and binary versions are available here, while distribution-specific packages are available from locations such as Rpmfind.net and are included in databases for apt-get, up2date, and urpmi.

If you are installing from source, download the tarball and perform these commands:
tar xpfz lsof_4.64.tar.gz
cd lsof_4.64
tar xpf lsof_4.64_src.tar
cd lsof_4.64_src
./Configure linux
make

During the configuration, you have the option of running an inventory to verify that all the files necessary for compilation are present. You will be asked if you want to customize the installation. This shouldn’t be necessary on most Linux platforms, so answering no is fine. If you encounter any problems running LSOF later, you may want to come back and try some customization. Customization may also be needed if you upgrade or modify your kernel.

After running make, the executable lsof should appear in the directory. There is no default make install rule, so you can create your own or just copy lsof to your directory of choice. Keep in mind that you may need to modify its permissions to be setuid-root if you want regular users to be able to see all open files. Not that this is necessarily recommended, but it’s good to remember, since the output generated when LSOF is run as root will be different from the output generated when it's run by a normal user account.

Looking at network connections
As its name indicates, LSOF deals with open files on a Linux system. An open file can be a regular file, a directory, a library, a stream, or a network socket. You can take advantage of various LSOF options, depending on what you’re looking for.

Running lsof by itself will output all open files corresponding to every active process on the box. This can be quite lengthy, so it’s best to know what you are looking for in advance. You can get a quick rundown of the options with lsof -h, and the man page goes into much further detail. Let’s take a look at some common switches and what they will show us.

lsof -i
The lsof -i command lists all open files associated with Internet connections. It is similar in format to netstat -a -p and will look something like Listing A.

By default, LSOF lists detailed information about each connection. In Listing A, we see the command or program involved, the process ID (PID), the user running the command, the file descriptor (FD), the type of connection, the device number, the Internet protocol, and the name of the file or Internet address. The -i option can be useful when you're attempting to secure your Linux box. You can quickly determine what ports are open and listening for incoming connections. LSOF will also associate them with a program name. Thus, you can quickly identify unnecessary security risks and shut them down.

Ports that are awaiting connections have the keyword LISTEN appended to them. These are ports that are open and accepting connections. Note that LSOF will not distinguish between ports that are completely open and ones that have filters applied. The keyword ESTABLISHED indicates that a connection on the given port has been made. In Listing A, there is an SSH session from labrat.remote.net to test.com. You can see multiple processes associated with the sshd daemon. The main daemon, PID 597, handles incoming requests and forks itself as needed. PID 8545 was spawned by sshd and is responsible for the 8547 process. The only noticeable difference between 8545 and 8547, besides the PID, is in the user field. Notice that lhutz is the user who has remotely logged in to this box. This is useful information that goes beyond merely presenting network connections.

You can narrow your search by specifying a particular port, service, or host name using techniques such as:
lsof –i :587
lsof –i :smtp
lsof –i @labrat.remote.net

LSOF will then output all matching connections. The above examples will list connections listening or established on port 587, list connections associated with the well-known service SMTP, and list connections coming from or going to the host labrat.remote.net, respectively. These techniques are handy if you know what you are looking for in advance. You can watch and see whether inbound SMTP connections are taking too long, possibly causing timeouts. You can verify that the service is in fact running and what port it is listening on. And you can see if anyone from a certain device is connected to your system, whether it is via SSH, Telnet, FTP, or just about any other way possible.

lsof -p 409
LSOF will also accept a PID and output all open files it is using. In this particular instance, we performed an lsof -I to determine what PID number NameD (BIND DNS service) was operating under. Once we discovered it was 409, we issued the command lsof -p 409. The output is shown in Listing B.

You'll notice the different FDs, or file descriptors, right away. The cwd variable represents the current working directory of the process; txt defines the program text, which is the executable itself; mem is a file held in memory, in this case a library; the 4 and 21 represent files in use by this particular process; and the u designator defines them as having both read and write access. These all help you determine whether something physically exists on the system, is being used by the process, or is being held in memory.

lsof +d
The command lsof +d /var/log/apache/ is similar to fuser. It basically associates open files with their processes. In this case, we are looking at all regular files in the /var/log/apache/ directory. The output would look something like Listing C.

In this example, Apache is keeping track of two sets of log files, an access and an error log for two domains. As you can see, there are some differences between regular files and Internet connections. For one thing, the TYPE is now REG, indicating a regular file. Also, a SIZE variable is present, which indicates the actual size in bits the file takes up. Notice too that the DEV variable indicates they all use the same device, in this example, a single hard drive. The +d flag that was issued with LSOF tells the command not to leave the top-level directory, while +D would perform a recursive check on all subdirectories.

lsof -F  <…>
The -F switch provides an excellent way to format LSOF output. This built-in feature allows you to pipe information directly into external programs, such as a Perl script, a C program, or even a monitoring program like MRTG. You do this by specifying which fields you would like printed. For example, lsof -F pcfn would print the process ID, the command name, the file descriptor, and the filename. Many options are available, and this can save you time in working with the raw data yourself.

We've covered just a few of the options LSOF provides, but the man page covers the full spectrum of LSOF capabilities. Some LSOF commands may be processor intensive, due to the sheer number of processes on a system, so be as selective as possible when running commands.

Summary
LSOF is an excellent utility for managing and tracking network connections on your Linux system. Although a number of utilities can perform similar functions, none is quite as robust as LSOF. With LSOF, you can list open ports, identify connections currently being made to your system, and determine what resources a process is using. Not only that, but you can also determine what processes a particular user has and find detailed information about file and directory usage.
0 comments