Filtering PF firewall logs

The PF firewall's pflog facility provides a powerful, flexible set of capabilities for generating network traffic logs. Because it stores logs in tcpdump's native pcap format, a wide range of other tools provide a powerful, flexible set of capabilities for filtering those logs.

The core packet capture and filtering technology of tcpdump was abstracted out by the tcpdump core developers into a library that could be leveraged by other applications. That library is called libpcap, and it was eventually ported to MS Windows as the WinPcap library. The API for that set of functionality, provided by these two libraries, is known as pcap.

Rather than reinvent the wheel, the PF firewall for BSD Unix systems adopted the binary data file format associated with pcap for its own logging facility, known as pflog. Thanks to this fact, and the fact that a number of network traffic analysis and other activity and logging capture, filtering, and visualization tools have adopted the same format, a surprisingly large number of tools are well-suited to parsing and sorting logs for PF firewalls, including tcpdump itself.

Instructions on how to deploy, configure, and maintain a PF firewall are well beyond the scope of this article. A simple introduction to PF on the desktop can provide a starting place for the new PF administrator, but more in-depth investigation is important to ensure you do the job correctly. Peter Hansteen's The Book of PF can provide a more substantial introduction to PF administration, coupled with the documentation that comes with the software on the major BSD Unix systems. See the manpages for pf and pf.conf on your BSD Unix system of choice for a starting point.

Ensuring that your FreeBSD system running PF as its firewall is using pflog to build network traffic logs is as simple as adding a single line to the /etc/rc.conf file, as explained in the aforementioned desktop firewall article:


The default location for PF logs is /var/log/pflog, though for recent activity on a default PF setup on FreeBSD the /var/log/ and /var/log/pf.yesterday files offer more abbreviated, date-separated plain text logs. Having these logs is only the most superficial start on making use of the value offered by pflog, however.

The pflog file itself is, as mentioned, stored in the binary pcap format established by the tcpdump utility. Being a binary format, this means it is not designed to be read as-is by the human eye with a text editor. Being tcpdump's native format, however, means that all the power of that utility is available for filtering and formatting the data -- and tcpdump is available on pretty much every major operating system in existence. Its ubiquitous availability, as well as the widespread adoption of its data format and API via the libpcap library (such as by Nmap and Wireshark) is owed not only to its utility but also to its licensing model.

In its simplest form, a tcpdump command that will provide an unfiltered view of the activity logged in the /var/log/pflog logfile would look like this:

tcpdump -r /var/log/pflog

The only types of network traffic that will be logged by PF are those specified in the pf.conf file, however. If you use a default deny model for firewall configuration (probably a very good idea), your PF configuration file might begin with this:

block all

To ensure that all network packets blocked by that rule are logged by the pflog facility, you would need to add the word "log" after the filter action name (either block or pass). For example, that same rule with logging would look like this:

block log all

By default, the log parameter for a PF filtering rule will only log the first packet that is part of an ongoing connection: PF tracks the state of ongoing connections, and defaults to only indicating the establishment of such a connection. To ensure that every packet that matches a particular rule is logged, and not just the first packet to establish a given connection that matches that rule, (all) is added to the log parameter:

block log (all) all

A virtual network interface called pflog0 is used to direct logging to the logfile, by default. Additional interfaces can be created, using the ifconfig create command for instance, or via configuration files appropriate to your specific operating system. On FreeBSD, virtual interfaces such as pflog1 and pflog2 can be created using the following lines in your /etc/rc.conf file:



In the terminology of the ifconfig utility, this creates a network "pseudo-device". Given a name identical to an existing device, except with a different unit number -- as in the case of creating pflog1 when there is already a pflog0 -- the pseudo-device is a clone of the original, with essentially the same properties.

By specifying a particular interface where you want logging for specific pf.conf rules sent, you can ensure that pflog organizes some of your network traffic logging for you. This can help simplify your job as a firewall admin by providing simple, unique handles by which you can sort and filter your log data.

Thanks to the default textual format of tcpdump output, additional filtering tools that are common across Unix systems, such as awk and grep, can be used to filter, select, and manipulate data further. Visualizations such as those provided by tools like Wireshark or -- especially for realtime monitoring -- applications like Conky or GKrellM can be used to offer immediate feedback for critical, hands-on attention. For realtime textual monitoring, tcpdump can be used to continuously display the latest logged traffic activity by targeting a pflogN virtual interface rather than a logfile.

The tcpdump tool itself offers a great deal of flexibility and power in its filtering and formatting capabilities. Couple that with the Unix pipeline and standard Unix core utilities as well as programming languages well suited to writing filter scripts, and the possibilities for producing usefully filtered, simplified data become effectively endless. You just need to know what kind of data you want to keep, and what kind to throw away.