Monitoring user activity is a first line of defense in discovering unauthorized activity on your system. Jim McIntyre covers monitoring tools and system accounting procedures for your Linux system in this Daily Drill Down.
The ability to monitor user activity on a Linux system is one of the first skills a Linux administrator should master. This ability often provides a first line of defense in discovering unauthorized activity. This Daily Drill Down discusses the monitoring tools that are available as part of the Linux operating system, and demonstrates how to employ system accounting procedures to enhance system security.
Connection accounting is the process of tracking current user logins and logouts. Connection accounting on a Linux system is managed through the following utilities:
- dump-utmp Converts raw data from /var/run/utmp or /var/log/wtmp into readable ASCII text. Best suited for use in scripts.
- ac Reads /var/log/utmp and outputs user connect time in hours. The default is a summary for all users.
- last Shows a list of last logged-in hours. Provides more detail than the ac command.
- who Shows currently logged-in users. Reads /var/log/utmp file.
- utmp Used to log active user sessions.
- wtmp Used to record login and logout activity. The wtmp utility also records system reboots and run-level changes.
Several programs are involved in providing information related to connection accounting. This eliminates the need to run a specific daemon to initiate connection accounting. However, the following files must exist on your system:
- /var/run/utmp Contains all data associated with utmp.
- /var/log/wtmp Contains all data associated with wtmp.
If these files don't exist on your system, the following commands will create them:
These files are owned by root, and the file permissions should be set to 644 (rw-r-r--). The information contained in these files is not readable by a user running the cat command, or a similar command. However, there are utilities that will convert this data into text format.
The ac command
The ac command provides statistics on user connections. A typical use for the ac command would be to check the amount of time each user has spent logged into the system.
Although limited, the ac command does provide some useful information related to intrusion detection. For example, if I run the ac command, and I see that user Susan has spent several hours logged in the last two days, when I know Susan is on vacation, I have an indicator that someone may be using Susan's account to gain unauthorized access to the system.
The last command
The last command provides per-user login and logout times, as well as information on system reboots and run-level changes. By default, last will display every connection and run-level change recorded in /var/log/wtmp. The amount of information presented by the last command can be overwhelming. A more typical use for the last command would be to run the command last -5. This would provide the five most recent entries in /var/log/wtmp.
The important point to note here is that the last command not only shows who has been logged into the system, but also when they logged in, for how long, and from where they logged in. This information can be used to identify unauthorized activity on your system. For example, if the last -5 command showed that Susan had logged into the system after working hours, this could be an indication that someone else was using a legitimate user account to access the system. The administrator could then disable the account, to prevent it from being used for further access, prior to performing a more thorough security analysis. Running the command last -x will tell you about run-level changes on your system.
The last command can provide an administrator with clues about suspicious activity. Unexpected logins, unauthorized run-level changes, and system crashes can all be detected using this utility. As a system administrator, running the last command on a daily basis will allow you to spot unusual activity as you become more familiar with user activity on your network.
The who command
The primary purpose of the who command is to report which users are currently logged into the system. The who command can provide the following information:
- Which device a user is logged in from.
- Where the login occurred.
- The hostname if a remote login is used.
- The X display if X windows is used.
- Whether or not the user will accept write or talk messages.
The command who -iwH will provide the user's ID, the idle time for the user, the write or talk status, and will also print a header.
The value of using the who command is that the administrator gets a snapshot of current user activity on the system. This command is often the administrator's first line of defense in detecting suspicious activity on a Linux system.
The lastlog command
Another command used for connection accounting is the lastlog command. The lastlog command reads the /var/log/lastlog file to produce a record of the last time a user logged in. While this command is limited in its flexibility, lastlog provides a quick way to check user activity or spot unusual login times or users who are not known to the administrator. The lastlog command is useful for spotting unusual login activity on a Linux system.
Process accounting takes into account all the procedures and commands used to monitor process activity. The data for process accounting is logged in the /var/log/pacct file. This file should be owned by root, and should have its file permissions set at 600 (rw-------). The /var/log/pacct file must exist before process accounting can be activated. If this file does not exist on your system, you may create it with the command:
Once this file is created, process accounting is activated with the command:
The /var/log/pacct file may be replaced with any file you would like to use for process accounting. Just remember to create the file and to set the permissions correctly. The command to activate process accounting must be run at each reboot. To accomplish this, place the following script in the /etc/rc.d/rc.local directory, or in the startup script for any run-level you choose:
#Initiate process Accounting
#First, verify accton command exist, and is
if [ -x /sbin/accton ]
echo ``Process Accounting Activated''
Once process accounting is activated, you have three commands at your disposal for interpreting the information contained in /var/log/pacct:
- dump-acct Similar to dump-utmp. (Because the dump-acct command is better suited for scripts, I won't be discussing it in this Drill Down.)
- sa Provides a summary of process accounting.
- lastcomm Provides a list of commands executed on the system.
To turn process accounting off, simply run the command accton, with no arguments.
The sa command
Like the ac command, the sa command is a statistical command. The sa command will produce a summary of process usage on either a per-user or a per-command basis. It will also provide information on system resource usage. The sa command is most useful when the administrator is looking at the activities of a particular user; otherwise the volume of information generated by this command may be overwhelming.
The lastcomm command
Lastcomm provides output on a per-command basis only. It also prints a date stamp associated with the execution of each command. This makes lastcomm a more interactive security tool than the sa command. The lastcomm command can accept any of the following as command arguments:
- Command name
- Terminal name
This allows lastcomm to perform more finely tuned searches of the process accounting database. Process accounting becomes more useful after the administrator has identified suspicious activity on the system. Using lastcomm can be useful in tracking user activity or command execution at a specific time, but it must be activated with process accounting for it to be available.
Managing system accounting files with logrotate
The files /var/log/utmp, /var/log/wtmp, and /var/log/pacct function as dynamic database files. Two of these files, /var/log/wtmp and /var/log/pacct, grow by having entries appended to them. On a busy network, these files can become quite large. Linux provides a program called logrotate, which allows administrators to manage these files.
Logrotate reads the files in the /etc/logrotate.d directory. By placing a simple script in this directory, the administrator is provided with an easy way to manage dynamic database files. A typical script for managing the /usr/log/wtmp file might look like this:
This script performs the following actions:
- Rotate 5 Keeps one current copy and five old copies of this file.
- Weekly Rotates this file weekly, usually on the first day of the week.
- Errors Sends error reports to this mail address.
- Mail relating to logrotate is mailed to this address.
- Copies the active log file to create the backup, then truncates the active log file. Copytruncate allows processes to continue writing to the active log file.
- Compress Uses gzip to compress the old log file.
- Size 100k Rotates files automatically when they exceed this size.
Security tip: When a system is broken into, one of the first actions the intruder will perform is to replace commands like who and sa on the system with versions that will cover unauthorized access. If you suspect that you have been hacked, make sure you are running versions of these programs that you know you can trust. Reliable versions are available from the FTP sites of all Linux vendors.
This Daily Drill Down provided an overview of connection and process accounting, and log file management. We discussed the commands and procedures that administrators need to know to enhance system security, the interpretation of information produced by these commands, and the security value of this information.
The authors and editors have taken care in preparation of the content contained herein but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for any damages. Always have a verified backup before making any changes.