Power Linux with pipes and redirection

Despite the fact that most people consider command-line computer use a thing of the past, there are some really interesting things you can do with the command-line. Despite pretty GUIs and eye candy being the order of the day, there is a definite advantage to being familiar and comfortable with the Linux command-line.

Two such advantages are the use of pipes and redirection. With pipes and redirection, you can "chain" multiple programs to become extremely powerful commands. Most programs on the command-line accept different modes of operation. Many can read and write to files for data, and most can accept standard input or output. This means that you can direct the output of one program as input to another program. You can then take the output of the second program and redirect it as input to yet another program, or redirect the output to a file.

In fact, most programs have two modes of output: standard output (stdout for short) and standard error (stderr). These are both different streams of output that are distinct from each other, which can allow you to redirect the stderr of a program to the same "stream" as the standard input (stdin), or redirect them to different places.

For example, to redirect the output of the ps program to a file, use:

$ ps ax > output.txt

If you wanted to merge two separate files together into a single file that already contains data, you could use the cat command and redirection:

$ cat 1.txt 2.txt >> 3.txt

A single > will redirect output to a file, overwriting the contents of the file if it exists. A double >> will redirect output to a file, but will append the output to the file if it exists, and create a new file if it doesn't.

By default, redirection works on stdout, while stderr will typically go to the screen. If you wish to redirect stderr to the same "stream" as stdout, use:

$ strace ps ax > output.txt 2>&1

strace is a unique program; the stdout of strace is the actual output of the program, while all the trace information goes to stderr. In the above command, strace is tracing the system calls and signals of the ps ax command and both stdout and stderr are being redirected to the file output.txt. This is accomplished by redirecting stderr (represented by the number 2) to wherever stdout (represented by &1) is directed. If you wanted to just have the trace output redirected to a file with the ps output on the screen, you would use:

$ strace ps ax 2>strace.txt

Pipes work a little differently. When you use a pipe, the output of the command on the left of the pipe is redirected as input to the command on the right of the pipe. For instance, to find all sshd processes in a ps listing, you would use:

$ ps ax | grep sshd | grep -v grep

This redirects the stdout of ps ax as stdin to grep sshd, which displays all of the processes that match sshd. This is then redirected as stdin to another grep command that filters out the previous grep's entry in the process table, resulting in a few lines that display only those processes that contain the word sshd in them.

This could be further tuned to use awk and kill each process:

$ for pid in `ps ax | grep sshd | grep -v grep | awk {'print $1'}`; do kill -9 $pid; done

The above command constructs a shell for-loop that iterates through the output of one ps command, two grep commands, and an awk command that return the process ID numbers of all sshd processes and then executes kill -9 on each one of them.

Despite how nice a good GUI looks, there is nothing like a good quick CLI command to make life more efficient and interesting.

Delivered each Tuesday, TechRepublic's free Linux NetNote provides tips, articles, and other resources to help you hone your Linux skills. Automatically sign up today!