Linux

Save time managing multiple systems with Parallel SSH

For a user caring for multiple systems, OpenSSH is extremely useful, but being able to execute OpenSSH commands in parallel is even more so. Vincent Danen introduces Parallel SSH, or pssh, which allows you to execute commands on various hosts in parallel, copy files in parallel, and more.

OpenSSH is perhaps one of the most powerful and versatile tools available to any Linux user. It allows you to securely connect to a remote system via a shell or encrypted FTP and also allows you to copy files securely to and from remote systems.

For a user caring for multiple systems, OpenSSH is extremely useful, but being able to execute OpenSSH commands in parallel is even more so. This is where Parallel SSH, or pssh, comes in. Pssh provides parallel versions of the OpenSSH tools, meaning you can execute commands on various hosts in parallel, copy files in parallel, and so forth. Pssh is essentially a frontend to OpenSSH written in Python. It includes pssh, pscp, and prsync, as well as pslurp (the opposite of pscp in that it downloads rather than uploads) and pnuke (a frontend to the kill command).

Using pssh is extremely easy. There are no manpages, but calling the command with no arguments will bring up the help, which describes each option.

Every command uses a plaintext "hosts" file that is a simple text file containing the hosts to act upon, one per line. As an example, assume you wanted to make sure that the time on each server was identical. This could be done using the date command, but to do it with regular SSH, you would have to execute the command at the same time on each host using screen or multiple terminals. With pssh, this is one simple command:

$ pssh -h hosts -P date
hades: Wed Nov 12 10:21:11 MST 2008
hades: [1] 10:21:11 [SUCCESS] hades 22
odin: Wed Nov 12 10:21:11 MST 2008
odin: [2] 10:21:11 [SUCCESS] odin 22
$ cat hosts
hades
odin

Contrast that to using ssh directly:

$ for host in hades odin; do ssh ${host} "date"; done
Wed Nov 12 10:24:02 MST 2008
Wed Nov 12 10:24:02 MST 2008

Not much difference, other than the one command is easier and executes faster. Calling the date command itself won't take up much time, but imagine for a moment that you were performing a database dump or some other time-intensive task. Using ssh directly in a serial mode, if each dump took one hour, would cost you two hours of ssh execution on the host. In parallel, it would only take one hour. If there are 10 systems to execute the command on...well, you get the idea.

You can use pscp to copy files to the hosts:

$ pscp -h hosts example.txt /tmp/example.txt

Or you can use pslurp to copy files from the hosts:

$ pslurp -h hosts -L ~/tmp /tmp/example.txt example.txt

The above will download /tmp/example.txt as example.txt, and will store the file in ~/tmp/[host]/, where [host] is the corresponding hostname to that found in your hosts file.

Finally, prsync will also upload to the hosts using rsync rather than scp.

The Parallel SSH tools can be great time savers if you work with multiple systems and need to be able to copy files to or from these servers and/or execute similar commands on each host. This can be great for updating servers with the latest security fixes, performing automated maintenance such as database backups and retrieval of dump files, and so forth.

Get the PDF version of this tip here.

Delivered each Tuesday, TechRepublic's free Linux and Open Source newsletter provides tips, articles, and other resources to help you hone your Linux skills. Automatically sign up today!

About

Vincent Danen works on the Red Hat Security Response Team and lives in Canada. He has been writing about and developing on Linux for over 10 years and is a veteran Mac user.

11 comments
scarville
scarville

I can already think of a good use. Everytime the developers need a new servlet module for tomcat installed I have to restart at least four instances -- sometimes more.

Photogenic Memory
Photogenic Memory

Parallel ssh seems interesting and a great tool to distribute commands and files. I think I'll give it a try. However, the Linux screen command can deliver a similar functionality albeit singly if you bounce back and forth between terminals. Overall though, the same be generally accomplished. I assume though that Parallel ssh would be really great initiating updates or even locking down an entire system in case of a security breach. Sounds like a lovely tool!

guygo
guygo

Excellent tool! I had something similar cobbled up in Perl, but this looks ready-out-of-box, so I'll definitely take it for a spin. One minor comment: The pscp name for "parallel scp" crashes with PuTTY's version of scp, but that's in another world, so I guess it won't really matter. guygo

tl-news
tl-news

If you don't get how useful this tool is then you'd probably have no use for it.

ppuru
ppuru

Does pssh really do it in parallel? multiple connections at exactly the same time?

herouxb
herouxb

The problem running multiple command on more than one server is that you need to have a user + its own ssh key, unless you will have to enter the password for each server. Hence, if you have some sort of RSA key to enter before, it will take a longer time unless you change your shell.

orange_Ego
orange_Ego

I think this Parallel is only useful in the single console (single process) linux. For most of the linux with windows, I think it is easy to open several console at the same time, and run SSH seperately on them. So, you can also connect different remote machine and operate parallelly.

tl-news
tl-news

Our team uses a similar (home-grown) tool for use on their cluster nodes. This would be especially handy for that.

jmcglash
jmcglash

I didn't try this yet but I did read the documentation on the site... Usage: pssh [OPTIONS] -h hosts.txt prog [arg0] .. -h --hosts hosts file (each line "host[:port] [user]") Key management will still be a pain unless something like Keychain from the Gentoo project (http://www.gentoo.org/proj/en/keychain/) is used.

esteban.aliverti
esteban.aliverti

You didn't understand the idea. Maybe the best solution will be to have all the servers in one room so you can directly lunch the commands in the server's keyboards!