Linux

Tech Tip: Manage large file listings with xargs


Linux administrators have traditionally used the xargs command to manage large argument lists, such as file listings. The xargs command also has many other uses.

But handling long file listings is a very useful function. For example, if you have a directory with a large number of files, you may have experienced the "too many arguments" problem associated with trying to list or delete files in a directory.

Let's say that you have a directory full of clip art and want to delete all of the JPEG files. You attempt to use the following:

 rm *.jpg

You may find that rm can't handle the large number of files, and the command fails. You can still remove the files, but it requires the use of a few extra commands.

Use the following command to remove all of the JPEG files from the current directory and all subdirectories:

$ find . -type f -name '*.jpg' -print|xargs -l50 rm -f

This command uses the find command to locate all files in and below the current directory with the name "*.jpg" and print them to standard output. The command then pipes the output from find as input into xargs, which then executes rm -f on a maximum of 50 files at a time.

You can use the xargs utility in many more situations. It's a powerful tool that allows you to efficiently chain together commands that you otherwise couldn't connect, manage files efficiently, and more.

2 comments
foxmajik
foxmajik

You took your formula directly from a O'Reilly Media book without even citing the source. The method is outdated and does not account for very high values or for spaces in filenames; nor does it account for variations in the file extension. Better you use this: find . -type f -iregex '\.jpe?g' -print0 | xargs -0 rm -f And cite your sources.

foxmajik
foxmajik

You took your formula directly from a O'Reilly Media book without even citing the source. The method is outdated and does not account for very high values or for spaces in filenames; nor does it account for variations in the file extension. Better you use this: find . -type f -iregex '\.jpe?g' -print0 | xargs -0 rm -f And cite your sources.

Editor's Picks