The curl command is quite useful and flexible. The intent of the tool is to transfer data, without user interaction, to or from a server, using one of the many supported protocols. It is that list of protocols that helps curl manage to be so flexible, as the command supports:
- DICT
- FILE
- FTP
- FTPS
- GOPHER
- HTTP
- HTTPS
- IMAP
- IMAPS
- LDAP
- LDAPS
- POP3
- POP3S
- RTMP
- RTSP
- SCP
- SMB
- SMBS
- SMTP
- SMTPS
- TELNET
- TFTP
That’s a healthy list of possibilities. It also means curl can transfer just about any type of data. Curl can even display the source code of a URL. All of this without requiring user interaction (a crucial feature for scripting).
I want to show you how to make use of this command (one you’ll frequently find in Linux howtos and scripts). I’ll be demonstrating on Elementary OS, but the command works on nearly every available Linux distribution. Curl is also available on macOS and Windows.
SEE: 20 quick tips to make Linux networking easier (free PDF) (TechRepublic)
Installing curl
Your Linux distribution should have curl installed by default. If not, the installation is quite simple. On a Debian derivative, the installation command would be:
sudo apt install curl
You can check the installation by issuing the command curl –version. You should see the version number of the installed application (on my Elementary OS machine, it’s 7.47.0).
Using curl
Let’s first see how curl can be used at its simplest. Say you want to view the source of a web site. We’ll use the curl site as an example. Issue the command:
curl https://curl.haxx.se | less
You can now scroll through the HTML for that site (Figure A).
Figure A
This is a great way to figure out how a site has been created or even troubleshooting your own sites.
Of course, that example is quite limited. Let’s use curl to pull down a file from a site. Let’s stick with the same example. Say you want to download the HTML for the curl site to view later. For this, we’ll use the -o switch like so:
curl https://curl.haxx.se -o curl.html
The above command would download the HTML code from the curl site and save it as curl.html. Of course, curl isn’t only capable of downloading source HTML. Say you have a file you want to download from a site. Curl can handle this like so:
curl http://SERVER_ADDRESS/FILENAME -o FILENAME
Where SERVER_ADDRESS is the URL of the server and FILENAME is the name of the file to be downloaded. Say for example, you want to download the latest release of Ubuntu Server. That can be done like so:
curl http://releases.ubuntu.com/18.04/ubuntu-18.04-live-server-amd64.iso -o ubuntu-server-18.04.iso
If that file is password protected, curl can handle that like so:
curl -u USERNAME:PASSWORD http://SERVER_ADDRESS -o FILENAME
Where:
- USERNAME is the username on the server.
- PASSWORD is the password for the user on the server.
- FILENAME is the file to be downloaded.
- SERVER_ADDRESS is the direct link to the file.
You can also use curl with an FTP server. Say you need to download a file from an FTP server that happens to be password protected. The command for this would be:
curl ftp://SERVER_ADDRESS/FILENAME -user USERNAME:PASSWORD -o FILENAME
Where:
- SERVER_ADDRESS is the address of the FTP server.
- FILENAME is the name of the file to be downloaded.
- USERNAME is the username on the FTP server.
- PASSWORD is the password for the user on the FTP server.
To upload a file to an FTP server, the command would be:
curl -T FILENAME SERVER_ADDRESS -user USERNAME:PASSWORD
Again where:
- SERVER_ADDRESS is the address of the FTP server.
- FILENAME is the name of the file to be downloaded.
- USERNAME is the username on the FTP server.
- PASSWORD is the password for the user on the FTP server.
At some point, the curl developers removed SFTP support from the libcurl. If you need to download a file, via SSH, you’ll need to use the sftp command.
Why use curl
One of the biggest benefits of curl is that it can be used without interaction. Because of that, it’s perfectly suited for scripting. You’ll run into many instances of Linux installer scripts that make use of curl. As an installer script runs, you might well see curl do its thing by downloading necessary dependencies. So if you’re writing a Linux bash (or installer) script, curl will most certainly be your friend. For more information on curl (and there’s a lot of it), issue the command man curl and read through the manual page.
Learn more about Linux in TechRepublic Academy!
Also see:
- How to use the Linux watch command for easier output tracking (TechRepublic)
- How to use the Linux screen command to keep your remote processes running (TechRepublic)
- How to set up an SFTP server on Linux (TechRepublic)
- How to monitor a Linux log file in real time (TechRepublic)
- ICANN makes last minute WHOIS changes to address GDPR requirements (ZDNet)
