The wget command is one of the best ways to transfer files from a remote server to a local machine without a GUI. I use this tool constantly on Linux servers and it never fails to pull down the files I need. Unless I’m behind a proxy. When that’s the case, if you simply issue the standard wget command, you’ll receive errors and the file in question will fail to download.
What do you do?
Fortunately, the developers of wget considered this and built in the necessary options for using the tool when behind a proxy.
You might think it’s as simple as issuing a command like:
wget --proxy=PROXY FILE
Where PROXY is the proxy server you’re machine is behind, and FILE is the name of the remote file to download. That’s not exactly how this works. Fear not, I’m going to show you how to make it possible.
SEE: Security incident response policy (TechRepublic Premium)
What you’ll need
- A Linux server or desktop with wget installed
- A user with sudo privileges
How to configure wget for a proxy
To allow wget behind a proxy, you must edit a configuration file and add the necessary addresses. The best way to do this is via the global wget configuration file. To open the configuration file for editing, issue the command:
sudo nano /etc/wgetrc
In that file, you’ll find three lines:
#https_proxy = http://proxy.yoyodyne.com:18023/
#http_proxy = http://proxy.yoyodyne.com:18023/
#ftp_proxy = http://proxy.yoyodyne.com:18023/
You’ll need to uncomment (remove the # character) each necessary line for your proxy. If you are behind both HTTP and HTTPS proxies you’d configure those lines like:
https_proxy = HTTPSPROXY:PORT
http_proxy = HTTPPROXY:PORT
Where HTTPSPROXY is your HTTPS proxy server address, HTTPPROXY is your HTTP proxy address, and PORT is the required port for your proxy. Save and close the file.
There might be a situation where you don’t want all users to be able to get beyond the proxy. For that, you could set the proxy variables in a specific user’s .bashrc file. For that, you’d open the file for editing with the command:
sudo nano /home/USER/.bashrc
Where USER is the username.
At the bottom of the file, add the following:
export http_proxy=HTTPPROXY:PORT
export https_proxy=HTTPSPROXY:PORT
export ftp_proxy=FTPPROXY:PORT
Where:
- HTTPSPROXY is your HTTPS proxy server address
- HTTPPROXY is your HTTP proxy address
- FTPPROXY is your FTP proxy
- PORT is the required port for your proxy
Save and close the file.
Finally, if your proxy requires user authentication, the configuration lines would look like:
export http_proxy="http://USERNAME:PASSWORD@HTTPPROXY:PORT"
export https_proxy="http://USERNAME:PASSWORD@HTTPSPROXY:PORT"
export ftp_proxy="http://USERNAME:PASSWORD@FTPPROXY:PORT"
Where:
- USERNAME and PASSWORD are the credentials used for proxy authentication
- HTTPSPROXY is your HTTPS proxy server address
- HTTPPROXY is your HTTP proxy address
- FTPPROXY is your FTP proxy
- PORT is the required port for your proxy
At this point, you should be able to use wget, in the usual method, without the proxy preventing you from downloading the file.
Subscribe to TechRepublic’s How To Make Tech Work on YouTube for all the latest tech advice for business pros from Jack Wallen.
Image: iStock/iBrave
