How to Set a Proxy for wget: Step-by-Step

Setting a proxy for wget takes about 30 seconds once you know which of the four supported methods to use. wget reads its proxy configuration from environment variables, from ~/.wgetrc, from /etc/wgetrc, and from inline command-line flags — and the precedence is exactly the reverse: command-line flags win, then user .wgetrc, then system /etc/wgetrc, then environment variables. This guide walks through all four methods on Linux and macOS, covers HTTP/HTTPS/SOCKS5, shows authentication with username and password, explains the proxy-bypass list for internal hosts, and ships a working SpyderProxy configuration you can copy verbatim.

Every command in this guide was tested on Ubuntu 24.04 LTS and macOS 14 Sonoma with GNU Wget 1.25.0. SOCKS5 examples use proxychains4 4.16. The proxy used for testing is SpyderProxy Premium Residential at $2.75/GB.

Quick Start: 30-Second Setup

If you just want it working right now, this is the fastest path. Replace USERNAME, PASSWORD, and the gateway with your SpyderProxy credentials:

export http_proxy="http://USERNAME:[email protected]:7777"
export https_proxy="$http_proxy"
export no_proxy="localhost,127.0.0.1,::1"

wget https://httpbin.org/ip

That is it. Every wget command in the same shell session will route through SpyderProxy. To make the change permanent, append those three export lines to your ~/.bashrc (or ~/.zshrc on macOS) and source the file. The rest of this guide explains why each option exists, when to use which method, and how to debug it when something fails.

How wget Discovers Proxy Settings (Precedence)

When wget starts up it walks four sources in order. Later sources override earlier ones:

1. Environment variables — http_proxy, https_proxy, ftp_proxy, no_proxy. wget also checks the uppercase versions (HTTP_PROXY, etc.) but the lowercase forms are recommended.
2. System config — /etc/wgetrc. Set by your distro / sysadmin.
3. User config — ~/.wgetrc. Set by you.
4. Command-line flags — -e use_proxy=yes, -e https_proxy=..., --proxy-user=..., --proxy-password=.... Highest precedence.

Pick one mechanism per use case. Mixing them is where most "why is wget bypassing my proxy?" questions come from.

Method 1: Environment Variables (Recommended for Most Users)

Environment variables are the simplest, the most portable across tools (curl, npm, pip, apt, and many others honour the same variable names), and the easiest to script. They are also the right answer for short-lived shells, CI pipelines, and Docker containers.

Temporary (current shell only)

export http_proxy="http://USERNAME:[email protected]:7777"
export https_proxy="$http_proxy"
export no_proxy="localhost,127.0.0.1,::1,*.internal.example.com"

wget https://example.com

Permanent (every new shell)

Append the three exports to your shell's startup file. On Ubuntu and most Linux distributions this is ~/.bashrc; on macOS Sonoma it is ~/.zshrc:

# Linux
echo 'export http_proxy="http://USERNAME:[email protected]:7777"' >> ~/.bashrc
echo 'export https_proxy="$http_proxy"' >> ~/.bashrc
echo 'export no_proxy="localhost,127.0.0.1,::1"' >> ~/.bashrc
source ~/.bashrc

# macOS
echo 'export http_proxy="http://USERNAME:[email protected]:7777"' >> ~/.zshrc
echo 'export https_proxy="$http_proxy"' >> ~/.zshrc
echo 'export no_proxy="localhost,127.0.0.1,::1"' >> ~/.zshrc
source ~/.zshrc

Per-command (no shell pollution)

http_proxy="http://USERNAME:[email protected]:7777" \
  https_proxy="http://USERNAME:[email protected]:7777" \
  wget https://example.com

This invocation sets the proxy only for that single wget call without touching your shell environment. Useful in Makefiles and one-off scripts.

Method 2: ~/.wgetrc (Recommended for wget-Only Use)

If you want wget to use a proxy but you do not want curl, npm, pip, and apt to also pick it up, set the proxy in ~/.wgetrc instead of environment variables. This file is wget-specific.

Create or edit ~/.wgetrc with these lines:

# ~/.wgetrc
use_proxy = on
http_proxy = http://gw.spyderproxy.com:7777
https_proxy = http://gw.spyderproxy.com:7777
proxy_user = USERNAME
proxy_password = PASSWORD
no_proxy = localhost,127.0.0.1,::1

Set the file permissions to 600 so that other users on the machine cannot read your password:

chmod 600 ~/.wgetrc

Now every wget invocation will route through SpyderProxy automatically:

wget https://httpbin.org/ip

Method 3: Command-Line Flags (One-Off Overrides)

For one-off downloads that need a different proxy than your defaults, pass flags inline. The relevant ones are -e (set a runtime config option, same syntax as .wgetrc), --proxy-user, and --proxy-password.

wget \
  -e use_proxy=yes \
  -e http_proxy=gw.spyderproxy.com:7777 \
  -e https_proxy=gw.spyderproxy.com:7777 \
  --proxy-user=USERNAME \
  --proxy-password=PASSWORD \
  https://httpbin.org/ip

Command-line flags override every other source, so this works even when your environment variables or ~/.wgetrc point at a different proxy. To explicitly bypass any configured proxy for one call, use --no-proxy:

wget --no-proxy https://internal.example.com

Method 4: /etc/wgetrc (System-Wide)

If you administer a multi-user Linux machine and want every user to route wget through the same proxy by default, edit /etc/wgetrc as root. The syntax is identical to ~/.wgetrc:

sudo nano /etc/wgetrc

# add or uncomment these lines:
use_proxy = on
http_proxy = http://gw.spyderproxy.com:7777
https_proxy = http://gw.spyderproxy.com:7777
proxy_user = USERNAME
proxy_password = PASSWORD
no_proxy = localhost,127.0.0.1,::1,.internal.example.com

Individual users can still override system settings with their own ~/.wgetrc or environment variables.

Proxy Authentication (Username and Password)

SpyderProxy and most paid residential providers require authentication. There are three places to put credentials, in increasing safety:

1. Inline in the URL (least safe — appears in process list and shell history)

export http_proxy="http://USERNAME:[email protected]:7777"

2. Separate flags (safer — passwords still appear in process list)

wget --proxy-user=USERNAME --proxy-password=PASSWORD \
  -e use_proxy=yes -e http_proxy=gw.spyderproxy.com:7777 \
  https://example.com

3. ~/.wgetrc with chmod 600 (safest — password is stored in a 0600 file only you can read)

# ~/.wgetrc
proxy_user = USERNAME
proxy_password = PASSWORD

chmod 600 ~/.wgetrc

For automated scripts, prefer environment variables sourced from a .env file (also chmod 600) rather than hardcoding credentials in the script itself.

URL-encoding special characters in passwords

If your password contains @, :, /, ?, #, or &, you must URL-encode them or wget will misparse the URL. Quick lookup:

Character	Encoded
@	%40
:	%3A
/	%2F
?	%3F
#	%23
&	%26
+	%2B
space	%20

If your password is p@ssw0rd!, the URL-encoded form is p%40ssw0rd%21:

export http_proxy="http://user:p%40ssw0rd%[email protected]:7777"

Setting wget to Use a SOCKS5 Proxy

wget does not support SOCKS5 natively. To route wget through SOCKS5 you wrap the call with proxychains4 (Linux + macOS) or tsocks (Linux). proxychains4 is the more actively maintained option.

Install proxychains4

# Ubuntu / Debian
sudo apt install proxychains4

# macOS
brew install proxychains-ng

Configure /etc/proxychains4.conf (or ~/.proxychains/proxychains.conf)

strict_chain
proxy_dns
remote_dns_subnet 224
tcp_read_time_out 15000
tcp_connect_time_out 8000

[ProxyList]
socks5 gw.spyderproxy.com 1080 USERNAME PASSWORD

Run wget through proxychains4

proxychains4 wget https://httpbin.org/ip

You will see proxychains4 print connection info on stderr ([proxychains] Strict chain ... 127.0.0.1:1080 ... gw.spyderproxy.com:1080 ... OK) followed by wget's normal output. The destination sees the SpyderProxy egress IP.

Using wget with a Rotating Residential Proxy

SpyderProxy's rotating residential gateway issues a new IP per TCP connection. Each fresh wget invocation gets a different residential IP automatically:

# Each call gets a different residential IP
for i in {1..10}; do
  wget -qO- https://httpbin.org/ip
done

For sticky sessions (10-minute or 24-hour persistence), append a session ID to your username. Use the same suffix to keep the same IP; change it to rotate:

export http_proxy="http://USERNAME-session-abc123:[email protected]:7777"
export https_proxy="$http_proxy"

wget https://example.com    # IP A
wget https://example.com    # IP A (same session)

# rotate by changing the session id
export http_proxy="http://USERNAME-session-def456:[email protected]:7777"
wget https://example.com    # IP B

no_proxy: Excluding Internal Hosts

The no_proxy environment variable (also no_proxy in .wgetrc) tells wget which hosts and domains should bypass the proxy. Standard practice is to always exclude loopback and any internal corporate domains:

export no_proxy="localhost,127.0.0.1,::1,.internal.example.com,.intranet"

The leading dot on a domain matches that domain and all subdomains. .internal.example.com matches both foo.internal.example.com and internal.example.com itself.

Verifying the Proxy Is Active

The fastest sanity check is fetching https://httpbin.org/ip twice — once with the proxy, once without — and confirming the IPs differ:

# Without proxy (your real IP)
wget --no-proxy -qO- https://httpbin.org/ip

# With proxy (SpyderProxy egress IP)
wget -qO- https://httpbin.org/ip

The first command prints your home/office IP. The second prints a residential IP from somewhere in the SpyderProxy pool — different country, different ASN, different reverse DNS. You can also use our free IP Lookup and Proxy Checker tools for richer output (geolocation, ASN, blacklist status).

Troubleshooting Common Problems

Problem: 407 Proxy Authentication Required

Your credentials are wrong, missing, or improperly URL-encoded. Re-check:

• Username and password match exactly what your SpyderProxy dashboard shows.
• Special characters in the password are URL-encoded.
• If using ~/.wgetrc, the file has chmod 600 and is in your home directory.

Problem: 502 Bad Gateway or connection refused

The gateway hostname or port is wrong, or your firewall blocks egress. Confirm with:

curl -v -x http://USER:[email protected]:7777 https://httpbin.org/ip

If curl fails too, the issue is network or credentials, not wget.

Problem: wget ignores my proxy settings

Check precedence. Command-line flags override ~/.wgetrc which overrides /etc/wgetrc which overrides environment variables. Some scripts call wget --no-proxy explicitly. Run wget -d for debug output that prints which proxy wget is actually using.

Problem: SSL certificate problem

Update your CA bundle: sudo apt install --reinstall ca-certificates on Linux, or brew install ca-certificates on macOS. Never use --no-check-certificate in production — it disables TLS verification. SpyderProxy gateways use publicly-trusted certificates so this should never be necessary on our service.

Problem: HTTPS works but HTTP fails (or vice versa)

You set only one of http_proxy / https_proxy. Set both. Most providers including SpyderProxy use the same gateway URL for both schemes.

Performance Tips

• Limit rate for politeness against the destination: wget --limit-rate=200k.
• Random wait between requests in batch mode: wget --wait=2 --random-wait -i urls.txt.
• Tries and timeouts: --tries=10 --timeout=30 --retry-connrefused.
• Parallel downloads via xargs: cat urls.txt | xargs -P 8 -n 1 wget.
• User-Agent to avoid default-wget blocking: --user-agent="Mozilla/5.0 (compatible; SpyderBot/1.0)".

Bottom Line

For most users, three lines in ~/.bashrc or ~/.zshrc are all you need to put wget behind a proxy. Use environment variables for cross-tool config, ~/.wgetrc for wget-only config, command-line flags for one-off overrides, and /etc/wgetrc for system-wide policy. For SOCKS5, wrap wget with proxychains4. SpyderProxy supports all of these patterns out of the box — Premium Residential at $2.75/GB ships rotating sessions, sticky sessions up to 24 hours, 130M+ IPs across 195+ countries, and SOCKS5 + HTTP/HTTPS gateways. For the protocol-level differences between wget and curl, see our curl vs wget deep-dive.