An HTTP 504 Gateway Timeout means a server acting as a gateway or proxy waited for a response from an upstream server and never got one in time. The gateway did its job — it forwarded your request — but the machine behind it was too slow or unreachable, so the gateway gave up and returned 504. It is fundamentally a waiting error, not a refusal error.
This guide breaks down what a 504 really means, how it differs from 502 and 503, the most common causes on both sides of the connection, and the concrete fixes whether you are scraping a slow site, browsing, or running the server that throws it.
HTTP 504, defined in RFC 9110, is returned by an intermediary — a reverse proxy, load balancer, CDN, or API gateway — when it does not receive a timely response from the upstream server it is trying to reach. The intermediary accepted your request and tried to fulfill it, but the backend (an application server, database, or another service) took longer than the configured timeout allowed.
The request chain usually looks like: client to gateway (Nginx, Cloudflare, AWS ALB) to upstream (your app server). A 504 is the gateway saying: "I asked the upstream and waited, but it never answered, so here is a timeout."
These timeout and gateway codes are constantly confused. Here is the clean distinction:
| Code | Meaning | Who is at fault |
|---|---|---|
| 502 Bad Gateway | Gateway got an invalid or broken response from upstream | Upstream returned garbage or crashed mid-response |
| 503 Service Unavailable | Server is temporarily unable to handle the request | Server is overloaded or in maintenance |
| 504 Gateway Timeout | Gateway waited for upstream and got nothing in time | Upstream too slow or unreachable |
| 408 Request Timeout | Server gave up waiting for the client to send the request | The client was too slow to send |
The simplest mental model: 408 means you (the client) were too slow to ask. 504 means the upstream was too slow to answer. 502 means the upstream answered with something broken. 503 means the server is up but deliberately not serving right now.
Give slow targets room to respond. A 20 to 30 second read timeout is reasonable for heavy pages:
import requests
proxy = "http://USERNAME:[email protected]:12321"
r = requests.get(
"https://slow-target.com/page",
proxies={"http": proxy, "https": proxy},
timeout=(10, 30), # 10s connect, 30s read
)Treat 504 as retryable. Back off so you do not pile onto an already-slow server:
import time, requests
def fetch(url, proxies, attempts=4):
delay = 2
for i in range(attempts):
try:
r = requests.get(url, proxies=proxies, timeout=(10, 30))
if r.status_code != 504:
return r
except requests.exceptions.Timeout:
pass
time.sleep(delay)
delay = delay * 2 # 2s, 4s, 8s
return NoneIf your own load is making the target slow, dial back. Fewer concurrent requests per target domain often eliminates 504s entirely. Two to five concurrent requests per domain is a safe starting point.
A congested datacenter IP shared by many abusers can be throttled or deprioritized by the target, adding latency. Switching to a fresh residential IP or a low-latency static datacenter proxy reduces both throttling and round-trip time.
Where the site supports it, use pagination, filters, or an API endpoint that returns smaller payloads instead of scraping a giant page that strains the server.
If you operate the gateway, a 504 means your upstream is too slow or unreachable. Common fixes on Nginx:
# nginx: give a slow but legitimate upstream more time
location /api/ {
proxy_pass http://app_backend;
proxy_connect_timeout 10s;
proxy_read_timeout 120s;
proxy_send_timeout 120s;
}When scraping, the proxy is part of the gateway chain, so its speed matters. A slow, oversubscribed proxy pool adds latency to every request, and that extra time can be the difference between a 200 and a 504 against an already-slow target. High-quality proxies reduce 504s in two ways: lower latency per request, and cleaner IPs that targets are less likely to throttle. SpyderProxy offers low-latency static datacenter proxies with 99.9% uptime at $1.50/proxy/month, and rotating residential from $1.75/GB for high-trust access.
It means an intermediary server such as a reverse proxy, load balancer, or CDN forwarded your request to an upstream server and did not receive a response within its timeout window. The gateway is fine; the upstream was too slow or unreachable.
A 502 Bad Gateway means the upstream returned an invalid or broken response. A 504 Gateway Timeout means the upstream returned nothing at all in time. Both involve the gateway, but 502 is a bad answer and 504 is no answer.
Usually the website's, since the upstream behind its gateway was too slow. But it can be partly yours if your client timeout is too short, your connection is unstable, or your scraping load is what is slowing the target down in the first place.
Increase your read timeout to 20 to 30 seconds, retry with exponential backoff, reduce concurrency per domain, and rotate to a faster, cleaner IP. If your own load is causing it, slowing down is the most effective fix.
Yes. A slow or overloaded proxy adds latency to every request, and combined with a slow target the total can exceed your timeout. A low-latency proxy with clean IPs reduces both the delay and the throttling that lead to 504s.
Increase proxy_read_timeout and proxy_send_timeout for the slow location, but more importantly fix the slow upstream: optimize database queries, scale workers, add health checks, and cache expensive responses. Raising the timeout alone only hides the problem.
HTTP 504 is the chain telling you that something upstream took too long to answer. As a visitor, refresh and wait. As a scraper, lengthen your timeouts, retry with backoff, ease off the throttle, and run through fast, clean proxies. As a server owner, treat the timeout as a symptom and fix the slow backend behind it. In every case, the fix follows from identifying which link in the gateway chain ran out of patience.
For low-latency proxies that keep timeouts rare, start with SpyderProxy static datacenter at $1.50/proxy/month or residential from $1.75/GB.