Does Python requests retry by default?

No. Plain requests.get() makes exactly one attempt and raises on failure. Retries require manually mounting urllib3.util.Retry onto a requests.Session via HTTPAdapter, or using a third-party library like tenacity.

What's a good backoff_factor for production?

1.0 is a reasonable default. That gives delays of roughly 1s, 2s, 4s, 8s, 16s. For latency-sensitive APIs use 0.5 (faster recovery). For heavily rate-limited APIs use 2.0 (less aggressive). Always cap with total=5 to bound total wait time.

Should I retry on 429 Too Many Requests?

Yes — 429 is the most retry-worthy status. The server is explicitly asking you to slow down. Always respect the Retry-After header (urllib3 does this automatically with respect_retry_after_header=True, the default in 1.26+). If no header, use exponential backoff.

Can I retry POST requests safely?

Only if the API is idempotent (uses idempotency keys, or POST is naturally idempotent like setting a value). Otherwise retrying POST can create duplicate resources — two new users, two charges, etc. Either remove POST from allowed_methods, or include an idempotency key like a UUID with each POST.

Why does my Retry config not seem to retry?

Three common causes: 1) You're using requests.get() directly instead of session.get() (the adapter is on the session). 2) The error type isn't in status_forcelist (4xx errors are skipped). 3) total=0 — check your Retry params. Add print debugging in a custom HTTPAdapter to confirm retries are firing.

What's the difference between requests Retry and tenacity?

urllib3.Retry is at the connection level (mounted via HTTPAdapter), retries before the response reaches your code. tenacity is a decorator at the function level — it sees your full function logic and can retry based on the result, not just status code. Tenacity is more flexible; Retry is built-in. Use Retry for HTTP retries, tenacity for application-level retries (e.g., retry if response JSON has a 'pending' status).

How do I add jitter to my retry backoff?

urllib3.Retry doesn't support jitter directly. Either use a manual retry loop with random.uniform() jitter, or use tenacity's wait_random_exponential. Jitter prevents thundering-herd when many clients retry at the exact same time after a server outage.

How do I rotate proxies between retries?

urllib3.Retry can't change the proxy mid-retry. You need a manual retry loop: each iteration picks a fresh proxy URL and calls requests.get() with that proxy. For SpyderProxy, append '-session-{random}' to the username for a fresh sticky-session IP each attempt.

Python Requests Retry on Failure (Complete 2026 Guide)

Quick verdict: Python's requests has no built-in retry — you mount urllib3.util.Retry onto a requests.Session via HTTPAdapter. Retry on connection errors, 429, and 5xx; never retry on 4xx (auth/permission errors); use exponential backoff (backoff_factor=1 gives 0.5s, 1s, 2s, 4s); cap at 3-5 attempts; respect Retry-After headers automatically.

The Standard Retry Pattern

from requests import Session
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

def make_session():
    s = Session()
    retry = Retry(
        total=5,                       # max 5 attempts total
        backoff_factor=1,              # 0.5, 1, 2, 4, 8 seconds
        status_forcelist=[429, 500, 502, 503, 504],
        allowed_methods=["GET", "POST", "PUT", "DELETE", "HEAD"],
        respect_retry_after_header=True,
    )
    adapter = HTTPAdapter(max_retries=retry)
    s.mount("http://", adapter)
    s.mount("https://", adapter)
    return s

session = make_session()
r = session.get("https://api.example.com/data", timeout=10)

Every parameter matters. Skip the next sections to read the rest of the article, or stay here for the why.

total=5

Maximum total retry attempts. Counts both connection failures and HTTP errors against the same budget. total=5 means up to 5 retries (so up to 6 actual requests including the first try).

Granular alternatives if you want different budgets per error type:

connect=3: max 3 retries on connection errors (DNS, connect refused)
read=3: max 3 retries on read errors (server closed connection mid-response)
redirect=5: max 5 redirects (separate from retries)
status=3: max 3 retries on bad HTTP statuses

If you set both total and a granular value, total wins as the global cap.

backoff_factor

Exponential backoff between retries. Formula: {backoff_factor} * (2 ** ({attempt} - 1)).

backoff_factor	Attempt 1	Attempt 2	Attempt 3	Attempt 4	Attempt 5
0.5	0s	1s	2s	4s	8s
1	0s	2s	4s	8s	16s
2	0s	4s	8s	16s	32s

Note: first retry has zero delay. The library logic is "wait {factor} * 2^(retry_count - 1) before retry n+1," so retry #1 (after the first failure) waits factor * 2^0 = factor seconds — but the table shows the cumulative delay, where the first retry happens after 0s if the request itself returned instantly.

Default: backoff_factor=0 — no backoff, retries immediately. Don't use the default. backoff_factor=1 is a sensible production value.

status_forcelist

HTTP statuses to retry on. The standard list:

429 — Too Many Requests. Almost always transient; respect Retry-After header.
500 — Internal Server Error. Server fault, may resolve.
502 — Bad Gateway. Upstream server issue.
503 — Service Unavailable. Server overloaded; retry usually works.
504 — Gateway Timeout. Upstream timed out.

Never retry on:

400 — bad request, your fault
401/403 — auth/permission, retry will not fix it
404 — resource gone, retry will not bring it back
410 — permanently gone

POST is a special case — some APIs are not idempotent, so retrying a POST can create duplicate resources. Either ensure the API supports idempotency keys, or remove POST from allowed_methods.

Retry-After Header

HTTP 429 and 503 often include a Retry-After header telling you when to retry. With respect_retry_after_header=True (default in urllib3 1.26+), urllib3 honors it — sleeps the specified time instead of using backoff_factor.

The header value can be:

Seconds: Retry-After: 120 (wait 120 seconds)
HTTP date: Retry-After: Mon, 15 Sep 2026 12:00:00 GMT (wait until that time)

If the header asks you to wait longer than your remaining retry budget, urllib3 raises MaxRetryError.

Connection Errors

For DNS failures and connection refused, urllib3 retries automatically with total=N. No status_forcelist needed:

retry = Retry(total=5, backoff_factor=1)
# This retries on connection errors AND timeouts

Connection-level retries handle: DNS failure, ConnectTimeout, ConnectionRefusedError, SSL handshake failures, broken pipe.

Retries Through a Proxy

If you are scraping behind a proxy and the proxy is flaky, the same Retry config applies:

session = make_session()
proxies = {
    "http":  "http://USER:PASS@gw.spyderproxy.com:8000",
    "https": "http://USER:PASS@gw.spyderproxy.com:8000",
}
r = session.get("https://target.com", proxies=proxies, timeout=15)

For rotating proxies, you might want to rotate IP between retries (not retry the same IP that just failed). Wrap the request in a manual retry loop and use a fresh proxy URL each iteration. See rotating proxies with Python requests for the pattern.

Manual Retry Loop (Full Control)

Sometimes urllib3's Retry is too rigid. A manual loop with custom logic:

import time, requests, random

def fetch_with_retry(url, max_attempts=5, base_delay=1.0):
    for attempt in range(1, max_attempts + 1):
        try:
            r = requests.get(url, timeout=10)
            if r.status_code == 200:
                return r
            if r.status_code in (429, 500, 502, 503, 504):
                # transient, retry
                pass
            else:
                r.raise_for_status()
        except (requests.ConnectionError, requests.Timeout):
            pass

        if attempt == max_attempts:
            raise RuntimeError(f"Failed after {max_attempts} attempts")

        # Exponential backoff with jitter
        delay = base_delay * (2 ** (attempt - 1))
        delay += random.uniform(0, delay * 0.1)  # 10% jitter
        time.sleep(delay)

    raise RuntimeError("unreachable")

The jitter prevents thundering-herd: if 1,000 clients all retry at exactly t+2s, they all hit the server at the same instant. Adding 0-10% random jitter spreads the load.

Library Alternative: tenacity

For the cleanest decorator-based retry, use tenacity:

pip install tenacity

from tenacity import retry, stop_after_attempt, wait_exponential, retry_if_exception_type
import requests

@retry(
    stop=stop_after_attempt(5),
    wait=wait_exponential(multiplier=1, min=1, max=60),
    retry=retry_if_exception_type((requests.ConnectionError, requests.Timeout)),
)
def fetch(url):
    r = requests.get(url, timeout=10)
    r.raise_for_status()
    return r

Tenacity has nicer composability than urllib3 Retry — you can retry_if_result, log each attempt, or stop on a specific exception. Trade-off: extra dependency.

Async / httpx

If you have moved to async with httpx, the pattern is similar but with tenacity's async support:

from tenacity import retry, stop_after_attempt, wait_exponential
import httpx

@retry(stop=stop_after_attempt(5), wait=wait_exponential(min=1, max=60))
async def fetch_async(url):
    async with httpx.AsyncClient(timeout=10) as client:
        r = await client.get(url)
        r.raise_for_status()
        return r

Common Mistakes

Retrying on 4xx: waste of time. 4xx is your fault; fix the request, do not retry it.
No timeout: retries on a hung connection wait forever. Always set timeout=N on every request.
Infinite retries: total=None means unlimited. Cap at 3-5 for production.
Same proxy on each retry: if the proxy is the problem, retrying the same proxy will not help. Rotate IPs between attempts.
Retrying non-idempotent POSTs: can create duplicate resources. Use idempotency keys, or only retry GET/PUT/DELETE.