The complete 2026 guide to defeating Cloudflare bot detection — proxy selection, TLS fingerprinting, JS challenges, Turnstile, and working Python code.
Daniel K.
Apr 15, 2026
Cloudflare sits in front of roughly 20% of the entire web — including most sites worth scraping. Its bot protection is the single biggest reason scrapers fail. Hit a Cloudflare-protected site with a basic Python script and you'll see 403 Forbidden, error 1020, or a blank challenge page within seconds.
The good news: Cloudflare can be bypassed reliably in 2026, but only if you understand how it detects bots and pair the right proxies with the right techniques. This guide walks through exactly how to do it, with working code and proxy configurations that actually ship to production.
Cloudflare is a CDN and security layer that sits between visitors and the origin server. When you request a page, Cloudflare inspects the request before the site's server ever sees it. If anything looks automated, Cloudflare challenges or blocks you.
Cloudflare's detection stack runs on multiple layers, each of which has to be defeated:
| Detection Layer | What It Checks | How to Defeat It |
|---|---|---|
| IP Reputation | Known datacenter ASNs, blocklists, abuse history | Use residential or mobile proxies |
| TLS Fingerprinting (JA3/JA4) | TLS handshake signature | Match a real browser's JA3 (curl_cffi, tls-client) |
| HTTP/2 Fingerprinting | Frame order, settings, pseudo-headers | Use browser-like HTTP/2 clients |
| Headers & Order | Accept-Language, order of headers, sec-ch-ua | Send full browser header set in correct order |
| JavaScript Challenge | JS execution + canvas/WebGL fingerprint | Use headless browser with stealth patches |
| Turnstile / hCaptcha | Interactive widget, behavioral signals | Solver service or session reuse |
| Rate Limiting | Requests per IP per window | Rotate IPs with sticky sessions |
Most "bypass Cloudflare" tutorials only address one or two layers. That's why they work for ten minutes and then break. To bypass Cloudflare reliably, you need to pass all of them.
Cloudflare shows different challenges depending on your threat score. Knowing which one you're hitting tells you exactly what to fix.
A 5-second interstitial that runs JavaScript in your browser. If your client can't execute JavaScript, you never reach the actual page. Requires a real browser engine — not just requests library.
The successor to hCaptcha. An invisible or interactive widget that analyzes browser behavior, mouse movement, and fingerprint. You cannot bypass Turnstile without either solving it (captcha service) or maintaining a valid browser session cookie.
Cloudflare's adaptive challenge. Might be invisible, might show Turnstile, might require a tap. The challenge level depends on your risk score at that moment — which changes based on IP, headers, and behavior.
You've exceeded requests per window from a single IP. Not a bot detection — just a rate cap. The fix is straightforward: rotate through more IPs and slow down per-IP request rates.
Proxy choice is the foundation. Everything else — stealth browsers, header tuning, solver services — is wasted effort if your IP is already flagged.
| Proxy Type | Cloudflare Success Rate | Cost | Best For |
|---|---|---|---|
| Free / Public | 0-2% | Free | Nothing — all flagged |
| Datacenter | 10-30% | $ | Sites with low Cloudflare settings only |
| ISP / Static Residential | 70-85% | $$$ | Logged-in sessions, account management |
| Rotating Residential | 85-93% | $$ | High-volume scraping, product data |
| Mobile 4G/5G | 95-98% | $$$$ | Enterprise-grade, highest-value targets |
Why datacenter proxies fail: Cloudflare identifies the ASN (Autonomous System Number) of every IP. ASNs from AWS, DigitalOcean, OVH, Hetzner, etc. are automatically scored as high-risk. Even clean, brand-new datacenter IPs start with negative reputation.
Why residential proxies work: Residential proxies use IPs assigned by actual ISPs to real homes. The ASN belongs to Comcast, BT, Telefonica — networks Cloudflare can't blanket-block without breaking the web for real users.
Why mobile proxies are nearly untouchable: Mobile 4G/5G proxies share carrier-grade NAT IPs with thousands of real phones. Cloudflare cannot ban a carrier IP without taking out every T-Mobile, Vodafone, or Verizon customer hitting that site.
Here are three production-tested approaches, from simplest to most robust.
The curl_cffi library wraps libcurl with Chrome's TLS fingerprint baked in. This alone passes basic Cloudflare checks on many sites when paired with residential proxies.
pip install curl_cffifrom curl_cffi import requests
PROXY_HOST = "gate.spyderproxy.com"
PROXY_PORT = "10000"
PROXY_USER = "your_username"
PROXY_PASS = "your_password"
proxies = {
"http": f"http://{PROXY_USER}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}",
"https": f"http://{PROXY_USER}:{PROXY_PASS}@{PROXY_HOST}:{PROXY_PORT}",
}
# Impersonate Chrome 124 — matches TLS + HTTP/2 fingerprint
response = requests.get(
"https://target-site.com/products",
proxies=proxies,
impersonate="chrome124",
timeout=30,
)
print(response.status_code)
print(response.text[:500])
This bypasses TLS fingerprinting, HTTP/2 fingerprinting, and header-order checks in a single library. It will not pass JS challenges or Turnstile — for that, you need a real browser.
When the site throws a JS challenge, you need an actual browser. Playwright with stealth plugins is the most reliable option.
pip install playwright playwright-stealth
playwright install chromiumfrom playwright.sync_api import sync_playwright
from playwright_stealth import stealth_sync
PROXY = {
"server": "http://gate.spyderproxy.com:10000",
"username": "your_username",
"password": "your_password",
}
def scrape_with_browser(url):
with sync_playwright() as p:
browser = p.chromium.launch(
headless=True,
proxy=PROXY,
args=[
"--disable-blink-features=AutomationControlled",
"--no-sandbox",
],
)
context = browser.new_context(
user_agent=(
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/124.0.0.0 Safari/537.36"
),
viewport={"width": 1920, "height": 1080},
locale="en-US",
timezone_id="America/New_York",
)
page = context.new_page()
stealth_sync(page)
page.goto(url, wait_until="networkidle", timeout=45000)
# Wait for any Cloudflare challenge to resolve
page.wait_for_load_state("networkidle", timeout=45000)
html = page.content()
browser.close()
return html
html = scrape_with_browser("https://target-site.com/products")
Stealth-patched Playwright defeats most JS challenges automatically by removing the navigator.webdriver flag, patching canvas/WebGL signatures, and normalizing timing behaviors.
FlareSolverr is a separate service that runs a full browser on your infrastructure and exposes a simple API to solve Cloudflare challenges. It handles JS challenges and basic Turnstile on its own.
# docker-compose.yml
version: "3"
services:
flaresolverr:
image: ghcr.io/flaresolverr/flaresolverr:latest
environment:
- LOG_LEVEL=info
- TZ=UTC
ports:
- "8191:8191"
restart: unless-stopped
import requests
def solve_cloudflare(url):
resp = requests.post(
"http://localhost:8191/v1",
json={
"cmd": "request.get",
"url": url,
"maxTimeout": 60000,
"proxy": {
"url": "http://gate.spyderproxy.com:10000",
"username": "your_username",
"password": "your_password",
},
},
)
data = resp.json()
if data["status"] == "ok":
return data["solution"]["response"] # Full HTML
return None
html = solve_cloudflare("https://target-site.com/products")
Run FlareSolverr behind a queue if you're processing high volume — each request holds a browser instance for the challenge duration (typically 5-15 seconds).
Getting past Cloudflare once is easy. Staying past it across thousands of requests is the real challenge.
TLS fingerprint, HTTP/2 frames, headers, canvas, WebGL, timezone, language, viewport — they all need to tell the same story. A US English user agent with a Tokyo timezone and a Paris IP will get flagged instantly.
If you're working with cookies, sessions, or carts, use a single IP for the full session. Rotating IPs mid-session looks exactly like account hijacking — Cloudflare flags this pattern immediately.
For stateless scraping (product pages, listings), rotate IPs on every request or every handful of requests. This spreads your fingerprint across enough IPs to stay under per-IP rate limits.
Real users don't hit 60 pages a minute. Add random delays between 2-8 seconds for standard scraping, longer for sensitive sites. The "most realistic" patterns include occasional longer pauses of 30-60 seconds, like a human taking a break.
Before hitting your target URL, visit the homepage, wait a few seconds, then navigate. A session that jumps straight to /api/products?page=100 looks like a bot. One that homepages → category → product looks like a customer.
When you hit a block, don't retry immediately on the same IP. Rotate the proxy, wait 30-60 seconds, and try again. Repeated retries from the same IP raise your threat score further.
Once you've passed a Cloudflare challenge, you get a cf_clearance cookie valid for 30-60 minutes. Reuse it across requests from the same IP. This dramatically reduces your solve rate and cost.
Running 200 concurrent threads from your server — even with different proxies — creates a burst signal Cloudflare can correlate. Scale horizontally across regions or cloud providers for large jobs.
| Error Code | Cause | Fix |
|---|---|---|
| Error 1020 | Firewall rule blocked you (WAF, country, ASN) | Rotate IP, check proxy ASN |
| Error 1015 | Rate limited from this IP | Slow down, rotate proxy |
| Error 1010 | Browser signature flagged | Use stealth-patched browser |
| Error 1006 | Your IP banned at the WAF layer | New IP from different subnet |
| Error 1003 | Direct IP access — need valid Host header | Use domain, not IP |
| 403 Forbidden | Generic block — usually TLS/JA3 mismatch | curl_cffi or real browser |
| 503 Service Unavailable | Under Attack Mode — site-wide JS challenge | Real browser only |
If you're hitting a mix of these, your fundamentals are wrong. Fix proxy type first, fingerprint second, pacing third.
Bypassing Cloudflare to access publicly available content generally falls under the same legal framework as web scraping. Key points:
For commercial scraping, consult a qualified attorney in your jurisdiction. This guide is technical, not legal advice.
For small volumes or local testing, yes — a stealth-patched browser from your home IP often works. For any production scraping, no. Your IP's request rate will trigger rate limits within minutes, and your home IP can end up on Cloudflare's flag list. Use residential or mobile proxies for anything beyond a handful of requests.
The old cloudscraper Python library is effectively dead against modern Cloudflare. It handled the legacy IUAM challenge (the old 5-second one) but does not solve Turnstile or modern managed challenges. Use curl_cffi for basic bypasses and Playwright or FlareSolverr for anything harder.
Turnstile is Cloudflare's in-house replacement for hCaptcha. Same concept (invisible behavioral analysis + optional visible challenge), but tuned specifically for Cloudflare's threat signals. Cloudflare is rolling out Turnstile as the default on new deployments. Both can be solved with captcha-solving services like 2Captcha or CapSolver.
For moderate scraping (100k requests/day), rotating residential proxies run roughly $50-150/month depending on bandwidth. Mobile proxies are 3-5x more expensive but achieve near-100% success rates. See our pricing page for current plans.
Default Playwright leaks the navigator.webdriver flag and other automation signals. Without stealth patches, it's detected within seconds. With playwright-stealth applied plus residential proxies and proper pacing, Playwright is reliable against 90%+ of Cloudflare deployments.
A combination: mobile or residential proxies for clean IPs, curl_cffi for stateless requests where TLS fingerprint matters, and stealth-patched Playwright or FlareSolverr for JS challenges and Turnstile. Cache cf_clearance cookies aggressively to minimize solve rates.