Quick playbook: Detection in 2026 stacks across five layers. (1) IP reputation — use residential or mobile proxies, not datacenter. (2) TLS fingerprint (JA3/JA4) — impersonate a real Chrome via curl_cffi or browser. (3) HTTP headers — full browser header set in correct order, real User-Agent, real Accept-Language. (4) JS runtime / browser fingerprint — use Patchright / undetected-playwright; never bare Playwright with defaults. (5) Behavior — human pacing, organic click paths, cookie persistence. Skip any one of these and a modern anti-bot system (Cloudflare, DataDome, PerimeterX, Akamai, Kasada) will catch you.
| System | Primary signal | Catches |
|---|---|---|
| Cloudflare WAF / Bot Fight Mode | IP rep + JA4 + JS challenge | ~80% of automation |
| DataDome | Fingerprint + behavior | Aggressive on e-commerce, news |
| PerimeterX (HUMAN) | JS challenge + telemetry | Sneakers, ticketing |
| Akamai Bot Manager | Multi-layer scoring | Enterprise sites, airlines |
| Kasada | JS challenge + Web Crypto | Sneakers, ticketing |
| Imperva (formerly Incapsula) | Behavioral + signature | Financial, e-commerce |
| F5 Distributed Cloud Bot Defense | Behavior + fingerprint | Banking, retail |
Anti-bot systems classify IPs by ASN, geolocation, and historical bot activity. The first 50ms of any detection check is an ASN lookup; datacenter ASNs (AWS, Hetzner, OVH, Vultr, Linode, DigitalOcean) are pre-flagged. Most aggressive systems will challenge any request from these ASNs.
Pricing: Budget Residential $1.75/GB · Premium Residential $2.75/GB · LTE Mobile $2/IP/month.
Every TLS client — Chrome, Firefox, curl, Python requests, Go's net/http — sends a unique ClientHello message. The ordered list of cipher suites, extensions, and ALPN protocols becomes a fingerprint (JA3 historically; JA4 since 2023 is the new standard). Anti-bot systems hash this fingerprint and check it against a list of known clients. Python requests has a JA4 hash that screams "Python script."
Fix with TLS impersonation:
# curl_cffi — drop-in requests replacement with Chrome TLS
from curl_cffi import requests
r = requests.get("https://target.com", impersonate="chrome131")
print(r.status_code, r.text[:200])
# Other impersonations: chrome131, firefox133, edge131, safari17_5For non-Python: curl-impersonate, azuretls-client (Go), tls-client (Node). When using a real browser (Playwright) you get Chrome's real TLS for free.
A real Chrome 131 sends ~10 headers in a specific order with specific values. Python requests sends 4 with default values that scream "bot." Match Chrome exactly:
HEADERS = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/131.0.0.0 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,"
"image/avif,image/webp,image/apng,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9",
"Accept-Encoding": "gzip, deflate, br, zstd",
"Sec-Ch-Ua": '"Chromium";v="131", "Google Chrome";v="131", "Not.A/Brand";v="24"',
"Sec-Ch-Ua-Mobile": "?0",
"Sec-Ch-Ua-Platform": '"Windows"',
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "none",
"Sec-Fetch-User": "?1",
"Upgrade-Insecure-Requests": "1",
}For subsequent requests within a session change Sec-Fetch-Site to same-origin and add the actual Referer. Header order matters — Python's requests reorders alphabetically; use curl_cffi or httpx with explicit ordering.
When the site renders JavaScript, the browser env exposes ~30 signals: navigator.webdriver, plugin list, language, screen resolution, WebGL renderer, AudioContext fingerprint, canvas fingerprint, font list, timezone offset, hardware concurrency, deviceMemory. Bare Puppeteer/Playwright leaks "I'm a headless Chrome" in seconds:
navigator.webdriver === trueFix with stealth libraries:
# Patchright (the maintained successor to playwright-stealth)
from patchright.sync_api import sync_playwright
with sync_playwright() as p:
browser = p.chromium.launch(
headless=True,
proxy={"server": "http://USER:[email protected]:8000"},
)
page = browser.new_page(
viewport={"width": 1920, "height": 1080},
user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
locale="en-US",
timezone_id="America/New_York",
)
page.goto("https://target.com")
html = page.content()
browser.close()Also consider: undetected-chromedriver (Selenium), nodriver (Python, the modern UC successor), Camoufox (Firefox-based, harder to fingerprint).
Modern systems profile your behavior over a session: mouse movements, scroll patterns, key timings, click sequences, page-load reactions. Specifically:
localStorage. Use Playwright's storage_state to persist across runs.# Human-like pacing
import random, time
time.sleep(random.uniform(1.5, 4.0))
# Synthetic mouse + scroll with Playwright
page.mouse.move(random.randint(100, 1800), random.randint(100, 900))
page.mouse.wheel(0, random.randint(200, 600))
time.sleep(random.uniform(0.5, 1.5))| Layer | Minimum | Recommended |
|---|---|---|
| IP | Residential proxy | Mobile or residential with sticky session |
| TLS | curl_cffi impersonate | Real Playwright TLS |
| Headers | Full Chrome header set, correct order | Same + correct Referer + cookies |
| JS env | Patchright / nodriver | Same + storage_state + locale match |
| Behavior | Random sleeps | Mouse moves + scrolls + organic paths |
https://nopecha.com/demo/cloudflareRelated: How to bypass CAPTCHAs · FlareSolverr guide · Cloudscraper tutorial · Browser fingerprinting.