spyderproxy

Best User Agents for Web Scraping (2026): List + Rotation

D

Daniel K.

|
Published date

Wed May 27 2026

|9 min read

The best user agents for web scraping are the current, real strings sent by mainstream browsers — the latest Chrome on Windows, Chrome and Safari on macOS, Firefox, and modern mobile Chrome and Safari. Using a popular, up-to-date user agent makes your scraper blend in with the crowd; using a rare, outdated, or obviously-bot string makes it stand out and get blocked. But a user agent is only one signal, and rotating it without matching the rest of your request is a common way to get caught.

This guide gives you a copy-paste list of strong 2026 user agents, shows how to rotate them properly in Python, and explains the part most tutorials skip: why the user agent has to be consistent with your other headers and your IP to actually work.

What a User Agent Is and Why It Matters

The User-Agent is an HTTP request header your client sends with every request. It identifies the browser, its version, the operating system, and the rendering engine. A real Chrome request announces itself with something like a long Mozilla/5.0 string, and servers use it to decide how to render the page — and, increasingly, whether the visitor looks human.

For scraping, the user agent matters for two reasons. First, many clients send a dead-giveaway default — Python's requests library sends "python-requests/2.x" unless you override it, which is an instant flag. Second, anti-bot systems profile the user agent against the rest of the request: a Chrome user agent that arrives without the headers a real Chrome would send is more suspicious than no user agent at all. Getting this right is part of the broader topic of avoiding detection while scraping.

Best User Agents for Scraping in 2026 (Copy-Paste List)

These are realistic, current-generation strings across the most common browser/OS combinations. Keep them updated as browsers release — a user agent claiming a version that no longer exists is a flag.

Desktop

# Chrome on Windows 11
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36

# Chrome on macOS
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36

# Edge on Windows 11
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0

# Firefox on Windows
Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0

# Safari on macOS
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.2 Safari/605.1.15

Mobile

# Chrome on Android
Mozilla/5.0 (Linux; Android 14; Pixel 8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Mobile Safari/537.36

# Safari on iPhone (iOS)
Mozilla/5.0 (iPhone; CPU iPhone OS 18_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.2 Mobile/15E148 Safari/604.1

Why these and not something exotic? Because the goal is to look common. The most effective user agent for scraping is the one millions of real people are also sending right now. A rare browser on a rare OS is memorable, and memorable is the opposite of what you want.

How to Rotate User Agents in Python

Rotating a small pool of current user agents spreads your requests across several apparent browsers. Here is the basic pattern with the requests library:

import random
import requests

USER_AGENTS = [
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0",
]

def make_headers(ua):
    return {
        "User-Agent": ua,
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
        "Accept-Language": "en-US,en;q=0.9",
        "Accept-Encoding": "gzip, deflate, br",
        "Connection": "keep-alive",
        "Upgrade-Insecure-Requests": "1",
    }

ua = random.choice(USER_AGENTS)
proxy = "http://USER:[email protected]:7777"
r = requests.get(
    "https://example.com",
    headers=make_headers(ua),
    proxies={"http": proxy, "https": proxy},
    timeout=30,
)
print(r.status_code)

Notice that the user agent never travels alone — it ships with a full set of headers a real browser would send. That is the part most rotation snippets miss.

Why a User Agent Alone Won't Stop Blocks

Rotating user agents is necessary but nowhere near sufficient. Modern anti-bot systems cross-check several things, and an inconsistency anywhere gives you away:

  • The IP is the bigger signal. A hundred different user agents all coming from one datacenter IP is more suspicious than one user agent from a residential IP. User-agent rotation without IP rotation fools nobody.
  • Headers must match the user agent. A Chrome user agent should arrive with Chrome's Sec-CH-UA client-hint headers and Chrome's header ordering. Claim Chrome but send Firefox-shaped headers and you fail the cross-check.
  • TLS and HTTP fingerprints leak the real client. Tools profile your TLS handshake (JA3/JA4) and HTTP/2 frame order. A Python client claiming to be Chrome has a Python TLS fingerprint, which no user agent can hide. This is part of browser fingerprinting.
  • JavaScript challenges ignore the user agent entirely. Pages behind Cloudflare or similar run JS checks a plain HTTP client can't pass — see how to bypass Cloudflare and use a headless browser for those.

The takeaway: user-agent rotation is the floor, not the ceiling. It only works as one layer on top of quality residential IPs and consistent, browser-accurate headers.

Common User-Agent Mistakes

  • Leaving the default. Shipping "python-requests/2.x" or "Go-http-client/1.1" is the single most common scraping mistake.
  • Using stale versions. A user agent claiming Chrome 90 in 2026 is years out of date and stands out. Refresh your list periodically.
  • Mismatching UA and headers. A mobile user agent with desktop Accept headers, or a Chrome UA with no Sec-CH-UA hints, fails cross-checks.
  • Rotating UA but not IP. Many user agents from one IP is a pattern, not camouflage.
  • Spoofing a browser without rendering JS. If the site needs JavaScript, claiming to be a browser while behaving like a raw HTTP client gets caught immediately.

Frequently Asked Questions

What is the best user agent for web scraping?

The best user agent is a current, common one — typically the latest Chrome on Windows, since that is the most widespread browser/OS combination. The goal is to blend in with the largest group of real users, so popular and up-to-date beats rare or exotic every time.

Should I rotate user agents when scraping?

Yes, rotating a small pool of current user agents helps, but only as one layer. It must be combined with rotating residential IPs and headers that match each user agent. Rotating user agents from a single datacenter IP does not prevent blocks.

Why does my scraper get blocked even with a real user agent?

Because the user agent is only one signal. Anti-bot systems also check your IP reputation, whether your headers match the claimed browser, your TLS and HTTP fingerprints, and whether you can pass JavaScript challenges. A real user agent on a flagged datacenter IP with mismatched headers still gets blocked.

How often should I update my user agent list?

Refresh it every few browser releases, roughly every couple of months. Browsers update frequently, and a user agent claiming a version that no longer exists becomes a flag rather than camouflage.

Do I need proxies if I rotate user agents?

Yes. The IP address is a stronger signal than the user agent. Without rotating residential proxies, all your varied user agents still come from the same address, which is an obvious pattern. User-agent rotation and IP rotation work together, not as substitutes.

Can I just copy a user agent from my own browser?

Yes, that gives you one real, current string, and it is a fine starting point. For scraping at volume, build a small pool of current strings across a few browser and OS combinations and rotate them alongside your proxies and matching headers.

Conclusion

The best user agents for web scraping are current, common browser strings — and the right way to use them is as one layer in a larger disguise, never on their own. Set a real user agent, ship the matching headers, rotate a small pool, and most importantly route everything through rotating residential IPs so your varied browsers come from varied addresses. That combination is what actually keeps a scraper unblocked.

The hardest layer to fake is the IP, and it is the one a user agent can never substitute for. SpyderProxy residential proxies start at $1.75/GB with 10M+ IPs across 195+ countries, automatic rotation, and city-level targeting — the foundation your rotated user agents sit on top of.

The Layer a User Agent Can't Fake

Rotate user agents all you want — the IP is the real signal. SpyderProxy residential proxies from $1.75/GB: 10M+ IPs, 195+ countries, automatic rotation, city-level targeting.