It is tempting to think of blocking as a list: your IP is on it, or it is not. That model is about a decade out of date, and it leads people to fix the wrong thing - rotating IPs harder against a problem that has nothing to do with IPs.
What actually happens is scoring. A site collects a handful of signals, weights them, and sets a threshold. Cross it and you get a block, a challenge, or - increasingly - quietly wrong data. Here is what goes into that score.
Signal one: where the address lives
Every IP belongs to an ASN, and ASNs are public. Anyone can look up that a given range belongs to a hosting company rather than a consumer ISP. That check is free, instant, and catches the majority of unsophisticated scraping.
This is the entire reason residential proxies exist. Not because they are faster - they are slower - but because the address resolves to a real ISP with real subscribers behind it, and the cheapest possible check comes back clean.
It is also why datacenter proxies are not useless. Against a site that does no ASN check, they are faster and cost a fraction as much. The economics only fail when the target actually looks.
Signal two: what your client looks like
Here is where most people get blocked without understanding why.
When your client opens a TLS connection it announces a very specific set of things: cipher suites, in a particular order, extensions, elliptic curves, protocol versions. That combination is stable per client and different between clients. Python's requests library has one signature. Chrome has another. They are not remotely similar, and telling them apart takes no effort at all.
So a request can arrive from a perfectly clean residential IP in the right country, with a flawless User-Agent claiming to be Chrome, and be rejected instantly - because the TLS handshake underneath said Python and the two do not match. That mismatch is a stronger signal than either fact alone. It does not just look automated, it looks like automation trying to hide.
Rotating IPs against this changes nothing. You will burn through a pool and see identical failures, because the IP was never the problem.
Signal three: how you behave
Humans are irregular. They pause, misclick, read for eleven seconds and then four, open things in a strange order, get distracted.
Automation is regular. Requests exactly 800 ms apart. Pages fetched in perfect sequence. No mouse movement, no scroll events, no dead time. None of these is damning alone, but together they describe something that does not browse like a person.
The practical upshot is that the fix is often slowing down rather than scaling up. A scraper doing 5 requests a second through 200 IPs looks far more automated than one doing 5 a minute through 10.
Signal four: the trap
Some sites plant links no human can click - hidden with CSS, positioned off-screen, sized zero by zero. A browser renders them invisible. A parser walking the DOM sees a link like any other.
Follow one and you have identified yourself perfectly. No ambiguity, no scoring, no false positive. It is the cleanest detection method there is, and it costs the site nothing.
How the signals combine
None of these is usually decisive alone. Sites weight them, and the weights differ enormously.
| Target type | Typically checks | What matters most |
|---|---|---|
| Public API, docs, open data | Little or nothing | Speed and price |
| E-commerce, listings | ASN, rate | IP type, pacing |
| Social platforms | ASN, fingerprint, account age | Session continuity |
| Ticketing, sneakers, finance | Everything, aggressively | All of it at once |
This is why "which proxy is best" has no answer without naming the target. Against the first row, the cheapest pool wins outright. Against the last, IP quality is table stakes and you will still fail on fingerprint if that is all you fix.
What to actually do about it
Diagnose before you spend. The mistake we see constantly is upgrading to a premium pool to solve a fingerprint problem, then concluding proxies do not work.
Work through it in order. Does a plain request from your own connection succeed? Then it is not the site. Does it succeed through the proxy with curl but fail from your scraper? Then it is your client, not the IP. Does it fail from every IP in a large pool, immediately, identically? That is a fingerprint or behavioural signature - no amount of rotation will move it.
Only once you have ruled those out is the IP the thing to change. Check what your browser is broadcasting before you assume otherwise - it takes a few seconds and it is free, and it settles the question that most people guess at.
Frequently asked questions
Can a site tell I am using a proxy?
It can often tell your IP belongs to a datacenter, because those ranges are published and easy to look up. It generally cannot tell that a residential IP is a proxy from the IP alone - which is why detection moved to fingerprinting and behaviour instead.
Do residential proxies get blocked?
Yes, just for different reasons. A residential IP rarely fails the address check, but it will still get blocked if your TLS fingerprint says automation or your request timing is inhumanly regular. The IP is one signal out of several.
Why does the same proxy work on one site and fail on another?
Because each site weights the signals differently. A news site may check nothing at all. A ticketing site may score ASN, fingerprint, timing and account history together. The proxy did not change - the threshold did.