"Use rotating proxies" is the standard advice, and for about half of all jobs it is actively wrong. Rotating your IP mid-session is how you get logged out, how carts empty, and how multi-step flows break. The real skill is knowing how long an identity should live — one request, ten minutes, or forever. Here is how the three modes differ and how to pick.
Why Rotation Exists
Rate limits are enforced per IP. One address making a thousand requests is the clearest bot signal there is, no matter how good your headers are. Rotation spreads that load across many addresses so no single one accumulates a suspicious count. That is the whole mechanism — and it is also why rotation solves nothing if the site is tracking you by session cookie instead.
The Three Modes
| Mode | IP changes | Best for | Breaks |
|---|---|---|---|
| Per-request | Every request | Bulk scraping of independent pages | Logins, carts, multi-step flows |
| Sticky session | After N minutes, or on demand | Checkout flows, paginated sessions, logged-in browsing | Nothing — but wastes IPs if held too long |
| Fixed / static | Never | Long-lived accounts, allowlisted access | Bulk scraping — one IP, one rate limit |
Per-request rotation
Each request exits from a different IP. Perfect for scraping thousands of independent product pages, search results or listings, where no request depends on the one before it. Maximum throughput per IP-reputation cost.
Sticky sessions
The same IP is held for a defined window — typically anywhere from a minute up to 24 hours. Essential whenever requests are related: logging in, walking a paginated result set that tracks state, adding to a cart, or anything where the server issued you a session cookie tied to your address.
Fixed IPs
A dedicated address that never changes, which is what static residential (ISP) proxies provide. Used for accounts that must always originate from the same place, or when a partner has allowlisted your IP. See static vs rotating proxies.
How Session Control Actually Works
With most residential providers you do not call an API to rotate — you encode the behaviour in the proxy username. A session identifier keeps you on one IP; changing it gets you a new one.
import requests
ENDPOINT = "geo.spyderproxy.com:12321"
def proxy_for(session=None, country=None):
user = "USERNAME"
if country: user += f"-country-{country}"
if session: user += f"-session-{session}" # same value = same IP
return {"http": f"http://{user}:PASSWORD@{ENDPOINT}",
"https": f"http://{user}:PASSWORD@{ENDPOINT}"}
# per-request rotation: no session id
for url in urls:
requests.get(url, proxies=proxy_for(country="us"), timeout=20)
# sticky: reuse one session id across a related sequence
s = "job42"
login = requests.post(LOGIN, data=creds, proxies=proxy_for(s, "us"))
page2 = requests.get(NEXT, proxies=proxy_for(s, "us")) # same IPCheck your dashboard for the exact username format and the maximum sticky duration your plan allows. The principle is identical across providers even where the syntax differs.
Choosing: One Question
Does this request depend on the previous one?
- No — independent pages, no cookies that matter → rotate per request.
- Yes — a login, a cart, a stateful paginated set → sticky session for the length of that flow, then drop it.
- Always the same identity — a managed account, an allowlisted integration → fixed IP.
Mixed jobs are normal. Use a sticky session to log in and reach the data, then rotate per request for the bulk collection that follows.
Five Rotation Mistakes
- Rotating mid-login. The site issued a session cookie to IP A; the next request arrives from IP B. Best case you are logged out, worst case flagged. Use a sticky session for the whole authenticated flow.
- Holding a sticky session too long. A 24-hour session doing continuous requests is just one IP taking all the load — you have recreated the problem rotation solves. Hold it for the flow, not the day.
- Rotating faster than a human could move. Twenty requests per second from twenty countries for one "user" is its own anomaly. Pace still matters — see how AI detects scrapers.
- Retrying a blocked IP. If a request is challenged, retry on a fresh IP with backoff. Repeating on the burned one deepens the block. Honour 429 and
Retry-After. - Mismatching geography. Rotating between countries mid-session, or a UK IP with a US locale and timezone, is a contradiction that is cheap to detect.
Beyond the Basics
- Rotate on failure, not just on schedule. Treat a 403 or CAPTCHA as a signal to change identity and slow down, not merely to try again.
- Separate identities per target. Do not carry the same session across unrelated sites; keep concurrency budgets per domain.
- Pin country deliberately. If the data is geo-dependent — pricing, search results, availability — the country is part of your dataset, not an accident. Fix it explicitly.
- Cap concurrency per IP. Ten parallel requests down one sticky session looks nothing like one person browsing.
The general pacing rules in web scraping best practices apply on top of all of this.
Frequently Asked Questions
What is proxy rotation?
Proxy rotation means sending requests through a changing set of IP addresses rather than one. Because rate limits are enforced per IP, spreading requests across many addresses stops any single one accumulating a suspicious request count and getting blocked.
What is a sticky session?
A sticky session holds the same exit IP for a defined period — typically a few minutes up to 24 hours — instead of rotating on every request. It is required whenever requests are related, such as logging in, filling a cart, or paginating through results that depend on server-side session state.
How often should I rotate proxies?
Rotate on every request when pages are independent. Use a sticky session for the length of any flow where requests depend on each other, then release it. There is no universal interval: the right answer is the lifetime of the task, not a fixed number of minutes.
Should I rotate proxies when logged in?
No. If you change IP mid-session the site sees a session cookie issued to one address arriving from another, which typically logs you out and can flag the account. Use a sticky session for the entire authenticated flow, and only rotate between separate sessions.
Conclusion
Rotation is not a setting you turn on, it is a decision about how long each identity should live. Independent pages rotate per request; anything stateful needs a sticky session for exactly as long as the flow lasts; accounts and allowlisted integrations need a fixed IP. Get that mapping right, retry on fresh IPs rather than burned ones, and keep geography consistent — and most "we keep getting blocked" problems disappear.
All three modes on one endpoint: SpyderProxy residential proxies from $2.75/GB — per-request rotation or sticky sessions up to 24h, across 195+ countries.