How to Scrape Amazon Reviews With Python: A Practical Guide

Q: Why does my scraper get a Robot Check page?

That is Amazon flagging the request as automated, usually from a datacenter IP, too many requests from one address, or unrealistic headers. Switch to a fresh residential IP, add delays, and send a genuine User-Agent and Accept-Language header.

Amazon reviews are one of the richest public datasets on the internet — real opinions, ratings, and language from millions of buyers. Teams use them for product research, sentiment analysis, competitor monitoring, and building training data. The catch: Amazon is one of the most aggressively protected sites on the web. This guide covers where reviews live, the defenses you will hit, a working Python example, and why residential proxies are the part you cannot skip.

Where Amazon Reviews Live

Each product has an ASIN (Amazon Standard Identification Number) in its URL. Reviews appear in two places: a preview on the product page, and the full, paginated list on the dedicated reviews page:

https://www.amazon.com/product-reviews/<ASIN>/

That reviews page supports query parameters for paging and sorting — for example a pageNumber parameter to walk through results and a sort parameter to order by most recent. Amazon changes these details over time, so always confirm the current URL structure by browsing the live page first.

Why Amazon Is a Hard Target

Before writing a line of code, be realistic: Amazon actively fights scrapers. Expect all of these:

Datacenter IP blocks. Requests from cloud/hosting IP ranges are flagged almost instantly.
Rate limiting and bans. Too many requests from one IP triggers throttling, a "Robot Check" / CAPTCHA page, or a temporary ban.
Bot fingerprinting. Missing or unrealistic headers, no cookies, and non-browser request patterns get filtered.
Shifting markup. Class names and layout change frequently, so brittle selectors break.
Access gating. Amazon periodically limits how many reviews it shows without a session.

Note also that Amazon's official Product Advertising API no longer returns review text, so scraping the public pages is effectively the only way to collect full reviews — which is exactly why doing it carefully matters.

What You Need

Residential proxies to rotate real IPs so no single address gets rate-limited or banned. This is non-negotiable for Amazon at any scale.
Realistic request headers — a genuine User-Agent, Accept-Language, and related fields.
Delays and backoff between requests to look human.
Python with requests and beautifulsoup4.

Scraping Reviews With Python

Amazon marks review elements with data-hook attributes, which tend to be more stable than CSS class names. Here is a minimal scraper that fetches a reviews page through a SpyderProxy residential proxy and extracts each review — treat the selectors as a starting point and verify them against the live page:

import requests
from bs4 import BeautifulSoup

proxy = "http://USERNAME:[email protected]:12321"
proxies = {"http": proxy, "https": proxy}
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
    "Accept-Language": "en-US,en;q=0.9",
}

def scrape_reviews(asin, page=1):
    url = f"https://www.amazon.com/product-reviews/{asin}/?pageNumber={page}"
    r = requests.get(url, headers=headers, proxies=proxies, timeout=25)
    soup = BeautifulSoup(r.text, "html.parser")
    reviews = []
    for el in soup.select('div[data-hook="review"]'):
        def txt(sel):
            node = el.select_one(sel)
            return node.get_text(strip=True) if node else None
        reviews.append({
            "title": txt('[data-hook="review-title"]'),
            "rating": txt('[data-hook="review-star-rating"]'),
            "body": txt('[data-hook="review-body"]'),
            "author": txt('.a-profile-name'),
            "date": txt('[data-hook="review-date"]'),
            "verified": bool(el.select_one('[data-hook="avp-badge"]')),
        })
    return reviews

print(scrape_reviews("B08N5WRWNW", page=1))

If the response contains a "Robot Check" page instead of reviews, the request was flagged — rotate to a fresh residential IP and slow down. For a broader look at avoiding blocks, see how to scrape Amazon without getting blocked.

Paging Through All Reviews

Loop over pages until Amazon stops returning review elements, rotating IPs and adding a delay each time:

import time

def all_reviews(asin, max_pages=10):
    out = []
    for page in range(1, max_pages + 1):
        batch = scrape_reviews(asin, page)
        if not batch:
            break
        out.extend(batch)
        time.sleep(2)   # be polite; rotate the proxy IP per request
    return out

With rotating residential proxies each request exits from a different IP automatically, so you can page through many products without a single address getting flagged. Save the results to JSON to keep the nested structure, or flatten to CSV for analysis — see JSON vs CSV.

Cleaning and Using the Data

Raw review text needs light cleaning before analysis: normalize the star rating (Amazon renders it as text like "5.0 out of 5 stars"), parse dates into a standard format, and strip whitespace. From there the data feeds product and price monitoring, competitor research, and sentiment models — the same pipeline used for building machine learning datasets.

Legal and Ethical Notes

Reviews are publicly visible, but a few rules apply. Amazon's terms of service restrict automated access, so scraping can breach the ToS even when the data is public — understand the difference between "public" and "permitted." Reviews also contain personal data (reviewer names), so handle them under applicable privacy laws, collect only what you need, and use the data for analysis rather than wholesale republication. When in doubt, consult the terms and your own legal counsel.

Frequently Asked Questions

Can you scrape Amazon reviews with Python?

Yes. You fetch the product's reviews page (built from its ASIN) and parse the review elements with a library like BeautifulSoup, extracting the title, rating, body, author, and date. Because Amazon aggressively blocks scrapers, you must route requests through rotating residential proxies and use realistic headers and delays.

Why do I need proxies to scrape Amazon?

Amazon blocks datacenter IP ranges almost immediately and rate-limits or bans any single IP that makes too many requests. Rotating residential proxies spread requests across many real home IPs so your scraper looks like ordinary shoppers, which is the only reliable way to collect reviews at scale.

Does the Amazon API give review text?

No. Amazon's official Product Advertising API no longer returns review content, so scraping the public review pages is effectively the only way to collect full review text. That makes careful, block-resistant scraping the practical approach.

Is scraping Amazon reviews legal?

The reviews are public, but Amazon's terms of service restrict automated access, and reviews contain personal data subject to privacy laws. Public does not automatically mean permitted. Collect only what you need, use it for analysis rather than republication, and review the applicable terms and laws for your use case.

Why does my scraper get a "Robot Check" page?

That is Amazon flagging the request as automated — usually from a datacenter IP, too many requests from one address, or unrealistic headers. Switch to a fresh residential IP, add delays, and send a genuine User-Agent and Accept-Language header.

Conclusion

Scraping Amazon reviews is less about clever parsing and more about not getting blocked. Fetch the ASIN's reviews page, pull the fields from the data-hook elements, page through the results, and — above all — route everything through rotating residential proxies so Amazon sees ordinary shoppers instead of one hammering IP.

Collect Amazon data without the blocks: SpyderProxy residential proxies from $2.75/GB — ethically sourced, rotating, across 195+ countries.