How to Scrape Yelp Data with Python in 2026

Yelp is one of the largest sources of information about restaurants, shops, contractors, salons, and other local businesses. A single listing can contain a business name, address, phone number, opening hours, categories, rating, review count, price range, photos, and customer feedback. If you need to compare many listings at once, collecting this information manually quickly becomes impractical.

In this guide, you’ll learn how to scrape Yelp with Python, Requests, and BeautifulSoup. We’ll cover individual listings, reviews, the search results page, CSV and Excel exports, common errors, proxies, and responsible scraping practices.

Keep in mind that Yelp changes its page structure regularly, so selectors shown here are starting points that may need to be updated before you run them.

Step-by-Step guide to scraping Yelp with Python

Before you start, install the libraries used in this tutorial. We’ll use Requests for HTTP requests, BeautifulSoup for parsing HTML, pandas for saving structured results, and openpyxl for Excel output.

pip install requests beautifulsoup4 pandas openpyxl

A browser’s developer tools are also useful. Open a Yelp page in Chrome or Firefox, right-click the element you want to collect, choose Inspect, and look at the surrounding HTML.

When possible, favor meaningful attributes such as data-testid, aria-label, or stable link patterns over generated class names like css-abc123, because those classes can change without warning.

Find and inspect a Yelp business listing page

Start with one public listing page. A typical business page includes the name, rating, number of reviews, address, phone number, categories, opening hours, and other details.

Before writing selectors, decide exactly which columns you want in the final dataset. A small schema might contain name, category, rating, review_count, address, phone, and url.

A more detailed project could also include price range, hours, geographic coordinates, or a short list of amenities. Defining the schema first prevents the parser from turning into a collection of unrelated selectors.

Another consideration is decoupling the discovery phase from the detailed information-gathering phase. The discovery phase identifies the URLs for each listing and its basic metadata; the detail phase visits each listing page only when you need more detail. This dual-phased approach makes scraping Yelp easier to debug because it avoids unnecessary requests.

If your goal is to scrape Yelp data for competitive research, it is better to begin with one listing and confirm that your parser works before adding pagination or hundreds of URLs. This makes debugging much easier.

Open the page source and look for a few useful patterns. Yelp often exposes business links under /biz/, ratings through accessible labels, and some structured information inside JSON or JSON-LD script tags. Structured data is especially useful because it's less fragile than CSS classes.

For example, the HTML may contain a script resembling this:

<script type="application/ld+json">
{
  "@type": "Restaurant",
  "name": "Example Restaurant",
  "aggregateRating": {
    "ratingValue": "4.5",
    "reviewCount": "125"
  }
}
</script>

Not every page will look the same. The important part is to inspect the live page before choosing selectors.

Send an HTTP request and parse HTML

Now create a reusable function that downloads a page and turns the response into a BeautifulSoup object.

import requests
from bs4 import BeautifulSoup
 
HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/131.0.0.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
}
 
def get_soup(url):
    response = requests.get(
        url,
        headers=HEADERS,
        timeout=20
    )
    response.raise_for_status()
    return BeautifulSoup(response.text, "html.parser")

Replace the URL below with the listing you are permitted to access:

url = "https://www.yelp.com/biz/example-business"
soup = get_soup(url)
 
print(soup.title.get_text(strip=True) if soup.title else "No title found")

requests.get() sends the HTTP request, while raise_for_status() produces an exception for unsuccessful responses. BeautifulSoup then parses the returned HTML so you can search it with CSS selectors and tag methods.

If the page relies on JavaScript for a field you need, the initial HTML may not contain it. In that situation, inspect the source first. If the information appears only after rendering, you may need a browser automation tool such as Playwright or Selenium instead of Requests alone.

Extract business information

There are two practical ways to extract data from a business page: read structured JSON-LD where available, then use HTML selectors as a fallback.

The function below looks through JSON-LD blocks and returns fields commonly associated with a local business.

import json
 
def extract_json_ld(soup):
    records = []
 
    for script in soup.select('script[type="application/ld+json"]'):
        if not script.string:
            continue
 
        try:
            payload = json.loads(script.string)
        except json.JSONDecodeError:
            continue
 
        if isinstance(payload, list):
            records.extend(payload)
        elif isinstance(payload, dict):
            records.append(payload)
 
    for item in records:
        if not isinstance(item, dict):
            continue
 
        item_type = item.get("@type", "")
        if isinstance(item_type, list):
            item_type = " ".join(item_type)
 
        if any(word in str(item_type) for word in [
            "LocalBusiness", "Restaurant", "Store",
            "FoodEstablishment", "ProfessionalService"
        ]):
            rating = item.get("aggregateRating", {}) or {}
            address = item.get("address", {}) or {}
 
            return {
                "name": item.get("name"),
                "rating": rating.get("ratingValue"),
                "review_count": rating.get("reviewCount"),
                "phone": item.get("telephone"),
                "price_range": item.get("priceRange"),
                "street": address.get("streetAddress"),
                "city": address.get("addressLocality"),
                "region": address.get("addressRegion"),
                "postal_code": address.get("postalCode"),
            }
    return {}

Run it with:

business = extract_json_ld(soup)
print(business)

This is a useful first layer, although structured markup may not include every field. For categories, additional attributes, or services offered, you may need HTML selectors.

A simple fallback helper keeps absent elements from crashing the script:

def text_or_none(element):
    return element.get_text(" ", strip=True) if element else None
 
name = text_or_none(soup.select_one("h1"))
rating_node = soup.select_one('[aria-label*="star rating"]')
 
rating = rating_node.get("aria-label") if rating_node else None

When you scrape Yelp, avoid hard-coding long generated class names whenever a structural selector will work. If a field suddenly becomes None for every listing, inspect the page again before assuming the request failed.

The exact business data you collect depends on your project:

• Lead-generation datasets may focus on company names, categories, phone numbers, and locations

• Market research may need ratings, review counts, price level, and restaurant data

• Directory-monitoring projects may care more about hours, closure status, or services offered

Scrape reviews

Reviews are more difficult because the layout can change and some content may be loaded dynamically. Before you scrape Yelp reviews, inspect whether the review text which you need is actually present in the returned HTML.

A defensive parser can look for review containers first and then search within each one:

def extract_reviews(soup):
    reviews = []
 
    containers = soup.select(
        '[data-testid="review"], '
        'li[class*="review"]'
    )
 
    for container in containers:
        text_node = container.select_one(
            'p[class*="comment"], '
            '[data-testid="review-comment"]'
        )
        rating_node = container.select_one(
            '[aria-label*="star rating"]'
        )
        date_node = container.select_one(
            'span[class*="date"], '
            'time'
        )
 
        review = {
            "text": text_or_none(text_node),
            "rating": (
                rating_node.get("aria-label")
                if rating_node else None
            ),
            "date": text_or_none(date_node),
        }
 
        if review["text"]:
            reviews.append(review)
 
    return reviews

This produces a list of dictionaries containing review data. You can add author names or other fields if they are required and appropriate for your use case.

There is an important limitation here: a plain Requests response may not always include the same review content you see in a fully rendered browser. If the returned HTML is incomplete, first verify that you received a normal page rather than a challenge or error page. Then check whether JavaScript rendering is required.

Also, bear in mind that customer reviews constitute user-generated content. Gather information only when you require it, refrain from creating any unwanted profiles, and familiarize yourself with the terms and conditions of the site along with relevant privacy and copyright laws.

Scrape Yelp search results

A search results page is useful when you do not already have a list of business URLs. For example, you might search for plumbers in Chicago, coffee shops in Boston, or restaurants in Austin and collect the visible listing information.

Yelp search URLs commonly include a search term and location. A simplified example looks like this:

https://www.yelp.com/search?find_desc=coffee&find_loc=Boston%2C+MA

You can build the URL safely with Requests:

from urllib.parse import urlencode
 
params = {
    "find_desc": "coffee",
    "find_loc": "Boston, MA",
}
 
search_url = "https://www.yelp.com/search?" + urlencode(params)
search_soup = get_soup(search_url)

Now identify organic result cards. Attributes used for testing are often more dependable than generated CSS classes:

from urllib.parse import urljoin
 
def extract_search_results(soup):
    results = []
    seen = set()
 
    cards = soup.select('[data-testid="serp-ia-card"]')
 
    for card in cards:
        link = card.select_one('a[href^="/biz/"]')
        if not link:
            continue
 
        href = link.get("href", "").split("?")[0]
        if not href or href in seen:
            continue
 
        seen.add(href)
 
        rating_node = card.select_one('[aria-label*="star rating"]')
        text = card.get_text(" ", strip=True)
 
        results.append({
            "name": link.get_text(" ", strip=True),
            "url": urljoin("https://www.yelp.com", href),
            "rating": (
                rating_node.get("aria-label")
                if rating_node else None
            ),
            "card_text": text,
        })
 
    return results

Run:

results = extract_search_results(search_soup)
 
for result in results:
    print(result)

This gives you the following information:

• Business name

• URL

• rating (when available)

• visible card text

From there, you can refine selectors for categories, neighborhood, review count, or price range.

If you need several pages of search results, inspect the pagination pattern rather than guessing. Yelp layouts and URL parameters can change. Add a conservative delay between page requests and stop when a page returns no new businesses.

The pagination algorithm must also ensure the de-duplication of items. The same listing may come up more than once due to different reasons such as promoted listings, changed order of results, or overlapping search queries.

A canonical URL such as /biz/ or any other unique identifier can be used for de-duplication purposes as opposed to using the name alone since two different companies can have the same name.

For larger files, it is wise to save the progress after each page completion. Should your script halt midway through, you will not have to download previous pages since you can simply pick up from where you left off.

import random
import time
 
time.sleep(random.uniform(3, 6))

A Yelp data scraper should favor reliability over speed. Sending a large burst of requests creates more errors, puts unnecessary load on the site, and increases the chance that access will be restricted.

Save data to CSV or Excel

Once you have a list of dictionaries, pandas makes export straightforward.

For CSV:

import pandas as pd
 
df = pd.DataFrame(results)
df.to_csv("yelp_results.csv", index=False, encoding="utf-8-sig")

For Excel:

df.to_excel("yelp_results.xlsx", index=False)

You can do the same with business details or review data:

reviews_df = pd.DataFrame(extract_reviews(soup))
reviews_df.to_csv("yelp_reviews.csv", index=False)

CSV is usually the simplest format for big datasets, while Excel is convenient for manual review and sharing with non-technical teammates.

Before saving, consider normalizing values. Convert rating strings to numbers, strip “reviews” from review counts, standardize phone numbers, and separate city, region, and postal code into dedicated columns. Clean structured data is much easier to analyze later.

It is also worth adding operational columns such as source_url, collected_at, and status. These fields make troubleshooting much easier when records come from different runs. If a value is missing, you can distinguish between “the listing does not publish this field” and “the parser failed on this run.” For recurring research, timestamps also let you compare snapshots and identify changes without mindlessly replacing older records.

Do not assume that the data is clean at the outset. There may be variations in how category names, addresses, business hours, and pricing information have been captured in the listings. Normalization should happen afterward rather than having the parser handle all the cleansing. It’s better for testing, too.

Full code

Below is a compact example that combines the main steps. It requests a business page, extracts structured business information and reviews, then saves the results.

import json
import time
import random
import requests
import pandas as pd
from bs4 import BeautifulSoup
 
HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
        "AppleWebKit/537.36 (KHTML, like Gecko) "
        "Chrome/131.0.0.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
}
 
def get_soup(url, proxies=None):
    response = requests.get(
        url,
        headers=HEADERS,
        proxies=proxies,
        timeout=20
    )
    response.raise_for_status()
    return BeautifulSoup(response.text, "html.parser")
 
def text_or_none(element):
    return element.get_text(" ", strip=True) if element else None
 
def extract_json_ld(soup):
    records = []
 
    for script in soup.select('script[type="application/ld+json"]'):
        if not script.string:
            continue
        try:
            payload = json.loads(script.string)
        except json.JSONDecodeError:
            continue
 
        if isinstance(payload, list):
            records.extend(payload)
        elif isinstance(payload, dict):
            records.append(payload)
 
    for item in records:
        if not isinstance(item, dict):
            continue
 
        item_type = str(item.get("@type", ""))
        if any(x in item_type for x in [
            "LocalBusiness", "Restaurant",
            "Store", "FoodEstablishment"
        ]):
            rating = item.get("aggregateRating", {}) or {}
            address = item.get("address", {}) or {}
 
            return {
                "name": item.get("name"),
                "rating": rating.get("ratingValue"),
                "review_count": rating.get("reviewCount"),
                "phone": item.get("telephone"),
                "price_range": item.get("priceRange"),
                "street": address.get("streetAddress"),
                "city": address.get("addressLocality"),
                "region": address.get("addressRegion"),
                "postal_code": address.get("postalCode"),
            }
 
    return {}
 
def extract_reviews(soup):
    data = []
 
    for container in soup.select(
        '[data-testid="review"], li[class*="review"]'
    ):
        text_node = container.select_one(
            'p[class*="comment"], [data-testid="review-comment"]'
        )
        rating_node = container.select_one(
            '[aria-label*="star rating"]'
        )
        date_node = container.select_one(
            'span[class*="date"], time'
        )
 
        text = text_or_none(text_node)
        if not text:
            continue
 
        data.append({
            "text": text,
            "rating": (
                rating_node.get("aria-label")
                if rating_node else None
            ),
            "date": text_or_none(date_node),
        })
 
    return data
 
url = "https://www.yelp.com/biz/example-business"
soup = get_soup(url)
business = extract_json_ld(soup)
reviews = extract_reviews(soup)
 
pd.DataFrame([business]).to_csv(
    "business.csv", index=False, encoding="utf-8-sig"
)
pd.DataFrame(reviews).to_csv(
    "reviews.csv", index=False, encoding="utf-8-sig"
)
 
time.sleep(random.uniform(3, 6))
 
print("Business:", business)
print(f"Reviews collected: {len(reviews)}")

This is a working prototype and should not be used as a final implementation for scraping Yelp. For each run, check live HTML, verify one URL, ensure fields are returned correctly, and implement logging to catch any changes to the layout.

It's worth noting that Python and various libraries are not limited to Yelp. Python is a powerful tool that can be used for other websites, such as Google Maps, Amazon, and many others.

Common errors and how to fix them

Even a small script can fail because of access restrictions, page changes, missing fields, or network issues. The most useful debugging habit is to inspect both the HTTP status code and the first part of the response body before changing your parser.

The approach for error handling must be built around the requests layer and not use the try/except blocks everywhere. Timeouts on networks, temporary issues with the server, failure to parse the page, and missing fields are separate issues and should be logged separately.

A production-friendly script can have a logging mechanism that logs URL, status codes, type of exception, and number of retries in a file.

Try limiting the number of attempts you make. It may make sense to retry a transient 502 or a timeout, after waiting for some time, but retrying a 403 error repeatedly for many seconds is unlikely to help at all.

403 forbidden errors

The 403 Forbidden response indicates that the server has received your request but will not fulfill the request for the web page. Scraping Yelp involves the chances of encountering 403 forbidden responses due to many automated requests, strange traffic behavior, no HTTP headers, and bad IP reputation.

Start by testing whether the URL is working fine in your browser and the speed at which your code sends the requests. Use an appropriate user agent, implement a timeout, and reduce the speed of requests. Stop requesting the page again and again when receiving 403 responses.

You can also add simple logging:

response = requests.get(url, headers=HEADERS, timeout=20)
print(response.status_code)
print(response.url)
print(response.text[:500])

If you receive an access-denied or challenge page, your parser is not the problem. Continuing to change BeautifulSoup selectors will not fix a blocked HTTP response.

Handling empty or incomplete responses

Sometimes the response is 200 OK, but the expected fields are missing. This usually points to one of four causes: the selector changed, the content is rendered with JavaScript, the page variant is different, or the response isn't the normal page you expected.

Add checks before extracting values:

if len(response.text) < 1000:
    print("Unexpectedly short response")

Save the HTML while debugging:

with open("debug.html", "w", encoding="utf-8") as file:
    file.write(response.text)

Open debug.html in a browser and search for the business name or review text. If the value exists, update your selectors. If it does not, investigate rendering or access issues.

Also design your parser to tolerate missing values. Real business data is inconsistent: one listing may have a phone number and price range while another does not. Returning None is better than terminating the entire job.

How to avoid getting blocked

The large amount of scraping will create more traffic compared to regular browsing and will lead to rate limit or even anti-bot measures being activated. The objective should not be overwhelming or breaking down the website, but decreasing the request amount, distributing them appropriately, and using the API if available for the task.

Good request policy implies predictability. Use requests.Session() if possible, cache pages that were scraped before, put pauses between the requests, and have minimal concurrency.

In case you need the fresh dataset every day, only update the records that could have been updated instead of starting from scratch. This approach ensures better stability without relying on anti-detection technologies more and more.

Monitoring is equally important. Keep track of the percentage of successful requests, the average response size, the number of records that have nothing in them, and the parsing error rate. The significant reduction in extracted fields often indicates the design change before it is seen in the final dataset.

Rotating IPs and proxies

In proxy use, the request goes via an additional IP address. In web scraping, proxy can be used for distributing allowed requests not to send all requests from the same computer.

For example, Requests accepts a proxy configuration like this:

PROXY_URL = "http://username:password@host:port"
 
proxies = {
    "http": PROXY_URL,
    "https": PROXY_URL,
}
 
soup = get_soup(url, proxies=proxies)

For location-sensitive research involving local businesses, residential proxies can also provide IPs associated with the geographic market you are studying. That can be useful when pages differ by region.

However, rotating IPs does not give permission to access content that a site prohibits. If a page repeatedly rejects automated access, stop and review the site rules or switch to an approved data source.

User agent rotation and delays

A User-Agent tells the server what browser and operating system a request claims to use. Some scrapers rotate user agents, but randomizing this header on every request can look less realistic rather than more realistic.

A better approach is to use a small set of valid browser headers and pair them with conservative delays. For example:

USER_AGENTS = [
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
    "AppleWebKit/537.36 (KHTML, like Gecko) "
    "Chrome/131.0.0.0 Safari/537.36",
 
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
    "AppleWebKit/605.1.15 (KHTML, like Gecko) "
    "Version/18.1 Safari/605.1.15",
]

Then choose one for a session and add pauses between page requests:

import random
import time
 
headers = {"User-Agent": random.choice(USER_AGENTS)}
time.sleep(random.uniform(3, 7))

The delay reduces load and makes retry behavior easier to control. For longer jobs, add exponential backoff so failed requests wait progressively longer instead of immediately retrying.

CAPTCHA handling techniques

A CAPTCHA is a strong sign that the site wants the current traffic to stop or prove that it comes from a human. Do not design your script to endlessly retry through a CAPTCHA.

However, identify the challenge page, stop the scraping process, reduce the number of requests per second, and determine if there is an official API or a licensed provider that could provide the same data from Yelp. You can also use the manual approach as part of a supervised workflow.

A simple detection check can look for common challenge text:

html_lower = response.text.lower()
 
if "captcha" in html_lower or "verify you are human" in html_lower:
    raise RuntimeError("Challenge page detected; stopping the job.")

This approach is safer and easier to maintain than building an escalating loop that keeps attempting blocked requests.

Real-world use cases for scraped Yelp data

The value of web scraping Yelp depends less on how many pages you collect and more on what question you are trying to answer. Define your fields before writing the scraper so you do not gather unnecessary information.

Lead generation and local business research

Public listing information can help teams research a market and recognize potential business prospects. For example, you might collect business names, categories, neighborhoods, public phone numbers, ratings, and review counts for a specific city.

That business data can then be filtered by category or location. A supplier serving restaurants could compare districts with many highly rated independent venues. A software company could research local businesses in a niche and enrich an existing prospect list with publicly visible listing details.

The trick is to keep the dataset relevant. Avoid collecting personal information just because it is technically visible, and do not assume that scraped contact details automatically permit unsolicited outreach. Marketing and privacy rules still apply.

Sentiment analysis and reviews monitoring

Review analysis can uncover the underlying theme that the numerical rating fails to capture. For instance, an eatery that gets a 4.2 rating can attract consistent positive remarks on its food quality while also facing negative reviews on its delivery time.

After you extract data, basic natural-language processing can group comments into positive, negative, and neutral sentiment or track keywords over time. Competitor research can also reveal which features customers value most.

If you scrape Yelp reviews for your analysis, you should maintain the scope of your work accordingly. The use of full-text reviews may present copyright and technical issues, and you should not reproduce large sets of reviews. However, you can still do much analysis based on counts and themes.

Is scraping Yelp legal?

Web scraping is not governed by one universal rule. Whether a project is lawful can depend on the jurisdiction, the type of information collected, how access is obtained, what the data is used for, and contractual terms.

This section is general information, not legal advice. For a commercial or high-volume project, legal review is worthwhile before deployment.

Yelp’s terms of service

Yelp’s Terms of Service restrict the use of robots, spiders, search/data retrieval applications, and other automated means to access, retrieve, copy, scrape, or index the service or its content unless Yelp expressly permits it. Yelp’s business terms also contain restrictions on automated scraping and data extraction.

That means a technically possible script is not automatically permitted. Before you scrape Yelp, read the terms that apply to your account, location, and planned use, and check any permissions or machine-readable rules Yelp provides.

Yelp also offers an official developer API. The current Yelp documentation places the familiar Fusion endpoints within the Yelp Places API, including business search and business details; review access is limited to excerpts and depends on the applicable plan.

If the API provides the fields you need, it is generally the cleaner option because you get structured responses lacking maintaining HTML selectors.

Ethical considerations for developers

Good scraping begins with minimization. Only get the fields you need, use a cautious request pace, cache requests rather than redownload pages repeatedly, and quit when the site indicates automated browsing should be stopped.

Protect your stored information as well. Text reviews, profile information, and contact information, while innocuous in isolation, could cause privacy or legal concerns when assembled into large datasets. Set a time limit on how long data is retained, restrict access to the information, and delete any unnecessary fields.

Lastly, keep the technical capabilities of scraping separate from permission issues. Proxies, browser automation, and parsing software are completely neutral technologies that should never serve as a reason not to comply with website guidelines and restrictions.

For those learning to scrape data from Yelp or building a Yelp scraper, the most effective process is as follows: start with small-scale testing on authorized samples, write defenses against unexpected changes in structure, collect the minimal amount of data necessary, and use the Yelp API if it meets your needs.

FAQ