Web Scraping Google Search Results
Building a Google scraper to download SERP data is generally a bad idea, as Google employs many measures to block automated bots and Google search scraping attempts.
However, this hasn’t prevented many companies from somehow achieving this and maintaining their own databases of Google search queries and results - making this data available through APIs.
Using Serp API
One company that offers an API you can use as a Google search scraper is Serp API, and the rest of this page will demonstrate how to use their Google Search API Endpoint to extract data from organic results, paid ads, local places & other components commonly found on Google search engine result pages.
You can sign up for a free Serp API account to scrape data right away with 100 free searches per month. And once you have a Serp API key, you can use that key in conjunction with our Google data scraper on this page to export Google search results to Excel in just a few clicks:
Google SERP Data
Serp API will return a large and nested JSON object, but our web scraping tools will collect data from the Google result pages into downloadable CSV files like the one shown below:
Just click to download button to save the organic search results data as a CSV file you can analyze in Excel or any spreadsheet. You will get a parsed collection for every type of result that Serp API supports, shown above is the one for “Organic Results” (which will always show), but depending on the query you may also see other downloadable collections like for ads, news, events, images, etc…
The number of results you get back may vary, but we usually see Serp API return up to 100 organic results just as the regular Google website does.
You can see the full Google Search API Documentation on Serp API’s website to understand exactly the types of data you may be able to get back depending on your query from several different Google services.
Location Querying
You can also change the location from which you query Google’s search result pages from their supported locations. They seem to support major U.S. cities but this list can change at any time.
Legal Concerns
Is it legal to scrape Google? We can’t say (since we don’t), but Serp API offers a “Legal US Shield” in which they take responsibility for the scraping and parsing of data included in their “Production” tier plans and above. As we do not actually scrape Google search data and instead just offer a wrapper on top of Serp API, we cannot comment on the legality of scraping Google’s public data.
However, since Serp API has been around since 2016, we think it’s safe to say the company has their legal strategy well thought out and will likely not be going anywhere as they are the trusted data source for several high-profile Google search term databases like Ahrefs!