How to Scrape ALL Google Maps Businesses in a City
Did you know that you can access the raw data behind Google Maps via an official API? It’s called the Places API and you can use up $200 per month in credits from Google FOR FREE.
If you’ve been using unofficial “screen scraping” to download Google Maps data, you may want to consider using the official Google Places API, as it’s an “official integration,” meaning it will not break suddenly and you don’t risk Google banning you for violating their Terms of Service.
The rest of this article will discuss how to use the Stevesie Data Platform to collect a large list of businesses (including details like website, phone number & revent reviews) for a given city, like New York. Stevesie Data is a paid platform and the rest of this article assumes you have the plus plan for running workflows.
If you do not want to use a paid platform, then you can refer to the links above and directly access the Google Places API at your own time, expense and effort. Disclaimer: I, the author of this article, happen to own the Stevesie Data Platform.
1. Get the Target City Coordinates
The first step is to get the latitude and longitude of the city you want to collect the business from. You can use the Place Search Endpoint from Google Maps to do this, as “cities” are also technically “places” according to the Google API.
The Stevesie Google Maps Place Search Integration will let you easily get the coordinates, just supply your API key and enter your target city under name, like “New York,” into the Query input, then hit execute:
The results will not only give us the center coordinates, but a “bounding box” (well… technically a rectangle) of coordinates we can use:
2. Build a Latitude-Longitude Search Grid
While it would be great if we could give Google the “bounding box” coordinates and have it return ALL the businesses within that box, unfortunately their API is not so nice and instead relies on us providing a single latitude and longitude to run a search on, and then the API will give us back only a mere 60 results per coordinate.
So naturally, the way around this per-coordinate limitation is to just use more and more coordinates, essentially running a lot of “micro” searches (getting up to 60 results back), but at slightly different locations to “sweep” the entire city.
You’re free to generate this list of coordinates as you’d like, but here we’ll suggest using a basic “grid” approach, where we split the “bounding box” of the city into smaller segments, like a grid, then run the search on each of these coordinates in the grid.
Below is a simple JavaScript snippet you can run in your browser to generate a list of “grid” coordinates based on the bounding box returned from step 1 above. Just right click somewhere on your browser to open up the developer console, then paste the following script into the console:
const NORTHEAST_LAT = PASTE_ME; // paste in the value for results[0].geometry.viewport.northeast.lat;
const NORTHEAST_LNG = PASTE_ME; // paste in the value for results[0].geometry.viewport.northeast.lng;
const SOUTHWEST_LAT = PASTE_ME; // paste in the value for results[0].geometry.viewport.southwest.lat;
const SOUTHWEST_LNG = PASTE_ME; // paste in the value for results[0].geometry.viewport.northeast.lng;
const DESIRED_GRID_LENGTH = 3; // start with 3 to get a 3x3 grid and so on...
let output = '';
let epsilon = 0.0000001; // because JavaScript and math don't get along
let intermediate_grid_length = DESIRED_GRID_LENGTH - 1;
let lat_step_size = ( NORTHEAST_LAT - SOUTHWEST_LAT ) / intermediate_grid_length;
let lng_step_size = ( NORTHEAST_LNG - SOUTHWEST_LNG ) / intermediate_grid_length;
for (let lat = SOUTHWEST_LAT; lat <= NORTHEAST_LAT + epsilon; lat += lat_step_size) {
for (let lng = SOUTHWEST_LNG; lng <= NORTHEAST_LNG + epsilon; lng += lng_step_size) {
output += lat + ',' + lng + '\n';
}
}
console.log(output);
Before hitting “enter” to run this, please provide the numbers for PASTE_ME
as is explained above. By default, the script will return 9 coordinates (think of a 3x3 grid), but if you want to get more results, you can increase the value of DESIRED_GRID_LENGTH
to grow the number of coordinates within the grid (e.g. if you set it to 10, you’ll get a 10x10 grid with 100 total coordinates).
After setting the const
values above, just hit enter inside the code block and you’ll get the following output (depending on the inputs you provided):
Now you can copy the output and you’ll have a “grid” of search coordinates you can use to search Google Maps and get back businesses matching your query:
40.4773991,-74.25908989999999
40.4773991,-73.979681
40.4773991,-73.7002721
40.6974881,-74.25908989999999
40.6974881,-73.979681
40.6974881,-73.7002721
40.9175771,-74.25908989999999
40.9175771,-73.979681
40.9175771,-73.7002721
3. Run a Search for Each Coordinate & Combine Results
Now let’s do a quick test using our center coordinate from the grid (the middle value) of 40.6974881,-73.979681
and get some businesses back from Google Maps, so using the Stevesie Place Search Integration, we’ll simply provide the coordinate pair under the “Location” input:
Now unfortunately, we MUST provide either a text query of a business to search for (we can be creative) AND/OR a “type” of business to find (there’s no easy way to get ALL the businesses). You can browse Google for the supported business types and pick the one(s) you need. Here we picked bar
to use as an example (it’s been a long pandemic), to get all the “bars” in New York:
Hit execute and we’ll get back up to 20 results (per page) and we’ll see basic business details, like the address, coordinates, name, user rating, number of user ratings & price level:
Now to get the next page of results, we can look for the next_page_token
in the root collection and run this again, however the Places API only returns about 3 pages of results total (60 results), so this is not the best way to get bulk data back. For this we want to import the Google Maps Places Search - Pagination Formula.
After importing the formula, you can paste in your list of coordinates to run searches for in the “Location” input, as well as your API key and the “Type” parameter (you can enter multiple types as well, but it will then double, triple etc… the number of API requests used based on the number of types provided):
Once entered, you can then execute the workflow and it will query the Google Places API on your behalf and collect & combine all the data:
Once the workflow is done, you’ll be able to download the Google_Maps_Places.csv file, which in this case collected back 280 results for the 9 search coordinates we entered:
If this CSV file is all you need (with the basic business info, like if you’re making a map), then you’re all done! However, if you need more information on each business like the phone number, website (perhaps to scrape emails for) and reviews, then the next step will show you how to “augment” this data.
Getting MORE Results!
Realistically, we know that there are way more than 280 bars in New York City… more like at least a few thousand. To get this full list, we simply need to provide more search coordinates. We ran this example with DESIRED_GRID_LENGTH
set to 3
in step 2 above, but if we set this value to 4
, then we would get a 4x4 grid of coordinates with 16 total (compared to 9 as we used in this example) and get up to 78% more results if we used a value of 4
… theoretically up to about 500 results.
If we set DESIRED_GRID_LENGTH
to 5
, then we would have a 5x5 grid or 25 coordinates and we could hypothetically get back 2.78x the results as our initial 280, or 778 results.
If DESIRED_GRID_LENGTH
were set to 6
, then we would run 36 searches and possibly get up to 1,120 bars in NYC.
We could keep going obviously, but you just want to keep an eye on how many Google Places API credits you’re using on the search step. Make sure you don’t use too many, otherwise you may get charged if you exceed the free threshold! Also, if you use way too many, then your searches may only be a few feet apart which is generally a waste of resources and time, so the trick is to find your “sweet spot” to get the most results in your particular city while not using an excessive number of searches (or Google WILL bill you).
4. Look Up Individual Business Details
After opening up your results from the Places Search workflow, you want to look for a column with place_id
in it, which will allow us to look up details about our results. Find and copy this list in the output:
Now we want to look up every individual place and combine the results together into more detailed CSV files. To do this, we’ll import the Google Maps Place Details - Multiple Place IDs Formula - just click “import” and then paste in the list of place_ids you copied (leave out the header):
A keen eye will notice that there are only 269 Items in the list. This is because the Stevesie Platform removed duplicates (overlaps from combining multiple searches), to avoid wasting API credits with Google.
We can now run the workflow, but PLEASE pay attention to how many place detail lookups you’re about to perform. Check the current Google Places API Pricing and make sure you won’t exceed the free trial, e.g. right now 12,000 lookups will result in exceeding the $200 free credit, so keep an eye on this to avoid any surprise bills from Google:
Since we’re only doing 269 lookups in this workflow, we’re not going to worry about this and just execute the workflow:
Let the workflow run, it will take a few seconds PER place_id
provided, so just be patient. When it’s done, you’ll be able to download the business details from the Google_Maps_Place_Details.csv file, where each row will correspond to a place and contain additional information like phone number, website, etc…
For other data types returned, with more than one instance per business, like reviews, you’ll see other collections you can download which will contain a reference to the parent place of business on the right hand side of the file.
BONUS - Scrape Emails from Local Businesses
Once you have a list of websites from your business details, there are a few services that will return a contact email based on the website provided. If interested, simply open up the Google Maps Places Details spreadsheet and find the column corresponding to the businesses’ websites and copy this list:
You can then choose from either UpLead or Hunter.io (or both) to use to look up the emails. Note, these are both paid services. Then simply import either of these workflows & paste in the list of websites into the input asking for the domain:
Then provide your API key and run the workflow and you’ll get a spreadsheet with websites and the corresponding emails if found. Remember to always be respectful if doing cold email outreach and follow all applicable rules, laws & regulations before sending anything!