Scraping the GrubHub API
GrubHub offers a wealth of information regarding local restaurants, menu items, prices & reviews - displaying all of this publicly on its mobile app & website! However, GrubHub does not offer an official API to access this data from, making it tricky to collect this public data if you’re looking to download a GrubHub dataset.
However, there is a 100% legal way we can scrape the GrubHub API without violating its Terms of Service using a little-known technique involving HAR files. We’ll walk through the process below; feel free to follow along or check out the video tutorial on this page!
1. Browse GrubHub Restaurants
To get started, simply head to the GrubHub Website and run a search for whatever restaurants you’re interested in. You can use whatever filters you’d like as well as you’d normally do on the website.
Once you see the data you want to scrape, right click on the screen and hit “Inspect” to open up developer tools and begin recording your web traffic. Now refresh the page (to get GrubHub to send a new batch of data to your browser) and simply scroll down through all of the results.
2. Export a HAR File
Once you’ve scrolled through the results, go to the “Network” tab under developer tools and click the down arrow labeled “Export HAR…” to download a HAR file containing the raw GrubHub data we recorded in the previous step.
You’ll then want to upload that file to the HAR File Web Scraper so we can extract out the data into downloadable files. Look for the group labeled /restaurants/search as in the screenshot above and click “Parse Group” to extract the data from the file.
3. Download GrubHub Data
You’ll then see downloadable collections on the next screen. Click “Download CSV” on the collection labeled search_result > results to download the restaurant data.
Inside the restaurant CSV file, you’ll find basic fields about each restaurant like name, address, location, reviews, phone number, photo URLs and a few more basic fields.
Scraping Menu Data
Unfortunately, the menu data is not returned with each restaurant in the search results (which makes sense, since the menu isn’t presented until you click in to each restaurant). The HAR scraping approach outlined above will work with individual restaurants though to download their menus, but you’ll need to visit each restaurant individually.
Just repeat the steps as above, but this time open the network tab (to record your traffic) on an individual restaurant’s page and then refresh the page to get it to load while you’re recording. Upload the HAR file to the HAR File Web Scraper and you should see a list of all the menu items for the restaurant.
Legal Concerns
GrubHub most likely does not want you to download its entire database of restaurants & menus, nor wants you re-distributing its content without consent. These are very bad things that have legal consequences.
However, if you were to manually access GrubHub and copy bits and pieces of information that interest you (e.g. through the method outlined above), perhaps for a research project, etc… this is most likely fine - but you should refer to GrubHub’s Terms of Service before doing anything with data you’ve collected.