Scraping the AirDNA API
If you need Airbnb market data to help guide your real estate investing decisions or to better manage your existing Airbnb listings, one popular source for analyzing Airbnb market data is AirDNA, which offers a completely free forever tier to use their service.
While you can browse AirDNA for free inside of their web application, they do not offer a data export option (that we could easily find). There is an official AirDNA API that appears to offer this functionality with endpoints to download Airbnb listings data from, but you must manually contact sales to obtain an API key and it’s likely to be very expensive since they don’t disclose public pricing information.
However, we can still legally access the Airdna Market Data API simply by signing up for a free account and using their web application, which will instead access their API on our behalf. We can then inspect and record these API interactions to see the data that AirDNA sends to our browser as we legally use the service and then later extract the data out from the recording instead of the website.
1. Browse AirDNA
Head to AirDNA and sign up for a free account. Note that you must accept the Terms of Service and agree to not scrape data directly from the service (nor republish it). We’ll instead just be browsing AirDNA in full accordance with their Terms of Service, with the only exception being that we’ll be recording our web traffic in developer tools.
To do this, navigate to a page in AirDNA you’d like to collect the data for and right click on the screen and hit “Inspect” to open up developer tools, which will begin recording your web traffic.
Now simply scroll through the listings or other data you’re interested in. This will force AirDNA to keep querying its API and loading more and more Airbnb listings into your browser that you’re capturing as you record the web traffic.
Note that you will only be able to browse as many listings or other relevant data as allowed in your AirDNA subscription, so you may need to upgrade to a paid version of AirDNA to legally access all the data you need.
2. Export a HAR File
Once you’ve browsed through enough listings, go to the “Network” tab in developer tools and click the down arrow labeled “Export HAR…” to download a HAR file containing a recording of your web traffic.
Upload the HAR file to the HAR File Web Scraper and look for the group(s) that begin with /api/ in the URL and end in the type of data you’re interested in, e.g. listings as pictured above.
3. Download AirDNA Market Data
Click “Parse Group” to extract the data out of the HAR file and look for the relevant collection, e.g. payload › listings as shown below.
You’ll be able to download all of the fields presented on the website, and potentially more fields if you pay for a premium AirDNA account. Some fields of interest for Airbnb listings are:
- Title
- Property ID
- Listing Type (Entire place, private room, shared room)
- Bedroom Count
- Bathroom Count
- Number of People
- Average Rating
- Number of Ratings
- Market ID
- Latitude & Longitude (Not Exact)
- Revenue Estimates
- Days Available Estimates
Legal Concerns
We should mention that AirDNA’s Terms of Service ironically do not permit scraping their own data (even though their entire business is presumably based on scraping data from Airbnb, presumably against the Airbnb Terms of Service).
Scraping a Recording
However, the approach outlined here is scraping, or extracting, the relevant data from a recording of your web traffic (which you legally acquired while using the AirDNA website in full compliance with the Terms of Service) instead of scraping the actual AirDNA website, which would constitute violative scraping as those interactions would be governed under the AirDNA Terms of Use.
Therefore, what you do with a recording of your web traffic after you’re done using the AirDNA cannot be reasonably governed by the AirDNA Terms of Service, which would only apply to the time spent interacting with said service, and not what you do with a recording that was passively acquired.
No Republishing
While the AirDNA Terms cannot reasonably dictate what you privately do with a recording of your web traffic, they can (and should) prohibit you from republishing their data for commercial use as it would violate copyright law.
So while it’s probably fine to use the recorded data for your own internal research, do not republish this data elsewhere or it will be a clear violation of the AirDNA Terms of Service.