Scraping the Zillow API
Can you export data from Zillow to Excel? Yes! And we offer the only 100% legal way to export Zillow data to Excel directly from the Zillow website without being detected or banned using a little-known method using HAR files & our Zillow data exporter.
This will let you extract Zillow listings complete with address & pricing information into Excel for your own market research or outreach campaigns in only a few minutes with our Zillow web scraper.
1. Browse Zillow Listings
Head to the Zillow Website and enter any location you want to scrape the housing data from. You can apply any filters you want (e.g. only show homes for sale, rent, foreclosures, etc…) to configure your Zillow data extraction.
Once the page loads, right click on the screen and hit “Inspect” to open up developer tools to begin recording your web traffic for scraping Zillow data. Refresh the page to reload the housing data now that you’re recording and your browser should look like this:
To scrape as many listings as possible, try panning around the map and zooming in and out. You can also scroll down on the listings and keep clicking the next button. This will get Zillow to load more JSON data payloads into your browser which you’re now recording, so the more data you get from Zillow the more you’ll be able to scrape.
2. Export a HAR File
Once you you’ve scrolled through enough Zillow listings, select the Network tab under developer tools and click the down arrow labeled “Export HAR…” to download the recorded Zillow listings data as a HAR file.
Upload this to theĀ HAR File Web ScraperĀ and look for a group of requests labeledĀ /search/GetSearchPageState.htmĀ or /async-create-search-page-state as below.
If you don’t see the group in the HAR file parse page as shown above, you’ll need to go back and re-generate a new HAR file. Make sure that you scroll through the map and results AFTER opening up developer tools, this way you can be sure you capture the data after you begin the recording session.
3. Download Zillow Data
After clicking “Parse Group” you can download the Zillow search results as a CSV file. It should be the first result - just click “Download CSV” and you’ll have the data as a CSV file you can then open in Excel or any spreadsheet program for detailed analysis.
You’ll get back a row for each listing. Be sure to check for duplicates as you will most likely want to de-duplicate by theĀ detailUrlĀ column to make sure you get unique results. You can also use this column to find the public URL of any Zillow listing.
Property Information
Scraping property information from Zillow will be limited to the data presented on the public search results page. You’ll also get back the following columns which should be useful in your analysis (note we cannot tell you exactly what these are since this is an unofficial API). Also, please note that these columns will vary if you’re searching for rentals vs. homes for sale, so these may not appear exactly in your CSV file depending on your exact scraping approach.
buildingIdĀ - This appears to be an identifier for each building, but is also just the latitude and longitude coordinates of the building combined together with aĀ -priceĀ - The monthly rental price or price to buyminBedsĀ - Most likely the number of bedrooms in the propertyminBathsĀ - Most likely the number of bathrooms in the propertyminAreaĀ - Most likely the square footage of the propertyimgSrcĀ - A link to an image of the listingbuildingNameĀ - Name of the buildingaddressĀ - Mailing address with street and city, but no zip codedetailUrlĀ - Explained earlier, used to get the public URL of the listinghas3DModelĀ - If the listing offers a 3D modellatLong.latitudeĀ - Latitude of the listinglatLong.longitudeĀ - Longitude of the listinghdpData.homeInfo.daysOnZillowĀ - Number of days the listing has been on ZillowhdpData.homeInfo.rentZestimateĀ - Zestimate value of the listing
Depending on your business needs, you can ignore some columns and use others. The easiest way to get started is to simply open up the CSV in Excel or Google Sheets and look for the columns that will be of interest to you.
Property Details
The method outlined above to legally export Zillow data is mainly useful for scraping search results from the website, as it’s an easy way to extract data from Zillow for hundreds of listings in only a few minutes. The details you’ll see for each property will be limited to what Zillow returns in its search results response as outlined above.
If you need to pull data from Zillow for more comprehensive information on each individual listing, then you can use the detailUrl column for each listing and just prepend https://zillow.com to that value and open up the full details for each Zillow page. E.g. if this value isĀ /b/wynd-27-wynd-28-miami-fl-9QD6KF/, then the public Zillow URL would be:Ā https://www.zillow.com/b/wynd-27-wynd-28-miami-fl-9QD6KF/
At this point, you may have a list of 100s or more of these properties to extract data from - and scraping Zillow will probably be extremely tempting at this point. While we can’t legally suggest you do this, depending on your coding skills, you may want to look into building a crawler that can go after the specific fields you need for this list of URLs.
Another option is to simply outsource the work - e.g. if you have a list of 500 properties from search results, you can divide the work into 10 lists of 50 and give that to 10 virtual assistants to manually record the data you need for each page. This too is a safer legal alternative to Zillow web scraping these individual pages.
Why Scrape Zillow?
Real estate professionals using Zillow data for market analytics have the competitive edge when it comes to buying, selling or flipping homes. While many traditional real estate databases have been around for decades, Zillow data offers a far more holistic view of any real estate market due to its public & open nature and Zillow scraping allows you to tap into this wealth of information.
Not only is Zillow one of the largest real estate websites in existence, but the real estate listings data on its website offers valuable insights into property information unavailable elsewhere, such as price trends & Zestimate pricing.
Real Estate Investing
When looking to grow your real estate business, investing in the right properties makes all the difference. Zillow properties are unique in that their property listings come with a Zestimate score you can use to compare the sale price with, leveraging Zillow’s data to find undervalued properties.
While such estimates from Zillow are far from perfect, when combined with your own intuition & business knowledge, harvesting and using this data from Zillow can help grow your business faster than your competitors.
One specific example would be finding pre-foreclosure properties on Zillow to purchase before they actually enter the legal foreclosure process. You can use Zillow search results to find these properties, and to do this at scale we can scrape data from Zillow for our own analysis, per the following video.
Contacting Real Estate Agents
In addition to the property details mentioned above, the individual Zillow listings will contain data about realtors you can get in touch with to purchase the property. While web scraping Zillow could technically get you details about each agent, you’ll have a tough time automating your outreach, as many agents need to be contacted directly through Zillow.
This is another example of why fully automated data scraping may not be a good idea, as you may be tempted to spam or abuse the Zillow platform into sending unsolicited messages that will likely get you blocked or banned.
Instead, you may want to simply visit each property listing manually and use the scraped data from the search results to prioritize your outreach. While data can help with your outreach, at the end of the day it’s still a human-to-human interaction that can only be automated so far.
Legal Concerns
The HAR file web scraping approach we outlined above is completely legal as the scraping occurs on a recording of Zillow’s web traffic to your browser, which can be acquired while obeying Zillow’s Terms of Service (e.g. you’re just using their website normally and happen to be recording your web traffic).
However, there are many other illegal screen scrapers out there that promise to scrape Zillow data but will typically just end up wasting your time. Furthermore, there is a highly restrictive official Zillow API that is available we’d like to make you aware of, however odds are you will not be able to get access to it.
Official Zillow API
The Zillow group maintains an API you can use to officially use to extract data from Zillow through their subsidiary Bridge Interactive, however you must apply for access (which usually means enterprise pricing). If you’re a large real estate business with the budget, then check out the Zillow API and see if you can work with it.
Zillow Terms of Service
If you’re not able to use Zillow’s official API, then your next option is to scrape Zillow using an illegal tool. However, the Zillow Terms of Service explicitly forbid “conduct[ing] automated queries (including screen and database scraping, spiders, robots, crawlers, bypassing “captcha” or similar precautions, or any other automated activity with the purpose of obtaining information from the Services) on the Services.”
While it’s not illegal to violate Zillow’s terms, if they catch you doing so they can ban your account and/or block your IP address. Furthermore, if you use a third party to help you violate Zillow’s terms, such as a Zillow data scraper, the provider of that service is breaking the law (see Tortious Interference) and likely to get sued and/or shut down.
Illegal Screen Scrapers
While we can’t legally suggest using a service to violate Zillow’s terms and scrape data, we can offer an overview of the typical tools you’ll find and explain why even the best Zillow scraper is not only illegal, but generally just doesn’t work.
Browser Add-Ons
A few companies offer browser plugins (usually for Chrome) that you install into your browser and they promise to scrape Zillow on your behalf.
The biggest issue with these data scraping plugins is that they have the ability to “hijack” your browser and control its activity (e.g. clicking and scrolling in an automated fashion), which will stem from your IP address and Zillow account if logged in.
Zillow has plenty of anti-scraping techniques it uses to prevent this type of automated access, so you jeopardize getting blocked or banned from Zillow if detected.
Furthermore, these plugins attempt to download data from Zillow HTML code hosted on Zillow’s web pages, which is constantly changing and shifting around. This usually results in missing or incorrect data from Zillow, which can be fatal for your business if using the wrong property prices.
Web Scraping
The other set of tools will live on server farms (or you can run through proxies) and make automated requests to Zillow (often making HTTP requests with a fake user agent to trick Zillow) and download HTML files that are then parsed to find extracted data.
As mentioned with browser plugins, this method of scraping data from HTML pages is extremely error prone since Zillow is subject to shift around how it presents data at any time, for example on its property page data fields.
However, if you have the coding skills, patience and risk tolerance to build or use your own Zillow scraper, you’re free to do so but we can’t legally recommend it!