Scraping the Foreclosures.com API
If you need foreclosure data, a good place to start is Foreclosure.com which lets you search through thousands of foreclosures all across the United States. This can be a great way to find an underpriced buying opportunity before anyone else does for your real estate investing portfolio.
We’ll show you how to export any set of search results from Foreclosure.com using only your web browser without violating their Terms of Service, as we’ll simply be recording your web traffic as you use the website normally, then perform the scraping on that recording instead of the website.
1. Browse Foreclosures
Head to Foreclosure.com and navigate to any area you want to scrape foreclosures from. Feel free to apply any filters you’d like or change the sort order… just use the site normally to load in the foreclosure data you’d like to export.
Once you’re happy, click the “Map” option on the search results page so we can dynamically load the data (allows us to record the raw data from the API) and right click on the web page and hit “Inspect” to open up developer tools, which will automatically begin recording your web traffic.
Now with the map & developer tools both open, move the map just a bit to force the website to send you additional data and wait for the data to load. To load multiple pages of results (disclaimer you may get duplicates), you can keep panning the map around to scrape more and more results. Keep doing this until you’ve swept through all the data you’d like to scrape.
2. Export a HAR File
After you’ve browsed through all the foreclosures you’d like to export, look for the “Network” tab under developer tools and click the down arrow labeled “Export HAR…” to download a HAR file containing all of the raw JSON data with the foreclosures in them.
Then upload that HAR file to the HAR File Web Scraper to combine the related requests together containing the foreclosure data. Look for the group labeled /listing/search/ui/event (or similar) as shown in the screenshot above. You should see fields like totalCount, query, price, etc… in the fields section.
To download the individual raw JSON to your downloads folder, click on the blue links in the request section. This will give you the raw data you can use with your own code to process and work with the data. However, if you just need a CSV file to work with or analyze in Excel, then click the “Parse Group” button on the HAR grouping.
3. Download Foreclosure Data
Once the parse completes, you’ll see the results collection with all of the scraped foreclosure data in a downloadable format.
Some columns you may be interested are listed below:
- Listing ID (Use to de-duplicate if needed)
- Listing Type (Pre-Foreclosure, Foreclosure, Redemption, Bankruptcy)
- Listing Classification (Short Sale, Auction, Redemption, Pre-Foreclosure, Foreclosure, Other)
- Street
- City
- ZIP Code
- State
- Price
- Number of Bathrooms
- Number of Bedrooms
- Size
- Latitude & Longitude Coordinates
- Details Page URL (Prepend with
https://www.foreclosure.comto access) - Date Listed
- Property Type (Single-Family, Condo, Townhouse, Multi-Family, Unknown, Other)
- Automated Valuation Model (AVM), High & Low
- Rent Estimate, High & Low
Other Foreclosure Data
You can also this same HAR file web scraping approach to scrape foreclosure data from Zillow & Redfin with our Zillow Scraper & Redfin Scraper which work the same way using HAR files. Simply filter to only search for foreclosures and you’ll be able to export them from their respective websites.