Watch on YouTube
Scraping the Nextdoor API
To harness Nextdoor’s distinct data & content, you can legally scrape data from your Nextdoor account, including your newsfeed, conversations, events and marketplace listings using a special technique involving HAR files. This approach allows you to legally record your web traffic without being detected (or blocked) and we then perform the web scraping from the HAR file.
You will NOT be able to scrape data that you’re not allowed to see while logged in to Nextdoor. So please don’t try to use this if you’re trying to scrape data from other neighborhoods and locations that you don’t have a Nextdoor account or access to.
1. Browse Nextdoor
Head on over to the Nextdoor Website in your web browser and log in if you need to. Then navigate to the section of Nextdoor you want to scrape, e.g. the news feed, marketplace, search results, etc…
Now right click inside your browser and hit “Inspect” to open up developer tools to begin recording your web traffic. Now refresh the page to force Nextdoor to reload its initial data to your browser now that you’re recording it.
Scroll down through all the posts while you’re recording your web traffic (you can also see the video on this page for a more step-by-step demo) until you reach the end or get through enough data.
2. Export a HAR File
When you’re done browsing the data you want to scrape, go to the “Network” tab under developer tools and click the down arrow labeled “Export HAR…” to download a HAR file containing all of the raw data from Nextdoor you just browsed through.
Upload that file to the HAR File Web Scraper and we’ll group together related requests so you can download the extracted data into single files from multiple requests. Look for a group like the one labeled above, it should begin with /api/ and look similar. Click the blue links to download the raw data, or “Parse Group” to download CSV files.
3. Download Nextdoor Data
When you see the group of requests with the desired data in it from the previous step, click “Parse Group” to extract the JSON data into downloadable CSV files on the next page. Find the CSV files you’d like to download and hit “Download CSV” to save a copy to your computer. The collection will likely end in edges as shown below, but this is subject to change.
You can now open this data in any spreadsheet program like Excel for private use, research & analysis. Please do not republish any of the content, images or videos from Nextdoor, as it will likely violate copyright law and also would violate other Nextdoor users’ expectations of privacy they have on the platform.