Scraping the SoundCloud API
You can legally scrape data from SoundCloud directly from their website without violating the Terms of Service using a little-known technique involving HAR files.
The method works by recording your web traffic as you browse data on SoundCloud (such as search results), and our scraper then extracts data from the recording instead of the SoundCloud website. Read on below to get started and see for yourself.
1. Browse SoundCloud
You can scrape anything that interactively loads on SoundCloud, but you’ll probably get the most useful data while using the SoundCloud Search Feature to scrape the tracks, artists or playlists you’re interested in.
Just run a search for anything on SoundCloud and apply any filters you’d like to use. Once you see the data loaded you want to scrape, right click in your browser and hit “Inspect” to open up developer tools and begin recording your web traffic.
Now refresh the page (to force SoundCloud to reload the initial results while you’re recording) and scroll down through the results. They offer an infinite scroll, so keep going until you’ve scrolled through enough content that you’d like to scrape.
2. Export a HAR File
Once you’re done scrolling, look for the “Network” tab in developer tools and click the down arrow labeled “Export HAR…” to download a HAR file containing all of the raw network data that SoundCloud sent to your browser.
Upload that file to the HAR File Web Scraper and look for the group labeled /search/tracks (or similar, but it will likely need to start with /search) - note that you may need to scroll down a bit to find it (see example screenshot above).
3. Download SoundCloud Data
Click “Parse Group” on the HAR File Web Scraper and we’ll combine & parse the data together so you can quickly download the data to CSV files. You should see the first group labeled collection that will contain the track (or similar) data flattened out.
Below are some relevant fields you may find useful in the extracted Excel file:
- Track ID
- Title
- Description
- Artwork URL
- Comment Count
- Creation Timestamp
- Is Downloadable?
- Download Count
- Duration
- Genre
- License
- SoundCloud URL
- Play Count
- Purchase URL
- Repost Count
- Tags
- Waveform URL
- SoundCloud Username
- SoundCloud User Description
- User Follower Count
- User Full Name
- User Like Count
- User Track Count
- User URL
- User Subscription Level