Scraping TikTok Search Results
If you’re searching for influential TikTok creators and/or content, using the TikTok search feature can be a great way to scrape this data from TikTok.
You can search by general interest terms, e.g. “dance” and see top posts & creators for that niche. In this article, we’ll show you how to export these search results to CSV files using a recording of your web traffic.
1. Browse TikTok Search Results
To get started, head to the TikTok Search Page and enter any search term relevant to your niche. E.g. here we’ll be searching for “dance” as a general example, but you can search for whatever niche you’re interested in finding creators for.
After entering a search term, you can apply any filters you’d like or also change the search option to only show either videos or accounts if you’d like. Once you’re happy with the results shown that you’d like to scrape, right click on the page and hit “Inspect” to open up developer tools. This will begin recording your web traffic so we can legally scrape the data from this web traffic recording.
Once developer tools are open, refresh the page (to reload the 1st page of results since you’re now recording) and just keep scrolling down through the results. TikTok will cut you off after about 150 results, but you can change the search term and run a new search to keep loading more and more data into your browser’s recording.
2. Export a HAR File
Once you’ve scrolled through enough search results, look for the down arrow labeled “Export HAR…” under the “Network” tab in developer tools to download a HAR file containing the raw data we just browsed through. Then upload that file to the HAR File Web Scraper and click “Parse Group” on the following group in the results that should start with /api/search in the title:
3. Download TikTok Search Results
Once the parse completes you should see a data › item collection where each row will contain a video (if using the “Top” option for searching on TikTok). This may vary slightly if you selected the “Accounts” or “Videos” option while browsing TikTok.
What’s nice about scraping the “Top” section for video search results, is that the response data contains information on both the content and the author of the content with popularity metrics. So if you’re looking for influencers, you can sort this data by popularity metrics on both the content and creator level.
Some fields of interest in the results are:
- Post ID
- Post Caption
- Timestamp
- Video ID
- Video Duration
- Video Thumbnail URL
- Media URLs (may not be accessible)
- Author ID
- Author Username
- Author Full Name
- Author Profile Image URL
- Author Bio (Sometimes has IG & Email)
- Is Author Verified?
- Author Secondary ID (Can Use with TikAPI)
- Music ID
- Music Title
- Music Author Name
- Post Engagement Stats (Likes, Plays, Comments, Shares)
- Author Follower Count
- Author Following Count
- Author Heart Count
- Author Video Count
- First Hashtag Used
Hashtag Data
If you’d like to analyze the hashtags used in the search results, look for the data › item › challenges collection on the parsed results. Each row will contain a hashtag and post combination, so you have a quick way to analyze all of the hashtags used in the search results. Click “Download CSV” on the collection to see each hashtag’s parent post reference on the right-hand side columns.
