Scraping the TikTok API
If you need to download TikTok data and don’t want to mess around with broken TikTok scrapers or risk getting your account banned for automated TikTok scraping, we’ll show you a 100% safe & legal method for extracting data from TikTok without violating the Terms of Service through our HAR File TikTok data export tool.
Since our approach relies on extracting data from a recording of the official TikTok API from your web traffic (instead of the actual TikTok site), it is completely undetectable and allows you to scrape anything that interactively loads on TikTok, so you can scrape TikTok profiles, comments & posts or anything else that is served from TikTok’s API.
1. Browse TikTok
To get started, just open up TikTok on your web browser (Chrome preferred) and browse to anything you’d like to scrape. This will work best with lists of items, such as search results, posts for a person, comments on a post, etc…
Once you’re on the page you want to scrape, right click in your browser and hit “Inspect” to open up developer tools. This will begin recording your network traffic. Now refresh the page to force TikTok to re-send its data to you while you’re recording.
You’ll then want to scroll through the results you want to capture. The more you scroll, the more data TikTok will keep sending to your browser. You can realistically scrape a few thousand items per scrape, but in theory there’s no limit.
2. Export a HAR File
Once you’re done scrolling through the data you want to export, go to the “Network” tab in developer tools and click the down arrow labeled “Export HAR…” to download a HAR file containing all of the data TikTok sent to your browser.
Upload this file to the HAR File Web Scraper and look for a group with /api/ in it like in the screenshot above. This will likely contain the data you want to scrape. Click “Parse Group” to then see the captured data as downloadable files.
3. Download TikTok Data
You can now download the extracted collections, e.g. item_list for search results in JSON and CSV formats. Each row will contain TikTok data for each post found in the search results.
In this example, we can see basic information about the post including the following from our TikTok video scraper:
- Post ID
- Caption
- Timestamp
- Popularity Metrics (comments, shares and views)
- Video Data (length, bitrate, etc…)
- Links to Images of the Video Cover (links to actual video content will not work as they require special headers)
- Author Username
- Author User Profile (which sometimes contains an email address)
- Author’s Popularity Metrics
- Music Used & Popularity Metrics
- Hashtags Used (separate collection you can download)
Why Scrape TikTok?
If you’re interested in TikTok data scraping, you’re certainly in good company as TikTok has risen to become one of the most valuable social media platform, recently surpassing Instagram. As a result, it has built up a lot of interest from influencers & creators looking to extract TikTok data.
TikTok APIs
If you look for an official TikTok Data API for data scraping, you’ll come across the official TikTok Developer Portal which has lots of great resources for how to build apps around TikTok via the TikTok developer API, but nothing in the way of data scraping.
With some more searching, you’ll see that TikTok does not officially support scraping its public data (like Twitter & YouTube do), despite it being incredibly valuable for social media marketers, agencies & brands looking to monitor their performance and connect with TikTok influencers (all just making TikTok that much more valuable of a platform than Instagram ever was).
Unofficial APIs
While TikTok doesn’t offer an official API for scraping, there is TikAPI, which is a full-fledged TikTok Unofficial API Wrapper that allows anyone to extract public data from TikTok via their API. The API also has plenty of cool “write” features, allowing you to manage your own TikTok account, but this webpage only contains integrations focused on scraping public data from TikAPI.
Check out the links above or see our TikAPI Integration for how you can use their API for scraping data from TikTok for your own use. A good starting point is to use the search or discover endpoints to scrape some popular TikTok posts. You’ll see that TikAPI also refers to these “posts” as videos, as in other TikAPI endpoints that ask for a specific TikTok video ID, for example to get the video details including the play URL, heart counts, and play counts.
We suggest using our TikAPI Scraper when available, but you’ll eventually encounter some use cases where TikAPI only works for certain use cases. For example, if you need to download TikTok comments, you’ll notice that TikAPI doesn’t support this option in its Public Data Endpoints. So as an alternative, you can use our HAR File Web Scraping service as outlined in our TikTok Comments Scraper.
TikTok Web Scrapers
There are several implementations of an Unofficial TikTok Data API or scraping tools in Python you can find on GitHub and use as alternatives for web scraping TikTok:
The problem with these however is that they most likely violate the TikTok Terms of Service, are subject to breaking at any time, and require you to actually run the code and put your IP address (and TikTok account) at risk of getting banned or blacklisted. While you’re welcome to use this code for whatever reasons you can imagine, if you simply need to export data from TikTok, this may be overkill.