TikTok Hashtag Scraper
If you need to extract data from TikTok, such as from a list of targeted hashtags relevant to your business or niche and their respective posts, we’ll show you how to legally scrape this data directly from TikTok without violating their Terms of Service using a little-known technique involving HAR files.
Scraping TikTok hashtags can be especially helpful for finding user profiles who post relevant content ideas on TikTok, based on hashtag posts & engagement metrics you can scrape. You can also analyze the posts of specific TikTok profiles to understand which hashtags they regularly use.
1. Browse TikTok Hashtags
Let’s say you’re interested in TikTok data scraping for hashtags about Dogs. Check out all of these posts & influencers on the #dogsoftiktok by visiting the TikTok Hashtag Page (or browse to any other TikTok web page you’re interested in).
When you get to the target page (you can also enter a hashtag in the search bar) and see the data you want to scrape, right click on the page and hit “Inspect” to open up developer tools. This will begin recording your web traffic so we can capture the raw data that TikTok sends to your browser, containing the specific TikTok data we’ll use with our TikTok scraper.
Now simply scroll down through all the hashtag posts (while recording) until you get through a good amount. We typically see our users get through a few thousand easily until the web browser becomes unresponsive.
2. Export a HAR File
Once you’ve browsed through the data you want to scrape from TikTok, look under the “Network” tab in developer tools and click on the down arrow labeled “Export HAR…” to download a HAR file containing a copy of all the data TikTok sent to your browser with the hashtag data in it.
Upload that file to the HAR File Web Scraper and look for a parse group that starts with /api/ similar to the screenshot above and click “Parse Group” to extract the hashtag data from the HAR file scraping tool.
3. Download Hashtag Data
You should then see an item_list (or similar) collection on the downloadable collections screen. You’ll be able to download the post data from that screen.
In addition to seeing data about the hashtag post in the scraped TikTok data, like caption & timestamp, you should also see engagement metrics about each individual post to sort them by which posts are getting the most attention. From there you can also see data about each author to help identify potential influencers. E.g. you should be able to see the follower count of each post’s author if you’re looking for sponsorship deals with influencers.
Automated TikAPI Hashtag Scraper
While the HAR web scraping approach is good for quick & reliable exports, you may need to use an automated approach if your goal is to scrape millions of datapoints, perhaps to build a database or perform a very large amount of data analysis.
This is where TikAPI comes in, the unofficial TikTok API that allows us to scrape relevant TikTok video posts & corresponding authors based on hashtags. Below we’ll cover how to use TikAPI to scrape hashtag data from one or multiple hashtags of your choosing.
1. Get a TikAPI Key
You can evaluate TikAPI for 5 days at no charge. Just visit their site and sign up, then you’ll receive an API key you can use to follow the rest of this article. You may also want to reach out to their customer support if you’re not comfortable providing payment information upfront and they can send you a free trial key like they did with us.
2. Get the Hashtag ID
In order to scrape large amounts of TikTok video posts & their influencer authors, we want to use the TikAPI Posts by Hashtag Endpoint, but provide the id input and not the name per their notice:
Note: You can only get a full list of results by using the hashtag ID. You can find the ID by sending a request using the hashtag name.
What this means is that we first need to run this endpoint and provide the name parameter, or hashtag name. In our earlier example for #dogsoftiktok, the name parameter would be dogsoftiktok (just remove the # sign).
Next, you can execute the endpoint and you’ll see the hashtag “ID” under the challengeInfo.challenge.id field in the response. For some reason TikTok calls hashtags challenges… probably some old naming from before they became super popular. The ID will simply be a number, like 323964 - copy this and we’ll use it next.
If you need help accessing this endpoint to get the Hashtag ID, you can use our TikAPI Hashtag Posts Integration on this page. Just scroll up and look for the green box and provide the hashtag you’d like to collect the data for. You will also need to provide your TikAPI key so we can make the API requests on your behalf:
Our scraper will query TikAPI on your behalf and let you quickly get the Hashtag Challenge ID.
3. Scrape Hashtag Posts
Once we have the hashtag ID (or challenge ID), we can use the TikAPI Posts by Hashtag Endpoint again, but this time provide a value for the id parameter and leave the name parameter blank. If you use the green box as shown above, you’ll see a follow-up screen allowing you to enter more options around the TikAPI endpoint. E.g. just remove the name parameter and provide the id parameter instead:
You’ll also want to set the count to 30 to get as much data back from TikAPI per request too. Execute the endpoint and you’ll get back a list of up to 30 TikTok posts as well as details about their respective authors in the response.
Pagination
To scrape back the full list of hashtag posts (or at least as many as TikTok will show on its public website), we need to use pagination and take advantage of the cursor parameter in the endpoint. In the response you’ll see a field called cursor, which will contain the cursor value you need to supply to get the next page.
For example, if you entered 30 for count on the initial request, the response should return 30 to signal the value you need to provide as the cursor input to get the next page of results. When supplied, you’ll get the 2nd set of 30 results and the cursor response should indicate 60, which you can provide back to the endpoint again to get the third page of results and so on.
Workflows
If you need help downloading this data into a CSV file and don’t want to deal with writing code to handle pagination, you can use the TikTok Hashtag Posts - Pagination Workflow, which will allow you to enter a single hashtag challenge ID (or a list of them), and the workflow will take care of pagination for you and combine all of the video post & influencer results into a single CSV file you can then analyze or upload into your own systems. Note this is a paid service that we provide, separate from TikAPI.