Download Tweets from the Twitter API
One of the main features of the Twitter API V2 is the ability to scrape a vast amount of Tweets. In fact, this is the main way that Twitter tracks its API’s usage, by number of Tweets scraped per month with tiers starting at 10,000 per month from the past 7 days.
There are several ways you can hit this quota and scrape Tweets, and this page contains links above covering the most popular use cases we help customers with on our platform. Below is a summary of these methods:
Search Results
Scraping Tweets by search results is the most common approach we see, as Twitter allows for some very powerful and precise targeting using keywords, hashtags, languages, and even locations (academic access needed). You can simply type in a keyword or hashtag (just as you can do on the Twitter website) and instantly download thousands of matching Tweets using our scraping service.
User Tweets
Another popular method is to scrape Tweets from a specific user, such as yourself. There is a separate Twitter endpoint for scraping a user’s timeline, but you can also use the search scraper to do this.
Conversation Tweets
If you’re running a giveaway and need to see who replied to a specific Tweet, then check out the Conversation Scraper above. You’ll be able to input a Tweet ID and get back the individual Tweets including author of everyone who replied. You should also be able to use this method to capture retweets and the page above will explain more.
Historical Archive
Back before Twitter was X, you could access the historical archive if you had a specially approved academic access account. Now it’s much more affordable and you only need to pay $5,000/mo. to access this data from the full archive. Since this is out of most people’s budget, we usually see people opt for the “Basic” Twitter API access tier, which is only $100/mo. and allows you to scrape the past 7 days of Tweets.
So if you have your heart set on scraping historical data, you may want to rethink your requirements and consider what you can do instead with more recent data from the past week. Twitter (or X) is meant to be a real time social media network, so this may be a more interesting approach.