Scrape Twitter Replies
To scrape replies for any Twitter conversation, the Twitter API features a special method to collect data from replies & retweets for any public Tweet using Conversation IDs.
We’ll walk through the basics here so you can scrape Tweet replies for any Tweet you’re interested in. This will assume you have “basic” Twitter API level access ($100/mo.) and as a result you’ll only be able to scrape Twitter data & replies from the past 7 days as this uses the Twitter search API endpoint.
1. Get the Conversation ID
If you already have the Tweet URLs to the Tweet(s) you want to scrape the replies of, like this Sample Tweet from Google you can find the Conversation ID from the Twitter website’s URL:
https://twitter.com/Google/status/1555434849908903936
So here the Conversation ID would be 1555434849908903936 that we can now use with the Twitter API to look up the replies. If you’re using our Twitter Scraper to find other Tweets you want to see the replies of, the conversation ID will be in the conversation_id column, and not in the id column of the Tweet as the Twitter API Separates out Conversations in its data modeling.
2. Scrape Replies
Once you have the ID, simply enter it here on the green box in the Twitter Scraper on this page to scrape all of the associated Tweets:
Under the hood, this will query our Twitter Search Results Scraper so you can use any of the other search or filtering options available on that page. For instance, you may want to add is:reply to your query, so the full query would be conversation_id:1511082913265160193 is:reply to get all the replies for this Tweet.