Scrape All YouTube Videos from Any Channel
Creating new ideas for YouTube videos is hard. Stealing is easy. This is why it may be tempting to scrape all of the videos (including stats) from similar channels to see what types of content works better than others before investing in creating the content yourself.
We’ll discuss how to use the official YouTube Data API to scrape the entire list of videos for any public YouTube channel, along with stats like view count, like count, comment count, publish datetime, etc…
In order to scrape data, you simply need to gather a list of Channel IDs from which to scrape the videos from. You can get these from the YouTube channel page from the URL. E.g. if you visit the page here: https://www.youtube.com/channel/UCArmutk8nAbYQdaYzgqKOwA you’ll see from the URL that the Channel ID is UCArmutk8nAbYQdaYzgqKOwA.
If the URL looks different, e.g. contains the username like https://www.youtube.com/c/StevesieData - you can get the Channel ID by clicking into one of the videos, and then from the video watch page click on the channel name to navigate to the URL with the Channel ID in it.
Using the YouTube Data API
In order to use the YouTube API, you need to get an API key. But don’t worry, this is very easy and free - we have an article here describing how to get your YouTube API key.
Channels Under 500 Videos
If all of the channels you want to scrape each contains under 500 videos (or you’re happy just getting the most recent or popular 500), then scraping becomes a little easier as we can use the YouTube API Search List Endpoint with the channelId parameter set to the Channel ID you want to scrape the videos for. You’ll also need to set the type parameter to video for it to work.
You’ll also want to set the order parameter to get the desired order (especially if the channel has over 500 videos and you only want a subset). Setting it to date may be a good idea, as it will return them with the most recent videos first. You can also sort by viewCount to get the most popular videos.
Scraping with Stevesie If you’d rather not write code and are OK using a paid service, you can use the Stevesie YouTube Search Integration to run the search. Simply provide your API key with the relevant inputs and you’ll get back the results like this, which you can download as CSV format:
Now to get back the next 50 results (up to 500 results), you need to pass the nextPageToken found in the YouTube API response into the pageToken input for the subsequent request and repeat for 10 total requests. You can also use the YouTube Search - Pagination Workflow to automatically do this for you and download the aggregate results in a single CSV file.
Scraping Video Details & Statistics
You may notice that the search results do not include any stats about the videos, like view count, comment count, etc… To get these, we need to use the YouTube API Video Details List Endpoint which can work with a list of Video IDs (that we collected in our previous step from the search results).
You’ll want to pay special attention to the part parameter here, which can accept a comma-separated list of values. To get the statistics, simply add statistics to the part parameter. A good starting point for the part parameter is id,snippet,contentDetails,statistics,status,topicDetails, which should return a good amount of data for each video.
You can refer to the YouTube Video Resource Documentation to see exactly what kind of data you’ll get back.
Scraping with Stevesie If you need help querying the YouTube API for a list of video IDs to get their statistics back, see the YouTube Video Details - Multiple Videos Workflow, which will let you enter a list of video IDs (one per line) and aggregate all of the results together into a single CSV file.
Channels Over 500 Videos
If you’re scraping the videos for a larger channel (over 500 videos) and insist on getting ALL the videos back, you’ll need to use “playlist scraping” to get them all back. It’s a 3 step process, where we first need to find the channel’s special Playlist ID for all of its uploads, then get the videos for this playlist, and then look up the video details for all videos in this response set.
1. Get the “Channel Uploads” Playlist ID
To get the Uploads Playlist ID for a channel, you can use the YouTube Data API Channel List Endpoint which will accept the channel ID and return a special playlist ID in the contentDetails.relatedPlaylists.uploads part of the response. You can see a sample of what this looks like from the Stevesie Channel Details Integration:
2. Get the Playlist’s Videos
Once you have the Playlist ID for all of the channel’s uploads (e.g. UUArmutk8nAbYQdaYzgqKOwA), we can then pass this into the YouTube API Playlist Items List Endpoint to get back all of the videos in the playlist. Note that you will need to paginate in order to scrape the full list by passing the nextPageToken around, which is something the Stevesie Data Platform can do for you (more about this later).
3. Look Up Video Details
We’re now at the point where we were in the previous section, under the Scraping Video Details & Statistics heading where we have a list of Video IDs and need to extract the details, statistics, etc… So once again we’ll want to use the YouTube API Video Details List Endpoint and pass in a list of video IDs to get back the statistics.
Scraping with Stevesie
While you’re welcome to access the YouTube API directly and perform this 3 step process (writing code, etc… to deal with the pagination and passing around inputs), you may want to consider the YouTube Channel -> Video Details Workflow which automates these 3 steps and can work with a list of Channel IDs. You simply provide the Channel IDs you need the videos (and details) for, and the workflow will process the list and return a combined CSV file of all the videos from your channel list with their statistics.