Download Public Repo & User Data from GitHub
If you’re trying to scrape data from the GitHub website, don’t. Check out the GitHub REST API instead, which offers a vast number of API endpoints you can use to collect public data from the GitHub website from. A few examples you may find useful is search result scraping, where you can enter the keyword of a language or technology you want to scrape repositories for. This will give you code stats and references to the repository maintainers, including contact information like email address when made public.
You can also scrape activity stats using some of the API endpoints, for example if you’re interested in tracking the weekly commit activity for repositories you’re interested in, the number of issues, pull requests, etc…
While you’re welcome to query the GitHub API directly, if you just need to download CSV files containing data around what you’re looking for, our service specializes in this. We query the GitHub API on your behalf and then parse out the response into downloadable CSV files you can quickly analyze in Excel or your tool of choice. Check out some of the endpoints here that we currently support or reach out to support here if you need us to support more endpoints from the GitHub API.
