Whether you're building an app or researching product pricing, trends & availability, you most likely need to know the pricing and inventory details of local products. Since this information is often publicly available on retailer apps & websites via APIs, it makes it easy to scrape.
You'll want to collect some key metrics for the products you're interested in. You will typically want to first build up a list of products by SKUs, UPCs, etc... and then look up their pricing & inventories at local retailers.
While you can manually browse through retail websites and apps all day to copy-paste this data into your own systems, many companies are resorting to automated data scraping to collect this public information. We'll discuss some common approaches to scraping this public data for use in your own apps, systems and services below.
You'll usually want to focus on scraping details about a fixed list of products you're interested in (it's not realistic to scrape an entire store catalog). So you can start by searching for product keywords, brands or categories you're interested in to generate a list of products if needed
You can perform keyword searches on several product retailer APIs, retrieving results back just as you would by using the normal website's search feature.
Once you have a list of products you're interested in tracking, you'll want to find local stores to check the inventories on.