Scraping the Home Depot API
You can legally scrape Home Depot catalog data directly from the Home Depot API just by using their official website (without violating their Terms of Service). You’ll be able to extract catalog data on products within a category or search along with price & inventory specific to the store you have selected, allowing you to use this approach as a Home Depot price scraper for continuous monitoring.
Since this method of scraping data doesn’t depend on using automated access, it can never be blocked and will always return 100% accurate and reliable data from Home Depot as shown above.
1. Browse Products
To get started, head to the official Home Depot Website and navigate to any product search results page (either by typing in a query or navigating to a category page). Then right click on the page and hit “Inspect” to open up developer tools. This will begin recording your web traffic so we can scrape the raw product data out of this recording. You’ll then want to refresh the page to reload the initial products into your web browser.
If you want to take a closer look at what’s going on, click on the Network tab in developer tools and type in graphql under the filter. You should see a request with ?opname=searchModel and if you click inside that request you’ll see the raw JSON data from Home Depot. Below is a sample of what one of the products will show:
{
"identifiers": {
"storeSkuNumber": "256276",
"specialOrderSku": null,
"canonicalUrl": "/p/4-in-x-4-in-x-8-ft-2-Ground-Contact-Pressure-Treated-Southern-Yellow-Pine-Timber-194354/205220341",
"brandName": null,
"itemId": "205220341",
"productLabel": "4 in. x 4 in. x 8 ft. #2 Ground Contact Pressure-Treated Southern Yellow Pine Timber",
"productType": "MERCHANDISE",
"modelNumber": "194354",
"parentId": "205220341",
"isSuperSku": false,
"sampleId": null,
"__typename": "Identifiers"
},
"installServices": {
"scheduleAMeasure": false,
"gccCarpetDesignAndOrderEligible": false,
"__typename": "InstallServices"
},
"info": {
"sponsoredMetadata": null,
"sponsoredBeacon": null,
"swatches": [],
"hidePrice": false,
"ecoRebate": false,
"quantityLimit": 0,
"categoryHierarchy": [
"Lumber & Composites",
"Decking",
"Deck Posts",
"Wood Deck Posts"
],
"sskMin": null,
"sskMax": null,
"unitOfMeasureCoverage": null,
"wasMaxPriceRange": null,
"wasMinPriceRange": null,
"productSubType": null,
"customerSignal": null,
"isBuryProduct": null,
"isGenericProduct": null,
"returnable": null,
"isLiveGoodsProduct": false,
"isSponsored": null,
"globalCustomConfigurator": null,
"augmentedReality": false,
"hasSubscription": false,
"samplesAvailable": false,
"totalNumberOfOptions": null,
"classNumber": "20",
"productDepartment": "Wood Deck Posts",
"__typename": "Info"
},
"itemId": "205220341",
"dataSources": "searchNav",
"media": {
"images": [
{
"url": "https://images.thdstatic.com/productImages/1ba2667d-0dc4-4b85-b047-7a4660cfe8a7/svn/pressure-treated-lumber-194354-64_<SIZE>.jpg",
"type": "IMAGE",
"subType": "PRIMARY",
"sizes": [
"65",
"100",
"145",
"300",
"400",
"600",
"1000"
],
"__typename": "Image"
},
{
"url": "https://images.thdstatic.com/productImages/c1a551bf-683f-4bac-af41-8363b6818415/svn/pressure-treated-lumber-194354-e4_<SIZE>.jpg",
"type": "IMAGE",
"subType": "SECONDARY",
"sizes": [
"65",
"100",
"145",
"300",
"400",
"600",
"1000"
],
"__typename": "Image"
}
],
"__typename": "Media"
},
"pricing": {
"value": 10.78,
"alternatePriceDisplay": false,
"alternate": {
"bulk": null,
"unit": {
"caseUnitOfMeasure": null,
"unitsOriginalPrice": null,
"unitsPerCase": 0,
"value": null,
"__typename": "UnitPricing"
},
"__typename": "Alternate"
},
"original": 10.78,
"mapAboveOriginalPrice": null,
"message": null,
"preferredPriceFlag": false,
"promotion": {
"type": null,
"description": null,
"dollarOff": 0,
"percentageOff": 0,
"promotionTag": null,
"savingsCenter": null,
"savingsCenterPromos": null,
"specialBuySavings": null,
"specialBuyDollarOff": null,
"specialBuyPercentageOff": null,
"dates": null,
"__typename": "Promotion"
},
"specialBuy": null,
"unitOfMeasure": "each",
"__typename": "Pricing"
},
"reviews": {
"ratingsReviews": {
"averageRating": "4.3775",
"totalReviews": "2644",
"__typename": "RatingsReviews"
},
"__typename": "Reviews"
},
"details": {
"installation": null,
"collection": {
"name": null,
"url": null,
"collectionId": null,
"__typename": "Collection"
},
"highlights": null,
"__typename": "Details"
},
"fulfillment": {
"anchorStoreStatus": false,
"anchorStoreStatusType": "ACTIVE",
"backordered": false,
"backorderedShipDate": null,
"bossExcludedShipStates": "AK,GU,HI,PR,VI",
"excludedShipStates": "AK,GU,HI,PR,VI",
"seasonStatusEligible": null,
"fulfillmentOptions": [
{
"type": "pickup",
"fulfillable": true,
"services": [
{
"deliveryTimeline": null,
"deliveryDates": null,
"deliveryCharge": null,
"dynamicEta": null,
"hasFreeShipping": false,
"freeDeliveryThreshold": null,
"locations": [
{
"curbsidePickupFlag": null,
"isBuyInStoreCheckNearBy": null,
"distance": 0,
"inventory": {
"isOutOfStock": false,
"isInStock": true,
"isLimitedQuantity": false,
"isUnavailable": false,
"quantity": 1146,
"maxAllowedBopisQty": null,
"minAllowedBopisQty": null,
"__typename": "Inventory"
},
"isAnchor": true,
"locationId": "6521",
"state": "TX",
"storeName": "Brownsville",
"storePhone": "(956)544-5466",
"type": "store",
"__typename": "Location"
}
],
"type": "bopis",
"totalCharge": null,
"__typename": "Service"
}
],
"__typename": "FulfillmentOption"
},
{
"type": "delivery",
"fulfillable": true,
"services": [
{
"deliveryTimeline": "Tomorrow",
"deliveryDates": null,
"deliveryCharge": null,
"dynamicEta": null,
"hasFreeShipping": false,
"freeDeliveryThreshold": null,
"locations": [
{
"curbsidePickupFlag": null,
"isBuyInStoreCheckNearBy": null,
"distance": 0,
"inventory": {
"isOutOfStock": false,
"isInStock": false,
"isLimitedQuantity": false,
"isUnavailable": false,
"quantity": 1395,
"maxAllowedBopisQty": null,
"minAllowedBopisQty": null,
"__typename": "Inventory"
},
"isAnchor": true,
"locationId": "6521",
"state": "TX",
"storeName": "Brownsville",
"storePhone": "(956)544-5466",
"type": "store",
"__typename": "Location"
}
],
"type": "express delivery",
"totalCharge": null,
"__typename": "Service"
}
],
"__typename": "FulfillmentOption"
}
],
"onlineStoreStatus": false,
"onlineStoreStatusType": null,
"__typename": "Fulfillment"
},
"taxonomy": {
"breadCrumbs": [],
"__typename": "Taxonomy"
},
"availabilityType": {
"type": "Browse Only",
"discontinued": false,
"buyable": false,
"status": false,
"__typename": "AvailabilityType"
},
"badges": [
{
"name": "bestseller",
"label": "Best Seller",
"__typename": "Badge"
}
],
"dataSource": "searchNav",
"favoriteDetail": {
"count": 16163,
"__typename": "FavoriteDetail"
},
"keyProductFeatures": {
"keyProductFeaturesItems": [
{
"features": [
{
"name": "Nominal Product Length (ft.)",
"refinementId": null,
"refinementUrl": null,
"value": "8 ft",
"__typename": "KeyProductFeaturesFeature"
},
{
"name": "Features",
"refinementId": null,
"refinementUrl": null,
"value": "Stainable",
"__typename": "KeyProductFeaturesFeature"
},
{
"name": "Features",
"refinementId": null,
"refinementUrl": null,
"value": "Paintable",
"__typename": "KeyProductFeaturesFeature"
}
],
"__typename": "KeyProductFeaturesItem"
}
],
"__typename": "KeyProductFeatures"
},
"specificationGroup": null,
"subscription": null,
"__typename": "BaseProduct"
}
With developer tools still open, you’ll want to scroll down through the results in Home Depot and keep clicking on the next button to load more and more products. Click through as many products as you’d like to scrape and when you’re done proceed to the next step.
2. Export a HAR File
To get the data out of your browser, you want to click on the Network tab under developer tools and look for the down arrow labeled “Export HAR…” and click it. This will download a HAR file containing the Home Depot data. You’ll then want to upload that to the HAR File Web Scraper to get the data out. Look for the following group that should be labeled with /federation-gateway/graphql:
This group contains the network data for all of the pages you browsed through in the previous step, and parsing it will extract the data out into downloadable collections.
3. Download Home Depot Data
You should see the product data under the data › searchModel › products collection in the response. This will contain a row for every product scraped:
If you get stuck or need help following any of these steps, please see our video walkthrough on this page. We’ll walk through some of the more interesting data fields you may be interested in below, such as price and inventory.
Product Data
You’ll see standard data about each product including the following fields:
- Home Depot Item ID
- SKU Number
- Home Depot URL
- Product Name
- Manufacturer
- Model Number
- Product Department
- Main Image URL
- Original Price
- Current Price
- Review Average
- Number of Reviews
- Inventory in Stock
Pricing & Inventory
Note that the pricing and inventory data will be specific to the store you have selected on the top left part of the Home Depot website. You’re free to change the store, but each time you do this you’ll need to regenerate a new HAR file for that specific store.
If you need inventory data, we highly recommend not relying on the main collection shown above as each product can have multiple fulfillment options and inventories. To see these, click on the “All Collections” tab and look for the data › searchModel › products › fulfillment › fulfillmentOptions › services › locations collection:
Click Download CSV and the downloaded CSV file will contain a reference to each row’s parent product so you can be sure you know what numbers you’re looking at. As this data is being scraped from an unofficial API that Home Depot uses to power its website, we highly suggest cross referencing a few examples to be sure you’re looking at the correct rows and columns, as these may be subject to change without notice.
Why Scrape Home Depot?
Collecting the product catalog data can be useful if you’re looking to get a sense for what manufacturers are experiencing strong sales in certain areas, or if you’re just performing more general market research. What’s great is that you can select a few target Home Depot stores and product categories (or search terms) and get a quick pulse of inventory levels and pricing throughout the United States.
If you want to track inventory and pricing, you can run these HAR file scrapes on a weekly basis and compare the inventory and pricing levels. You’ll be able to track which manufacturers are selling well (when you see continuous drops in inventory levels) as well as any price changes.