Stevesie
  • Apps
  • Pricing
  • Services
  • Docs
  • Login
Docs
  • Apps
  • Tasks
    • Requests
    • Inputs
    • Collections
  • Workflows
    • Inputs
    • Dependencies
  • Workers
    • Running
      • Inputs
        • Geolocation
        • Pagination
        • Interceptable
      • Vault
      • Scheduling
    • Scraping
      • Collections
      • Deduplication
    • Exporting
      • Formats
      • Realtime
  • Proxies
    • Anonymity
    • Interception
      • Connecting
        • iOS
      • Recording
      • Playback
    • Billing
  • API
    • Authentication
    • Endpoints
    • Clients

Deduplication

For task collections with identity fields, repeat scrapes of objects with the same identifier will upsert the new object into the existing one, eliminating duplicates.

Upserts

If a collection item is found that has the same identifying field as an object previously scraped (e.g. the same ID), then the new object will replace the old object. This makes it easy to continually scrape a source and not have to worry about deduplicating the results when there is overlap.

Latest
Machine Learning Image Popularity - Predict Image Success

Machine Learning Image Popularity - Predict Image Success

CSV to SQL Database - Import from CSV or Excel Into a Postgres Database

CSV to SQL Database - Import from CSV or Excel Into a Postgres Database

CSV to Google Maps - Plot Locations from CSV or Excel to Google Maps

CSV to Google Maps - Plot Locations from CSV or Excel to Google Maps

Airbnb Pricing Strategy - Competitive Analysis

Airbnb Pricing Strategy - Competitive Analysis

Home Depot vs. Lowe's Price Comparison

Home Depot vs. Lowe's Price Comparison

U.S. Cities Geographic Coordinates

U.S. Cities Geographic Coordinates

More...

  • Contact
  • ·
  • Terms
  • ·
  • Privacy
  • ·
  • © Stevesie, LLC