Submit Your Requirement

eCommerce Data Feeds for Real Time Pricing

The client from Research and Analytics industry needed highly customised web scraper for getting eCommerce data feeds in real time.

Scroll down to discover

E-Commerce Data Feeds for Real Time Pricing

Client Details

Data Analytics and Business research Shop for eCommerce and Retail

Submit requirement

E-Commerce Data Challenge

The client, from Research and Analytics industry, was looking to for a continuous free of quality and clean eCommerce data feeds, so as to power their research and analytics.

They needed easy access to a complete product listing data from specific categories, along with all the product specifications and pricing. The client previously had an internal data team that manually gathered data from various web sources but the results were limited and efforts were high.

Along with the intensive manual effort, data structuring while importing the eCommerce data into their database was a challenge. They needed clean data in their format, so it could be easily uploaded into their internal database, in order to run the comparison engine and perform other monitoring activities.

The client provided us with the list of sources to be crawled, data points required and frequency of data extraction, which was set for daily.

Our team set up the crawlers to fetch the required eCommerce data fields from the specific source sites. This use case comes under our site crawling offering since the source websites had different structuring and design.

The client needed the extracted data in CSV format and to be uploaded to their S3 servers. The initial setup was complete within a few days and the crawlers started delivering the required data immediately.

About 200K records were delivered to the client during the first crawl.

The Solution

    • Crawler Set Up: A crawler was set up that could extract product prices and required data fields for the predefined categories in an automated manner on a daily basis.
    • Data Template:Based on the schema provided by the client, a template was created using which structuring of the data (extraction) would occur.
    • Data Delivery:The final data was delivered in an XML format via the Data API on a daily basis without any manual intervention from either side.

Each record within a dataset had all details i.e. product name, product price, availability status, short and long descriptions, all image URL’s, SKU, dimensions, category, brand, source and the source URL from where it was fetched.

Related Use Cases

Use Case
Retail price monitoring

Retail Price Monitoring for Electronic Company

Read More

Use Case

Competitor Price Tracking

Read More

Use Case

Price Scraping And Matching In Ecommerce

Read More

PromptCloud Benefits

  • Any changes within the source sites were taken care of and clients were abstracted from such issues
  • Any changes with respect to schema was done as requested
  • Other categories could be added as per changing requirements
  • Productivity increased since the data team could work on other projects. Client expanded into other verticals
  • Low turnaround time of data improved the ability to market client’s services and capabilities
  • Value addition from the project was 50 times the spend
  • Data quality levels had increased alarmingly without any time investment from the team



Scraping Product Prices and Reviews


Read More


Amazon Product Scraping: To Identify Best Selling Amazon Products

Read More

location based scraping

Scraping Product Prices and Reviews


Read More

Contact Us

Contact Us
© Promptcloud 2009-2020 / All rights reserved.
To top