Offering: Site-specific crawl and extraction
Challenge: Client’s requirement was to crawl details of the best-selling products from Amazon in order to identify the potential best-sellers on their platform. Best seller products from the entire catalog as well as the best sellers under different categories were needed separately. The required data points were Product name, description, specifications, brand, model, , bestseller rank, reviews count and ratings.
The Solution: Client provided us with the list of specific data points to be extracted from Amazon. Fresh data sets were needed from Amazon every week. PromptCloud set up web crawlers by programming them in order to fetch the required data fields. This particular case comes under our Site specific crawl offering as the crawler is programmed specifically to fetch data from a particular source website. With our years of expertise in web crawling and scraping, we started delivering clean data to the client soon after the setup was successfully completed. The data delivery format opted for by the client was XML and the data was directly uploaded to client’s S3 location.
- Every technically complicated aspect of the crawl and extraction was taken care of by our team
- The setup was complete in a few days and the data was provided in record time
- Client had all the bestselling products from Amazon, category-wise, ready to be fed into their applications.Our advanced tech stack handled millions of records each week with ease
- Monitoring was set up for the source website to detect changes that need modification of our crawlers