EMAIL : sales@promptcloud.com
Client’s Challenge
Being an e-commerce platform, the client sought complete product and pricing data from its outlets spread across the United States. Before looking to employ web scraping services, the client used to gather relevant data manually to perform the analysis.
The requirements included extracting product data from their e-commerce platform and retail store outlets across the nation, filtered based on zip codes of store locations. Taking the above-mentioned location based data mining, a similar data collection process had to be performed on competitor e-commerce sites to extract price and product data.
This data was used for further analysis in product strategy and price benchmarking for a complete product catalog. The client wanted to deploy PromptCloud’s web scraping service to automate the entire data extraction process based on location, zip codes in particular.
The Solution: Location based Data Mining
Site-specific crawls were employed in this case, which focused on the client’s website. The solution extracted pre-defined data points from the client’s website; important data fields were the unique serial identifier of a product, product name, category, URL link, crawling timestamp, store location, price, and inventory stock availability.
Considering the client’s interest in pricing benchmarking, web scrapers were also created for the competitors’ sites. Crawlers collected data from fields such as the unique identifier of a product, URL link, product name, category, crawl timestamp, store, location, price, stock availability in the inventory.
The collected data from the above two executions were then classified by zip codes for location, and was used by the client for further analysis. The dataset was delivered to the client in JSON format via PromptCloud’s REST API
Web Scraping Benefits
- Noise-free data is made available to the client based on the requirements
- Cut-down on redundancy since the client listed out which stores they wanted to set web scrapers for data extraction
- No client intervention was required during the location based data mining process
- Reduced cost and time delay for the client, since clean data was delivered for analysis
- The schema was altered as per client’s request
- Periodical updates based on the frequency of scraping was also incorporated
