Client-specific site list
Continuous site monitoring and maintenance
Output Formats options like XML, CSV & JSON
Custom frequencies for crawls
Deep crawl + incremental crawl
Deduplication & client-specific normalization
A: Monthly bill is calculated based on data volume. Eg: If we are crawling a site and deliver, say, 75,000 records in a month, cost for this site on the monthly bill would be: $37.5 (for volume fee)+ $79 (towards site maintenance & monitoring fee) = $116.5 for the month.
Note – this is assuming volume fee to be $5 per 10k records (prorated) and monthly site maintenance & monitoring fee as $79/site.
A: The monthly maintenance and monitoring fee covers technical support, overheads in maintaining the data pipeline and related infrastructure as well as fixing the crawlers in case a target site undergoes structural changes.
A: Yes, our service comes with a lock-in period of 3 months. Once we setup a crawler, we would expect the site to be crawled for a minimum of 3 months at a frequency of at least one crawl per month.
A: Yes, our standard offering assumes a maximum of 10 fields. More fields can certainly be extracted, but at a higher fee.
A: Invoices get generated during the first week of every month and you will receive it in your registered email address. You can also login to CrawlBoard to view the invoices.
A: There are no additional charges based on frequency. However, volume charges may increase based on the number of records we deliver, which may be directly related to the crawl frequency.
A: Yes, new sites can be added any time. Just that the newly added sites will incur their own setup fee and time for setting up crawlers.
A: Yes! In fact, our offerings are classified based on volumes. We will also be happy to work out attractive discounts in case your monthly data volumes are expected to be in millions.
A: Yes, for customized delivery mechanisms (FTP, Amazon S3, Dropbox or Box), there will be an additional fee of $30 per month. Our default delivery mechanism is via PromptCloud Data API, which is free.
A: Although our model is most suited for recurring data needs, we do entertain one-time requests once in a while if your requirements excite us.
A: As a crawling company we respect the robots.txt file. Popular social media platforms – Facebook and LinkedIn legally disallow crawling and hence, it will not be feasible to extract data from such sites.
A: Add-on services and their pricing are as below:
A: Sample feeds are free and can be subscribed to without a credit card. We crawl a few selected sites/categories in a predefined format and frequency (weekly in most cases). The data is then made available as free sample feeds. There will not be any technical support or customizations for sample feeds.
Got more queries? Reach out to us at contact us