Submit Your Requirement

Download Web Data Acquisition Framework

Did you know that there are 12 factors to be considered while acquiring data from the web? If no, fret not! Download our free guide on web data acquisition to get started!

Scroll down to discover

Feature update: Now use Webhooks to Receive Crawl Event Updates and Automate your Data Processes – PromptCloud

December 10, 2018Category : Blog

We’ve been constantly working on accessibility improvements on our crawling services and the latest addition to the list of such platform upgrades is webhooks. While CrawlBoard already has alerting mechanisms in place to send out email notifications when specific events pertaining to your project happens, we wanted to take this a step ahead with Webhooks.

We understand how automation is the key when it comes to projects involving large amounts of data and the addition of webhooks will enable this enhanced-level of automation for our users.

Using webhooks on CrawlBoard

Using webhooks is simple and straightforward.

  1. Just click on the ‘Data Download’ option from the left panel and select ‘Webhooks’.
  2. In the next page, enter the Target URL where you wish to receive webhooks event updates.
  3. Now select your desired event from the dropdown menu and hit ‘Submit’.
  4. Once done, you can add another webhook event if required by following the same steps.

At present, the available webhooks events are: crawl_started, crawl_completed, upload_completedHooks are basically post requests made from to the target_url specified by the user when they subscribe to a hook. You can find the schema and other information on webhooks by selecting the ‘API details’ tab in the webhooks setup page.

Automation with Webhooks

The main advantage to using webhooks is the automation capabilities that comes with it. With webhooks set up, you no longer need to wait for crawl alerts and manually take action depending on various crawl events. For example, you can set up your system to automatically perform an indexing process on the data once it receives the upload_completed event hook. This way, you can further save time and cut down on manual intervention which is basically what automated web crawling is all about.

We believe the implementation of webhooks could prove to be extremely beneficial to our users and enhance the value proposition of our solution one step further. We’re excited about the release of this product upgrade and would love to hear your feedback and suggestions.

Happy data crunching!


Web Scraping Service CTA

Leave a Reply

Your email address will not be published. Required fields are marked *

Generic selectors
Exact matches only
Search in title
Search in content
Filter by Categories
eCommerce and Retail
Real Estate
Research and Consulting
Web Scraping

Get The Latest Updates

© Promptcloud 2009-2020 / All rights reserved.
To top