Contact information

PromptCloud Inc, 16192 Coastal Highway, Lewes De 19958, Delaware USA 19958

We are available 24/ 7. Call Now.
Web Scraping Techniques

We’ve been constantly working on accessibility improvements on our crawling services and the latest addition to the list of such platform upgrades is webhooks. While CrawlBoard already has alerting mechanisms in place to send out email notifications when specific events pertaining to your project happens, we wanted to take this a step ahead with Webhooks.

We understand how automation is the key when it comes to projects involving large amounts of data and the addition of webhooks will enable this enhanced-level of automation for our users.

Using webhooks on CrawlBoard

Using webhooks is simple and straightforward.

  1. Just click on the ‘Data Download’ option from the left panel and select ‘Webhooks’.
  2. In the next page, enter the Target URL where you wish to receive webhooks event updates.
  3. Now select your desired event from the dropdown menu and hit ‘Submit’.
  4. Once done, you can add another webhook event if required by following the same steps.

Dashboard of Promptcloud's Webhooks to Receive Crawl Event Updates and Automate Data Processes

At present, the available webhooks events are: crawl_started, crawl_completed, upload_completedHooks are basically post requests made from to the target_url specified by the user when they subscribe to a hook. You can find the schema and other information on webhooks by selecting the ‘API details’ tab in the webhooks setup page.

Automation with Webhooks

The main advantage to using webhooks is the automation capabilities that comes with it. With webhooks set up, you no longer need to wait for crawl alerts and manually take action depending on various crawl events. For example, you can set up your system to automatically perform an indexing process on the data once it receives the upload_completed event hook. This way, you can further save time and cut down on manual intervention which is basically what automated web crawling is all about.

We believe the implementation of webhooks could prove to be extremely beneficial to our users and enhance the value proposition of our solution one step further. We’re excited about the release of this product upgrade and would love to hear your feedback and suggestions.

Happy data crunching!


Sharing is caring!

Are you looking for a custom data extraction service?

Contact Us