There are use cases when you don’t want to crawl all data from a source and store, but instead crawl live, as and when your user enters a search term on your site or app. So for instance, if you are a re-commerce store, selling used items on your site, you’d like the crawl to occur in real-time when your user searches your website using an EAN.
How this works?
This works similar to our site-specific crawl offering with the only difference that the desired frequency of crawl here is real-time. The crawlers are setup in advance to extract information from your list of sources in your desired schema. We then provide you an API that you can hit with search terms that your users expect data for, based on a predefined input format. Our platform then internally crawls the results, extracts data and returns structured data via the same API.
