Submit Your Requirement
Scroll down to discover

Q. Can you crawl sites that disallow bots?

April 13, 2020

No. We respect the robots.txt and crawl a site only if bots are allowed in robots.txt file.

Get The Latest Updates

© Promptcloud 2009-2020 / All rights reserved.
To top