Faster setup with sitemap

To save time and streamline the setup process, it would be valuable if the software could automatically detect sitemaps in the robots.txt file or allow users to manually add a sitemap URL. This would reduce the time spent manually configuring crawls and ensure faster setup.

Requested features:

1. Auto-detect and add sitemaps listed in robots.txt.

2. Option for manual sitemap URL submission.

Implementing these features would optimize page discovery and significantly reduce setup time.

Thank you for considering this!

1 Голоса