The thing is that Google generally does not like automatic crawling. Therefore, at some point, all requests are blocked, and you start receiving Internal browser error for Google in the application interface. In such situations, we recommend:
1. Use User Agent - "Chrome".
2. Set the delay between requests → screenshot. You need to test several delay options until the blocking disappears.
3. Reduce the number of scan threads - then the scan will be a little slower, but the load on Google will be reduced, and the chance to scan more requests before Google puts a proxy on the captcha will increase.
4. If the previous three steps did not help, then the only option is to use a list of private proxies in combination with one of the services for automatic captcha solving.
Thus, you will be able to analyze the maximum number of pages and avoid being blocked by the search engine.
Was this article helpful?
That’s Great!
Thank you for your feedback
Sorry! We couldn't be helpful
Thank you for your feedback
Feedback sent
We appreciate your effort and will try to fix the article