How to whitelist Netpeak Spider's user agent in Cloudflare

Modified on Tue, 16 Jul at 2:18 PM

Cloudflare typically blocks site crawlers.


However, you can whitelist the Netpeak Spider user agent in the Cloudflare settings.


Here are the steps to do so:


1. Set custom user agent at Netpeak Spider 




2. Log in to the Cloudflare account, select your website.


3. Go the Security tab from the left menu and select WAF.  As a result it shows the Web Application Firewall security settings.


4. Click on Create Rule.


5. Give a name to the Firewall rule.


6. Set Field to User Agent, Equals as the Operator, and Value to the name from the Netpeak Spider settings.




7. Set action to Skip All remaining custom rules.




8. Finally, click Deploy in the bottom-right corner after completing the form.

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article