How to block pingdom bots in nginx?

Question

Considering that Pingdom is a high reputable site, I tried to subscribe for their uptime monitoring service. However, even though I setup for 5 minutes interval, their bot Pingdom.com_bot_version_1.4 keeps accessing my site not once every second, but tens of times every second! Resulting thousands of access per minute!

Then I tried to completely cancel the service but still get bombarded by their bot. I tried to block in robots.txt, but apparently they chokes it on. Next, I tried to block in nginx.conf with this command:

if ($http_user_agent ~* Pingdom.com_bot) {
        return 403;
}

It works, but I see a lot of 503 errors in access.log. How to not log this bot? Really really annoying. I regret ever subscribed to their service.

Answer 1

Here is a post about blocking w00tw00t which you could easily adopt.

The easiest option for you to adapt would proablably beeing the fail2ban one.

using a fail regex triggering on your 403 error.

So something like

[Definition]
failregex =  ^<HOST> .* "(GET|POST|HEAD).*HTTP.*" 403 [0-9]{1,} ".+" ".+"$
ignoreregex=

in /etc/fail2ban/filter.d/nginx-pindotban.conf

and

[pingdotban]
enabled = true
port = http,https
filter = nginx-pingdotban
logpath = /path/to/nginx/access.log
maxretry = 5
bantime = 360000

in /etc/fail2ban/jail.conf

You can test the regex with

fail2ban-regex logfile /etc/fail2ban/filter.d/nging-pingdotban.conf

Also the iptable variant could be adopted with something like

iptables -A INPUT -p tcp --dport 80 -m string --algo bm --string "the useragent" -j DROP