[Referrer-Domain] Microsoft Search Engine Spiders are blocked!

[HKPhysicist]

My sites have joined MS Clarity and MS search spiders began to crawl my sites frequently. nginxrepeatoffender began to jail their IPs.
Their general domain names are:
msnbot-xxx-xxx-xxx-xxx.search.msn.com

How do I whitelist them? In where xxx-xxx-xxx-xxx is a general IPv4 IP address.

[mitchellkrogza]

Please post some log line examples

[HKPhysicist]

Hello,
Here are some IPs from .search.msn.com which I saw today on fail2ban log files. They are not the same every time.

2024-03-30 21:48:43,595 fail2ban.filter [743]: INFO [nginxrepeatoffender] Found 40.77.167.41 - 2024-03-30 21:48:43
2024-03-30 21:53:24,688 fail2ban.filter [743]: INFO [nginxrepeatoffender] Found 52.167.144.20 - 2024-03-30 21:53:24
2024-03-30 23:12:03,360 fail2ban.filter [743]: INFO [nginxrepeatoffender] Found 52.167.144.20 - 2024-03-30 23:12:02
2024-03-30 23:12:51,766 fail2ban.filter [743]: INFO [nginxrepeatoffender] Found 52.167.144.20 - 2024-03-30 23:12:51
2024-03-30 23:12:52,299 fail2ban.actions [743]: NOTICE [nginxrepeatoffender] Ban 52.167.144.20