I had to ban the Exabot from our servers because it just loads pages constantly and indiscriminately using up needed resources better utilized by actual users.
In your .htaccess you can add the following to do the job:
CODE
order allow,deny deny from 193.47.80.* allow from all
I do not use robot text files because sometimes these bots / spiders tend to ignore it completely.
This site uses Cookies to dispense or record information with regards to your visit. By continuing to use this site you agree to the terms outlined in ourCookies used here:Privacy / Disclaimer,