Bad Robots

I was recently contacted by a company that asked me to prove if I had a licence for the premium fonts in use on a couple of my websites, which I did, however the whole process was a pain and I was particularly uneasy about the potential hassle if they started emailing all my clients in the same way.

This led me to add a script inside .htaccess file and I was able to reference @brechtryckaert book on this subject to get the formatting right.

Doing some more digging yesterday, I was shocked to uncover the shear number of bad robots out there crawling though our websites that we really don’t want, because they suck up our bandwidth, make the sites slower and much worse. Some are used by services to spy on our work and help the competition, while others run by cybercriminals and fraudsters are actively looking for exploit any security holes.

The point of this post is that while most of us are happy to have our our sites indexed and crawled regularly by the likes of Google or Bing, there are a big number we would do well to close the door on forcefully.

Although Blocs sites are not inherently insecure, I would advise anybody to buy @brechtryckaert book on htaccess if you have not done so already. Do your own research, and give your websites a boost, not just with the performance, but also the security side on several levels.


Totally agree!

There is more to websites than just your visual design.

1 Like

Very nice of you to take time out of your schedule to bring awareness of this issue. Thank you.


Thank you so much for those kind words :star_struck:

Thank you for helping me take my website header security from an F to an A.