WPMU Global Robots.txt

WPMU Plugin: WP Global Robots.txt

robot.txt get more important each day in my opinion.  If you take a look at what some of these bots are looking for it is a total waste of server resources.

I’ve had some of these repeatidly look for pages which were on a system we removed well over a year ago – I mean, come on a YEAR AGO.  And if you don’t find the file…please – stop looking.  Bah, whatever – you are blocked!  :)