dumb crawlers

I wonder who is teaching people that it is acceptable to download the entire content of a website, as fast as possible, and then come back and do it again the next day (and the next)…

larbin has been added to the list of blocked robots. Buh-bye!

posted at 9:19 am on Tuesday, September 28, 2004 in Site News | Comments Off on dumb crawlers

No Comments

No comments yet.

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.