I thought Grub was a bootloader?
April 23, 2003 10:32 AM Subscribe
Is Grub out of control? Barely more than a week old
, the distributed search engine is already causing headaches. It does not properly follow the Robot Exclusion Standard
and thus spiders sites against their owners' wishes. Because it is a distributed client run by thousands of volunteers (and therefore connects from many different IP addresses), it is non-trivial to block. The Wikipedia project, for example, is experiencing slowdowns
because of it. Let's hope they can solve these problems, as the idea seems to be quite cool.
posted by Eloquence (7 comments total)