Comments on: Bots, spiders and crawlers, oh my!

Bots, spiders and crawlers, oh my!

sluglicker — Thu, 16 Mar 2006 07:08:08 -0800

"Imagine this: a digital butler that roams the Internet, intuitively knowing your likes and dislikes, retrieving perfect strands of news and information that you never would have discovered through old-fashioned surfing." There's RumorBot and the Iliad fetchbot (perfect for bot newbies), or you can try your hand at writing your own in php using this as a tutorial, or if you prefer, C++ or Perl.

By: Yer-Ol-Pal

Yer-Ol-Pal — Thu, 16 Mar 2006 07:14:33 -0800

Ithought this was the purpose of Metafilter?

By: Faint of Butt

Faint of Butt — Thu, 16 Mar 2006 07:17:26 -0800

Y-O-P beat me to it. All of you guys are my content-aggregation bots.

By: sluglicker

sluglicker — Thu, 16 Mar 2006 07:23:22 -0800

@Yer-Ol-Pal Sure, if you only want to leech. If you want to contribute, try a bot! Correction: the Perl link should be Perl.

By: Brian James

Brian James — Thu, 16 Mar 2006 07:44:15 -0800

Haven't personalized content-aggregation tools such as this one been the Next Big Thing for the last, like, 15 years?

By: mekanic

mekanic — Thu, 16 Mar 2006 07:46:59 -0800

Speaking as a php, c++, etc. illiterate, it seems that this page might be a little more helpful as a tutorial for building your own bot. It has lots of other php beginner tutorials as well.

By: mikepop

mikepop — Thu, 16 Mar 2006 07:55:06 -0800

There is also The Ultra Gleeper

By: killdevil

killdevil — Thu, 16 Mar 2006 08:21:32 -0800

I begin to imagine a digital butler, thinking back to 1997 or so when such concepts were aired regularly, but then I start getting angry.

By: dammitjim

dammitjim — Thu, 16 Mar 2006 08:26:48 -0800

The main link, the one that the quote is taken from, reads as though it was written in 1996, in a breathless issue of Wired ("Push! The next big thing!"). There's a reason that nothing has really happened with this idea: nobody knows quite how to do it yet. At least, nobody knows what to build that will be:

broadly useful
simple to instruct and direct
more than just a script that periodically performs searches that the user tells it to

How do you make this "butler" universally available to the user? From where does it execute its actions? On the user's home computer? That's not very useful, not unless the user is home all the time. On the user's phone service, and talking to them through their mobile? This idea is not ready for prime time yet. Somebody will make a lot of money if it does come true, though.

By: ernie

ernie — Thu, 16 Mar 2006 08:45:01 -0800

Just ask Jeeves

By: Ynoxas

Ynoxas — Thu, 16 Mar 2006 09:46:05 -0800

Didn't I have something just like this back in like 1997 or 1998? And didn't it completely NOT work? And wasn't it pretty well decided that web viewing is an ACTIVE activity, not a passive one? Am I the only one that thinks push is unnecessary? I just fail to see much difference between having a story from CNN "pushed" to me so I click the link to view it, or viewing the link directly because I navigated to CNN.com. Agents have been highly overrated for a very long time.

By: Dr. Twist

Dr. Twist — Thu, 16 Mar 2006 12:05:55 -0800

does anyone else remember knowledge navigator?

By: Artw

Artw — Thu, 16 Mar 2006 13:33:38 -0800

Imagine a digital paperclip, appearing as you compose a document to give you help and advice...

By: reklaw

reklaw — Thu, 16 Mar 2006 14:27:18 -0800

ernie: Jeeves is gone.

By: Pinback

Pinback — Fri, 17 Mar 2006 00:17:30 -0800

Haven't personalized content-aggregation tools such as this one been the Next Big Thing for the last, like, 15 years? Yes. And, if it ever comes to pass, it'll be like all those people who listen religiously to Alan Jones, John Laws, or Bill O'Reilly - a self-selected audience who only see what they want to see, who never have their beliefs challenged or opinions confronted (except to foment outrage), and who believe everything they're told. Imagine a million individual LGFs - left wing, right wing, and every shade in between - consisting of one person plus their "intelligent agent" / "digital butler". Just as the porn industry can provide for your every fantasy no matter how twisted, so will the content industry come to supply information/opinion tailor-made for every prejudice and belief. The only winners will be those who provide this content. And those who choose not to play.

By: sluglicker

sluglicker — Fri, 17 Mar 2006 08:48:30 -0800

With the exception of dammitjim, did any of you actually read the linked texts? Your comments have nothing to do with the content of this post. And Pinback: WTF are you talking about? Bah!...Perl before swine.