Join 3,377 readers in helping fund MetaFilter (Hide)

2 posts tagged with robotstxt. (View popular tags)
Displaying 1 through 2 of 2. Subscribe:

The digital equivalent of stashing a dirty magazine under your mattress

When news came through of Yahoo! buying Tumblr, everyone wanted to know what that meant for all the porn on Tumblr. But it turns out that long before Yahoo! signed a check, Tumblr had been quietly doing something about it on its own: stopping adult blogs from being indexed. [NSFW links] [more inside]
posted by themadthinker on May 23, 2013 - 152 comments

We Don't Want a Traffic Jam?

You want us to pay you for directing eyeballs to your sites? Newspaper publishers in France want a law whereby Google (and other search engine services) have to pay for each click made from the search engine to their sites. You click on a link to a French newspaper site from a search engine, the Search Engine has to pay the newspaper for that click. If the law is passed it's likely Google will no longer include links to French sites that require payment for said links.
posted by juiceCake on Oct 19, 2012 - 107 comments

Page: 1