Marginalia Search - Serendipity Engineering

March 9, 2022 4:44 PM

Marginalia Search is "an independent DIY search engine that focuses on non-commercial content, and attempts to show you sites you perhaps weren't aware of." Clive Thompson describes Marginalia (also known as Edge Crawler and Astrolabe) as "a search engine with a fascinating design — rather than give you exactly what you’re looking for, it tries to surprise you... By up-ranking web sites that are text-heavy, and downranking ones that are highly visual, loaded with modern web cruft, and SEO-optimized. The upshot, as the creator suggests, is that you wind up with a lot of weird results very different from the usual fare coughed up by Google or Bing or even DuckDuckGo.... Call it 'serendipity engineering.'"

Hacker News: A search engine that favors text-heavy sites and punishes modern web design

From About search.marginalia.nu:

Remember when used to explore the Internet, when you used to discover cool little websites made by people and it wasn't just a bunch of low effort content mill listicles and blog spam?

I want to show you that that Internet you used to go exploring is still very much there. There are still tons of small personal websites, and a wealth of long form text from both the past and the present.

So it's a search engine. It's perhaps not the greatest at finding what you already knew was there, instead it is designed to help you find some things you didn't even know you were looking for.

If you are looking for facts you can trust, this is almost certainly the wrong tool. If you are looking for serendipity, you're on the right track. When was the last time you just stumbled onto something interesting, by the way?

...

This is in a sense the opposite of what most major search engines do, they favor modern websites over old-looking ones. Most links you find here will be nearly impossible to find on a regular search engine, as they aren't sufficiently search engine optimized.

This may seem a choice from some sort of nostalgia, which in part is true, but there is more to it. The hypothesis is something akin to the Lindy-effect: If a webpage has been around for a long time, then odds are it has fundamental redeeming quality that has motivated keeping it around all for that time. Looking at design elements is one way of determining the approximate age of a webpage, and thus predict its usefulness.

...

There's also a point to be made about how the technologies we use shape our view of the world. I'm hoping, by holding up this kaleidoscope to the Internet, you'll become more aware of the spectacles you were already wearing without thinking about them. It really does seem like a different reality.

If this were the looking glass through which you viewed the the Internet, and I'm not making a serious suggestion that it should be, but if it were, then wouldn't you agree that your idea of the web would by seem be very different, more human?

Here's a list of other similar projects.

Related article recommended by Marginalia's creator: Against an Increasingly User-Hostile Web: We're quietly replacing an open web that connects and empowers with one that restricts and commoditizes people. We need to stop it.

Related post by MonkeyToes: Rewild Your Attention: "If you want to have wilder, curiouser thoughts, you have to avoid the industrial monocropping of big-tech feeds. You want an intellectual forest, overgrown with mushrooms and towering weeds and a massive dead log where a family of raccoons has taken up residence." Clive Thompson's "Rewilding Your Attention" (Medium) is a brief reflection on the value of seeking out idiosyncratic content. Time to go for a random walk in the woods.

Vaguely related article on the Lindy-effect: The Lindy Way of Living: A technology lawyer named Paul Skallas argues we should be gleaning more wisdom from antiquity.

Unrelated article on Serendipity Engineering — Enabling Innovation through Happy Accidents

posted by homunculus (27 comments total)

A meandering path toward and through the internet I love, miss, and treasure still stumbling upon. So very excited to play with this tool.
posted by youarenothere at 5:20 PM on March 9, 2022 [3 favorites]

Here's Marginalia on Patreon, if anyone is interested: "I operate an independent search engine out of my living room. It's designed to help you navigate the sort of websites you can't otherwise find on the big web."
posted by homunculus at 5:42 PM on March 9, 2022 [2 favorites]

A few days ago someone pointed me to wiby.me, a search engine that punishes complex web design. It's also potentially of use!
posted by JHarris at 6:20 PM on March 9, 2022 [5 favorites]

It’s working for me. I searched for Ted Chiang, an author I discovered recently on the Blue and one of whose books (Exhalation) I am currently going gaga for.

Marginalia

Google

Way more interesting links on the former search. Thanks!
posted by Turtles all the way down at 6:58 PM on March 9, 2022 [1 favorite]

The Most Useful Rope Knots for the Average Person to Know (via wiby.me's "surprise me...", thanks JHarris!)
posted by chinesefood at 7:06 PM on March 9, 2022 [2 favorites]

This is fantastic, I started searching for information on mango trees and have now ended up on a random fascinating blog I'd never find otherwise.
posted by coolname at 7:10 PM on March 9, 2022

I like how it shows what flavor of HTML the page is coded in. Gives you a rough idea of how old the page might be, or whether it was produced by hand or a CMS.
posted by adamrice at 7:13 PM on March 9, 2022 [1 favorite]

Oops, I screwed up the Marginalia Patreon link above (too many tabs.) Here's the right one: Marginalia is creating an alternative search engine for off-the-beaten-path websites


[derail] What I actually linked is a fun fantasy web serial called "Liches Get Stitches" (aka "Lich, Please") a story about a village witch who gets accidentally turned into a lich. It's a fun (though gory) story so check it out if that sort of thing appeals to you. You can read it for free on Royal Road but the patreon is 4 chapters ahead.

Reborn as a powerful lich, Maud is suddenly faced with the attentions of all the righteous heroes, holy clerics, and nosy neighbours of the realm. Now instead of whiling away the days in her garden with her cat and her knitting, Maud must figure out how much force is required to crush a man's spine, the proper storage solution to keep a spoiling cadaver, and how best to display the remains of the fallen for maximum scare.

Featuring people mulch, head bouquets, revenant geese, and some very deadly embroidery, undead paradise never looked so good. [/derail]

posted by homunculus at 7:38 PM on March 9, 2022 [2 favorites]

golden fantastic homunculus.
posted by clavdivs at 7:40 PM on March 9, 2022 [1 favorite]

> A few days ago someone pointed me to wiby.me, a search engine that punishes complex web design. It's also potentially of use!

Cool! wiby.me is one of the other search engines listed under Similar Projects, along with 5 others. "Competition is perhaps not the word, we're all pulling in the same direction, only with subtly different goals and approaches."
posted by homunculus at 7:45 PM on March 9, 2022 [1 favorite]

Thank you.
posted by chance at 8:24 PM on March 9, 2022 [2 favorites]

I love the idea of pulling text-heavy sites rather than graphics and pictures. Take that, post-literate culture!
posted by scratch at 5:44 AM on March 10, 2022 [3 favorites]

What I'd really love is a way to find old-style unmonetized personal blogs that still update
posted by nebulawindphone at 6:16 AM on March 10, 2022 [9 favorites]

Basically, I want Google's forums search back.
posted by tigrrrlily at 6:46 AM on March 10, 2022 [1 favorite]

This is so cool!!!
posted by acantha at 7:14 AM on March 10, 2022

I'm a little disappointed my website isn't in their index. It's been online at the same domain since 1998, and it's still updated several times a month with new blog posts, and it's mostly text.
posted by COD at 7:18 AM on March 10, 2022 [2 favorites]

Seems like fun but I broke it immediately; maybe it's not for all queries. I found that the phrase "men's jackets that" returns zero results and the message:

"men's" could be spelled "menes", "mends", "menus" Try rephrasing the query, changing the word order or using synonyms to get different results. Tips." )

but "men's that jackets" returns a bunch of links about jackets.
(looking for how to make a men's jacket that look like trees. it's all I could think of to try)
posted by achrise at 8:04 AM on March 10, 2022

This thing is awesome, in my detailed sample size of exactly one search. I picked “notch filter tuning”, and immediately fell headfirst into the third result and read the entire thing straight through. I have missed that.
posted by bigbigdog at 9:04 AM on March 10, 2022

Oh my God, what a treasure! I've just learned that by mixing dried manure with clay one can get self-firing ceramics. That was not my search term. Instant bookmark!
posted by dbx at 10:22 AM on March 10, 2022 [1 favorite]

I picked a kind of obscure topic I’ve been reading about lately and the second result was a very long musing about a somewhat-related topic from 2001…which happened to include a link to a website with a still-quite-active listserv on the exact topic! Certainly did not find that on google when I searched for the same thing.
posted by acantha at 10:25 AM on March 10, 2022 [2 favorites]

What I'd really love is a way to find old-style unmonetized personal blogs that still update

LOL. I still do those, but nobody cares because it's not on social media. Which is probably all for the best because god knows if anyone sees you and something gets popular, they go crazy and try to cancel/dox you.
posted by jenfullmoon at 11:51 AM on March 10, 2022 [1 favorite]

digby, atrios, and kos are blog-a-riffic to this day. if you're into that sorta politics blog. add, but mostly unobtrusive.
posted by j_curiouser at 12:27 PM on March 10, 2022 [2 favorites]

The first result for my first attempted search (khaen) is fantastic. Perhaps I just got lucky. But, it's sure promising.
posted by eotvos at 1:46 PM on March 10, 2022

What I'd really love is a way to find old-style unmonetized personal blogs that still update

Checking the profiles of people favoriting your comment seems to work...

Great idea for a search engine. I entered a placename I'd blogged about in the past and one of my posts came up, which hasn't happened on Google in years.
posted by rory at 1:55 PM on March 10, 2022 [5 favorites]

I'm glad so many people are enjoying this. I'm curious, though: did anyone like the last 2 links in the op which were only tangentially related to the main links? I enjoyed the lindy-effect one but that's a topic which probably deserves its own fpp, and I only included the last one in case some was searching for "serendipity engineering" since that piece reflects more how the term is usually used.
posted by homunculus at 7:21 PM on March 10, 2022

Searching "Metafilter" brought up this... interesting blog post from 2001.
posted by cozenedindigo at 12:12 AM on March 11, 2022 [1 favorite]

Marginalia is featured in new The New Yorker story:

What Google Search Isn’t Showing You: The search engine has made up so much of our online experience for so long that it can be hard to imagine something better.
posted by homunculus at 10:41 AM on March 13, 2022 [3 favorites]