Where have all the Vo-wels gone...long time passing.

January 2, 2006 9:48 PM

Meet Retrievr... Web 1.0 surrenders. Draw a sketch, and instantly, Flickr images that look like your sketch magically appear. It works so well, it's a little disconcerting. Prizes awarded for people who can generate the most inappropriate search results.
posted by Jimbob (51 comments total)

I drew two eyes and a mouth, and got back a picture of a camera lens, looking down the barrel.
posted by Rothko at 9:52 PM on January 2, 2006

"Error in application retrievr
Maximum Thread count reached."

Bummer.
posted by jazon at 9:55 PM on January 2, 2006

what what what? It's not working for me, at all....but I sense great importance here, and will check back... I have great faith in the power of the Internets and her children!
posted by newfers at 10:02 PM on January 2, 2006

jazon, try sketching a cheaper fabric
posted by billder at 10:02 PM on January 2, 2006

Damn, thought we'd overload it. Check back, because it's really quite good. Rothko - I had great success drawing landscapes - beaches, sunsets.
posted by Jimbob at 10:03 PM on January 2, 2006

I've done 2 sketches and see little correlation between what I sketched and what it retrieved.

You know what would really be helpful?

1. A way to save the sketch. That way you could pass it on to people, reinput it, and ask why you are getting the results you are.

2. A gallery of sketch->results mappings. Maybe for some that don't seem mainly random.
posted by MonkeySaltedNuts at 10:17 PM on January 2, 2006

This sounds very cool. Too bad I can't see it.
posted by Justinian at 10:24 PM on January 2, 2006

From the poor results I saw, random images pulled out of flickr would be just as effective as whatever crazy algorithm it is using.

Actually, maybe if it just pulled back 1/3 oceans, 1/3 landscapes, and 1/3 faces it would seem really effective.

I mean, what sorts of things do people really draw in that little box? Sometimes, things are far more interesting as a hoax. Reminds me of that burger king webcam thing.
posted by sucka_mc at 10:25 PM on January 2, 2006

Reading the site, it looks like they're thinking of including a gallery of sorts. It seems to base calculations on the placement of colours in a scene rather than shapes. Drawing a face might not get you very far, because there are few photos of faces that look like a black outline on a white background. But fill in the whole area with patches of colour and it works well. (ie. paint the top half blue, the bottom half green, and put some splotches of white in the blue, and you'll probably get some pictures of grassy hills.)
posted by Jimbob at 10:27 PM on January 2, 2006

Hrm. An impressionist phallus retrieved no fewer than three human faces, with this one being, in my view, the funniest, given the original subject.

More disturbing is this result from the same sketch.
posted by aberrant at 10:29 PM on January 2, 2006

Works fine / incredibly awesomely for me. Go for abstraction and colors, not black and white sketch-pad style trees.
posted by Tlogmer at 10:33 PM on January 2, 2006

At the very least, this is interesting.

Will we someday be able to upload pictures as a search parameter to find similar pictures? Art to find similar art? Is this useful?
posted by ryanhealy at 10:34 PM on January 2, 2006

Here are 2 things that slightly work:

1) top half light blue, bottom half light yellow.
- returns some beach scenes and similar plus some inexplicables

2) left half black, right half red
- return photos which are mainly black and red though they don't seem to obey the left-to-rightness of my sketch.
posted by MonkeySaltedNuts at 10:38 PM on January 2, 2006

Wow -- not to be a reverse naysayer to some of the comments here . . . but dang this thing is amazing. I could waste a ton of time with it just kind of scribbling abstract areas of value and color and seeing what comes up -- the correlations have been fantastically astute but, how should I put it, intriguingly elliptical.

So helpful hint: think simplified, abstract areas of color and value. Mona Lisa, say, would just be a ball of pinkish orange-ish with a cascading black halo atop a black mountain, not a line drawing of the Mona Lisa, etc.
posted by Toecutter at 10:38 PM on January 2, 2006

Seems random to me.
posted by Osmanthus at 10:39 PM on January 2, 2006

An example for the doubters. I drew this - a river surrounded by trees under a cloudy sky. The site returned pictures like this:
1.
2.
3.
And also, strangely, 4.
posted by Jimbob at 10:41 PM on January 2, 2006

On second thought, I think it's the "intriguingly elliptical" part that's putting folks off. I don't think it actually works as a kind of search engine -- say for example, you wanted a picture of a locomotive and tried to draw one. Instead, it will show you pictures that features in common with what you have sketched. Spotting those features is the fun part. So maybe very interesting toy, not so great search tool.
posted by Toecutter at 10:43 PM on January 2, 2006

I tried reverse-testing it: I picked a Flickr image, then painted it as best I could. It didn't match it, even though (I think) I did a pretty good job.

Then I figured "maybe they don't have the entire place catalogued", which seems a fair bet. What I did then was pick one of the odd results it provided, and then painted that image. The first try (a bug on a stalk) didn't come up, but the second one (A guy in a black coat with an overexposed background) came up as the first hit! So, it seldom works as implied in the parent post, but it occassionally does pretty well.
posted by Bugbread at 10:45 PM on January 2, 2006

Great idea. Be even better if it worked.
posted by Citizen Premier at 10:46 PM on January 2, 2006

Wow, this is kickass. I did a quick, crude drawing of a red dot and a black line and got back some pretty impressive results.

It appeared like it wasn't working at first, but I just let it sit there for a minute or so and went back to notice the results were pretty damn good.
posted by mathowie at 10:52 PM on January 2, 2006

alright so it did bring back some eyes. I'll give it that much.
posted by Citizen Premier at 10:56 PM on January 2, 2006

Doing another reverse test, trying to draw this:


...gave me these results:


posted by Bugbread at 11:00 PM on January 2, 2006

Looks like the servers were melted, but wasn't there some news last year about a graphic format(a newer mpeg, or something like that) with an awareness of the appearance of the file, and a search capability could be exploited(pull up picture of a horse, tell it more like this please, and it searchs based on graphic content of files for similar pictures)?

Also, this is a heck of a thing with the "graphics as possible vectors for NotFun". One flickr image of a black and white smiley face with special attributes could get someone a lot of work done in a hurry at this point.

Some see the glass half-full, I ask who the hell left this urine sample.
posted by dglynn at 11:04 PM on January 2, 2006

I thought about making something like this once, only it would've worked more like a LiteBrite in that you change the colours of cells in a small grid to make a pixelated image that would serve as the search term. Of course, I didn't really have a clue about how to implement it, an this is far more complex. Damn.
posted by chrominance at 11:12 PM on January 2, 2006

machines that think -- I tells ya, they're gonna blow our mind this century. Computationally, the PC of the year 2000 wasn't really that different from the VAX of 1980 (it was faster, but only in degree and not qualitatively, other than presenting a pretty GUI the 1Ghz PCs of 2000 didn't do anything different than the minicomputers of 1980), but the PC of 2020 is going to be off-the-wall powerful, compared to the powerful stuff today.

I'm especially interested in automatic translation; if I could go back to grad school that's what I would pursue.
posted by Heywood Mogroot at 11:18 PM on January 2, 2006

Hopefully the face recognition software Homeland Security is using works a bit better than this.
posted by HTuttle at 11:22 PM on January 2, 2006

I got this. And the 'next' image is a little on the nsfwy side.
posted by delmoi at 11:24 PM on January 2, 2006

All my wasted years of practicing drawing Goatse Man on MS Paint has just paid off.
posted by uncanny hengeman at 11:24 PM on January 2, 2006

my crude grade-school-level gynecologically inspired sketch delivered this gem.
posted by quonsar at 11:27 PM on January 2, 2006

It seems like it's basicaly going by color quadrents, the chrominance describes. The implementation is actualy pretty easy. Break the sketch into cells (say 5 or 6) and just look for pictures that match the color.
posted by delmoi at 11:29 PM on January 2, 2006

Great idea. Be even better if it worked.
posted by Citizen Premier at 2:46 AM AST on January 3 [!]

amen
...this would be great with a larger canvas or smaller brush tools...
but, yeah. wonderful idea. and lovely results even when they're way off.
posted by es_de_bah at 11:31 PM on January 2, 2006

Ha ha! My Goatse Man doodle returned this.
posted by uncanny hengeman at 11:33 PM on January 2, 2006

I drew a figure 8. Interestingly, I got this and this.



dglynn, are you talking about eVision's visual search engine write-up? I had to read up on the technology behind image and video search engines sometime last year, and most of what I read lamented the inadequacy of keyword-based searches and pointed to the future of object-based searches. Specifically:

When we look at photographs, we look for patterns and objects. We identify a photograph that is 10% brown and 90% green as a brown horse in a grassy field. So when searching for similar images, we would not be confused by a photograph of a green river dotted with 10% brown fallen tree branches. But general-purpose VSEs could identify a horse in the field and tree branches in a green river as very similar. They look at the image as one big undifferentiated group of RGB values.

An object-based VSE like eVision, tries to first identify the objects in an image before doing a comparative search. While it can't attribute the meaning of horse to the brown object, it can say that the photo is composed of two distinct objects - a brown one with a particular shape and a green background. Then it runs visual comparisons to other images based on these regions.

For example, with the photo of a horse in a green field and a color similarity search, a general-VSE would say "This photo has 90% green in it and 10% brown so find photos that have this same proportion of colors." eVision would say "This photo has two objects in it, 1 object is 100% green and the other is 100% brown so find photos that contain a 100% green object and a 100% brown object." For the non-object way, you would get horses in fields, a forest (trunks are brown), brown scum in a green river, a green lawn covered in 10% dog droppings etc. With eVision you would get horses in fields, a horse-sized dog on a grassy background etc. The latter matches are certainly closer to the sample image and much much more like the way humans see things. We see objects, not distributions of color.

(See also: this Corbis search intro [pdf] and these Wired articles.)
posted by Lush at 11:53 PM on January 2, 2006

I have to say this not a very good way to find pornography.
posted by wfrgms at 12:05 AM on January 3, 2006

For some things it does very well.
posted by Kickstart70 at 12:09 AM on January 3, 2006

Interesting. Basically compositional. Sometimes doesn't work. Hmmm.
posted by furiousthought at 12:27 AM on January 3, 2006

I am in awe. For those complaining of it not working - try drawing something else, or write a better app. My child-like drawing of a christmas tree pulled up an almost identical image along with some semingly unrelated ones. That it even pulled that one similar image at all is amazing to me. Other examples above are even more impressive.
posted by blendor at 12:31 AM on January 3, 2006

I dont think it matters that its line detection is sketchy, this is seriously the most awesome palette selection-refinement tool i've seen.
posted by indigoskynet at 1:01 AM on January 3, 2006

I drew an apple tree and didn't get any back. :(
posted by Navek Rednam at 1:14 AM on January 3, 2006

I wonder if the results would be significantly better if you could rate the appropriateness of the results and it used those ratings as a weighting heuristic. If it keeps track of each sketch query then it could determine similarity of sketch to sketch in the same way as it does sketch to picture.
posted by juv3nal at 1:15 AM on January 3, 2006

Excellent! I don't see any real use for it, but it is a ton of fun.
posted by dg at 4:29 AM on January 3, 2006

Alright, I will not link my original drawing but this thread wouldn't be complete without posting the kitten photos it returned.

How this is even closely related is beyond me. Cool site though.
posted by purephase at 5:15 AM on January 3, 2006

I saw this on Bloglines with the title "Where have all the Vo-wels gone"

Made me laugh. I was thinking this three years ago when I got a job at CentrPort. Horrible company, btw, went out of business.

I guess I'm an Old Guy, unable to see the coolness of misspelling things.

PS: Why do the MF posts have a different title in the syndicated form than they do on the actual site?
posted by dno at 6:54 AM on January 3, 2006

Looks like retrievr's ovrloaded again today. Sounds fascinating on a nice, shiny, diversionary level, though.
posted by Spatch at 7:24 AM on January 3, 2006

I've seen this described as "a great use of the Flickr API". But it's not really, right? They just sucked a bunch of Flickr images down, and then do the processing locally. Then they link the results back to Flickr. Flick is just a convenient source of photos. They could have used AP or Reuters photo feeds instead. Not as buzzworthy though.
posted by smackfu at 8:25 AM on January 3, 2006

i drew a sketch of master shake and got back baby pictures.

web 1.0 still has some time left.
posted by 3.2.3 at 9:47 AM on January 3, 2006

I have to say this not a very good way to find pornography.

Agreed.
posted by 13twelve at 9:50 AM on January 3, 2006

dno, the post title is the same on the feeds and on Mefi. Using Firefox or IE, the post title appears as the window title for the specific thread. It's not the same as the text of the post; that's because the poster specifies a title when he/she posts. Probably not very clear if you've never posted a link.
posted by donnagirl at 10:32 AM on January 3, 2006

Drew a picture of a flower and got a image of a candy heart on a silk sheet that says 'Erect Nipples'...

here.
posted by ba11k at 11:19 AM on January 3, 2006

"Web 1.0 surrenders"? Someone was a farker!

(me too, don't be ashamed)
posted by poorlydrawnplato at 12:11 PM on January 3, 2006

I finally got it working, and it's pretty neat. Especially if you want stock photos that coordinate with your layout.

It would be even more impressive if they showed only photos that were good matches. It probably knows that some of them are better than the others, but it still shows them all and you're not very impressed.
posted by smackfu at 2:14 PM on January 3, 2006