Aaaaaand we'll just remove a nice, happy tree from riiiiight......abouuuuuut....there. That's nice.

March 26, 2010 11:50 AM

We've already seen seam carving for content-aware image resizing. Now, here's content-aware fill.
posted by lazaruslong (73 comments total)

That was very impressive, but can it make me a better photographer with my $99 special?
posted by jsavimbi at 11:59 AM on March 26, 2010

You can't believe your eyes. You never really could, but now you really can't. Give us what, 5 years before we have sunglasses that automagically remove things you don't want to see? We already have robots on Mars picking out objects of interest by themselves.
posted by jeffkramer at 12:00 PM on March 26, 2010

Note to self: Roll 401K over into "quantity of horrible, naked things emerging from 4chan" fund before this feature drops.
posted by griphus at 12:01 PM on March 26, 2010 [3 favorites]

I still don't understand the processing on the last picture. It seems to indicate that Photoshop is content-aware enough to add a bird in the top left corner.
posted by shakespeherian at 12:01 PM on March 26, 2010 [1 favorite]

I've seen this link floating around the blogotubes over the last few days and have been thinking, "Didn't Photoshop get content-aware mumblesomething over a year ago?" I guess content-aware fill works on a lot of the same principles as content-aware resizing, working in two dimensions instead of one, but seeing it in action still breaks my brain. Think of the evil I could do with this! Mwahahah...err, sorry.
posted by The Lurkers Support Me in Email at 12:04 PM on March 26, 2010 [1 favorite]

The last one seems way fake to me. What is the deal?
posted by wemayfreeze at 12:07 PM on March 26, 2010

So if you start with a blank canvas and do a content aware delete, does PS generate the photo you should have taken?
posted by CheeseDigestsAll at 12:08 PM on March 26, 2010 [25 favorites]

I saw this yesterday - I could not stop watching the little trees disappear.
posted by betweenthebars at 12:11 PM on March 26, 2010

Badass. I might have to go beg my boss for an upgrade.
posted by echo target at 12:12 PM on March 26, 2010

That's not a bird, it's an artifact of where the cursor was...
posted by Vamier at 12:14 PM on March 26, 2010 [2 favorites]

That's no moon.
posted by swift at 12:15 PM on March 26, 2010 [3 favorites]

That's pretty awesome.

That being said, it's pretty funny that he refers to the status bar crawling across the screen in the last one as "racing" and then reminds us of how long it would take without the feature.
posted by PhoBWanKenobi at 12:15 PM on March 26, 2010 [1 favorite]

So if you start with a blank canvas and do a content aware delete, does PS generate the photo you should have taken?

Content-aware fill is going to become a Zen teaching tool.

"Without thinking of copy or paste, content-aware-fill your original face before your mother and father are photographed."
posted by griphus at 12:15 PM on March 26, 2010 [5 favorites]

I want to know how Photoshop decided to put a bird in the last picture.
The "bird" is clearly the lasso mouse pointer. You can watch him select with it, and it even turns into the mac "wait" pointer while it is churning.
posted by Threeway Handshake at 12:16 PM on March 26, 2010 [2 favorites]

If a tree is content aware deleted in a forest and no one is around, did it ever exist?
posted by m@f at 12:19 PM on March 26, 2010 [2 favorites]

tiny little photoshop gnomes, infesting your computer. that's the only way it's possible. the reason adobe products are so expensive is because they have to put all those gnomes through art school.
posted by Jon_Evil at 12:20 PM on March 26, 2010 [8 favorites]

If you had a photo of just the top half of Spock's face and did a content-aware-fill of the rest...

Seriously, though... I am very curious to see how this would handle an exquisite corpse.
posted by m@f at 12:23 PM on March 26, 2010 [2 favorites]

For what its worth from a random internet dude: I work in computer vision aka image processing. This technology (image inpainting) is something the community has been working on for more than a decade. These results do not seem fake to me, as I have seen algorithms give results of this caliber plenty of times before. What does impress me is they trust the results enough to unleash it in the wild. I imagine there will be many cases where it leaves artifacts, but it seems they are encouraging it as a first-pass tool. Use the content-aware fill, and then touch up that result, because that is still faster than starting from scratch.
posted by LoopyG at 12:30 PM on March 26, 2010 [4 favorites]

So, okay, there is going to be a reason to get 5. I've been muttering for years that there's nothing wrong with 3 that 4 solves and grumble, bitch, I can't afford it, complain - but if this really works it's pretty damn amazing.
posted by mygothlaundry at 12:33 PM on March 26, 2010

The "bird" is clearly the lasso mouse pointer. You can watch him select with it, and it even turns into the mac "wait" pointer while it is churning.

Yeah, you're right, it is the cursor. Which actually makes me feel better, because I couldn't imagine creating the first part of the video (with the park photo) as a hoax-- it would be far too laborious and pointless. The last part didn't make any sense being a fake, so I'm glad to know I was wrong.
posted by shakespeherian at 12:36 PM on March 26, 2010 [1 favorite]

Adobe's presentation at SIGGRAPH (August 2009).

And the paper describing how itworks.
posted by Kabanos at 12:39 PM on March 26, 2010 [1 favorite]

argh. Need content aware MeFi comments to fix those typos!
posted by Kabanos at 12:43 PM on March 26, 2010 [1 favorite]

I bet this is extra-super-amazing when you're baked.

Kind of like most things. But still.
posted by uncleozzy at 12:48 PM on March 26, 2010 [1 favorite]

For what its worth from a random internet dude: I work in computer vision aka image processing. This technology (image inpainting) is something the community has been working on for more than a decade. These results do not seem fake to me, as I have seen algorithms give results of this caliber plenty of times before.

Yeah ditto the not fake. I've done a shoddy version of the thing that goes on at about 3:30 with the dirt road as an assignment in an undergrad class using matlab and that was 4(?) years ago. I don't remember much of how to do any of it, but it was a neat class.
posted by juv3nal at 12:49 PM on March 26, 2010 [1 favorite]

Here's Adobe's page on "PatchMatch" which provides a slightly nerdier video explanation of what's going on and might hint at another capability we have yet to see.
posted by pokermonk at 12:52 PM on March 26, 2010 [1 favorite]

I still think it's an early present for next Wednesday.
posted by cashman at 12:53 PM on March 26, 2010

Or, next Thursday.
posted by cashman at 12:53 PM on March 26, 2010

There are a lot of interesting things being done in machine vision and soforth. It kind of pisses me off that adobe acts like they came up with all of this totally on their own. Obviously integrating it into a product and making it useful for people is a big step forward. But this is hardly their work alone.

Anyway, anyone else notice the weird sky on the 'expansion'? It seemed like there was this blue artifact where the seam would have been on the left. Also, I think on some of the content aware fills it looked like there was a weird seam around where the edge was. Still, no doubt this would save you a TON of time. just cleaning up the glitches instead of doing all the work yourself.

But I don't think this would look as good if it wasn't packed down to a tiny, blurry youtube video. Certainly, they could have uploaded a higher-resolution version if they'd wanted too.
posted by delmoi at 12:58 PM on March 26, 2010

Er, I should say "It kind of pisses me off that adobe acts like they came up with all of this totally on their own in this video"
posted by delmoi at 12:59 PM on March 26, 2010

But really, getting it into Photoshop or some other consumer product is all that matters.
posted by smackfu at 1:14 PM on March 26, 2010

This is mindblowing stuff. It's mindblowing that it took Adobe so long to copy this feature from GIMP.
posted by East Manitoba Regional Junior Kabaddi Champion '94 at 1:16 PM on March 26, 2010 [15 favorites]

This will be perfect for newspapers and governments.
posted by Blazecock Pileon at 1:18 PM on March 26, 2010 [3 favorites]

Gimp plugin here: http://www.logarithmic.net/pfh/resynthesizer
If you're running Ubuntu, "sudo apt-get install gimp-resynthesizer".
posted by fings at 1:34 PM on March 26, 2010 [3 favorites]

My mouth fell open as the sky appeared, and hours and hours of my life flashed before my eyes.
posted by emeiji at 1:39 PM on March 26, 2010 [3 favorites]

Wait wait wait...someone using Photoshop to remove lens flares?
posted by Legomancer at 1:43 PM on March 26, 2010 [11 favorites]

Apparently there's been a GIMP plugin to do this for a while now.
posted by GuyZero at 1:45 PM on March 26, 2010

True, but nobody can figure out how to use it in GIMP.
posted by LordSludge at 1:52 PM on March 26, 2010 [4 favorites]

via?
posted by cjorgensen at 1:59 PM on March 26, 2010

This is mindblowing stuff. It's mindblowing that it took Adobe so long to copy this feature from GIMP.

Ha. That says more about GIMP than about Adobe.
posted by smackfu at 2:10 PM on March 26, 2010 [1 favorite]

It used to take the Soviet Union days to do this.
posted by twoleftfeet at 2:10 PM on March 26, 2010

Arrgh, damn it twoleftfeet you just beat me to it.
posted by XMLicious at 2:17 PM on March 26, 2010

So, okay, there is going to be a reason to get 5.

I'm curious if this will actually show up in CS5; they've been pretty careful to not mention a version number in relation to it. On youtube they describe it as "what could happen in the future," and when John Nack blogged about it recently, he says only "a future version of Photoshop".
posted by ook at 2:24 PM on March 26, 2010

This is mindblowing stuff. It's mindblowing that it took Adobe so long to copy this feature from GIMP.

CMYK CMYK CMYK I CAN'T HEAR YOU
posted by DU at 2:32 PM on March 26, 2010

So.. it's been a while since I've been in the Photoshop game, when exactly did Adobe start trafficking in magic, witchcraft, and other dark arts?

Because *points at screen* that's some sorcery shit I'm seeing right there.
posted by quin at 2:43 PM on March 26, 2010 [1 favorite]

I use the GIMP and honestly, it's not that bad once you get the hang of it. I was drooling over this video though, until I learned I could do the same thing with free software. I'm definitely going to give Resynthesizer a try.

Anybody know if content-aware resizing is also available for the GIMP? I so want to get my grubby hands all over that crap.
posted by The Winsome Parker Lewis at 2:47 PM on March 26, 2010

So if you start with a blank canvas and do a content aware delete, does PS generate the photo you should have taken?

My friend had this brilliant (silly) idea; he wants to write a program that takes a photograph sized array of pixels and have them as quickly as possible cycle through all the possible color combinations for each pixel and then save any interesting results. The upshot of this, is that if you had the storage space, eventually, you would end up with every photograph ever. You'd also have every frame from every movie, including the ones that have never been made. You'd have photos of the pyramids being built, and then photos of them being destroyed by the robot monsters that attacked during the dark ages. Same with the ones of Jesus and Batman fighting off zombie serial molester plague of 2125.

His idea is that any photos that don't already fall under someone else's copyright, would belong to him. And that would mean that he would hold the every photo into the future that hadn't already been taken. (I believe that he ended this last sentence with a maniacal, evil genius styled laugh).

All I want is a couple of the more dirty ones so that I can get rich off of hypothetical blackmail for events that haven't actually happened yet.
posted by quin at 3:09 PM on March 26, 2010 [7 favorites]

...and a roomful of monkeys with typewriters will eventually reproduce the entire oeuvre of Shakespeare... or not...
posted by The Winsome Parker Lewis at 3:19 PM on March 26, 2010

Well, you really only have to do all the variations of one page. Only like 26³²⁰⁰ or so.
posted by smackfu at 3:30 PM on March 26, 2010

As cool as this is, it is a little bit frightening as well.

What if content-awareness makes us aware of something we would rather not see?
posted by purephase at 3:32 PM on March 26, 2010

To summarize the thread thus far:

1. This is definitely fake
2. I definitely did this in undergrad / GIMP definitely has had this for years

We need to update the maxim to be "Any sufficiently advanced technology is indistinguishable from fakery"
posted by 0xFCAF at 3:48 PM on March 26, 2010 [3 favorites]

I was literally yelling at my laptop this week because of GIMP. I cut an orange logo and paste it into an image. And it turned green. What. The. Fuck. I got to the point of yelling and downloaded Paint.NET. No problem.

That content-aware fill is ridiculous. That can't be real, can it?

I love living in the future.
posted by callmejay at 4:10 PM on March 26, 2010 [1 favorite]

Favourited not just for the linked video but for the post title.
posted by maudlin at 4:12 PM on March 26, 2010 [1 favorite]

quin My friend had this brilliant (silly) idea; he wants to write a program that takes a photograph sized array of pixels and have them as quickly as possible cycle through all the possible color combinations for each pixel and then save any interesting results.

Jorge Luis Borges The Library of Babel.

Of course, it's a concept beyond mere impossibility, but it does raise interesting questions for copyright law. When Sony or something similar asserts copyright over Track4.mp3, they are asserting copyright over "sufficiently similar" works. That is, all works that are not meaningfully distinguishable from Track4.mp3, that can be listened to and identified as the same track. That is, containing enough of the same data. This can be varied quite a lot, and still be identifiable. Maybe up to 50% of the bytes can be varied. This means that for a 5MB mp3 file, there also exists a "shadow cloud" of 2 ^{(2.5 x 8 x 1024 x 1024)} works to which that copyright also applies, each of which the copy of Track4.mp3 on your iPod also infringes.
posted by aeschenkarnos at 4:20 PM on March 26, 2010

shopped
posted by ericost at 4:24 PM on March 26, 2010 [1 favorite]

The Winsome Parker Lewis: "Anybody know if content-aware resizing is also available for the GIMP? I so want to get my grubby hands all over that crap."

This is what people cite as the GIMP version, however last I checked it was buggy as hell. If you expand an image it picks the same path to resize over and over again. The original paper noticed this and documented a fairly obvious fix: when you grow by n pixels on a dimension, rather than find the best path to expand n times, find the n best paths to expand.
posted by pwnguin at 4:56 PM on March 26, 2010 [1 favorite]

The part that puzzles me is where the GIMP guy got the originals to recreate it. I'd also love to see the full-size images he created. I've seen a ton of stuff that looks good when zoomed out but has flaws all over in 100% mode. (For instance, nearly every panorama I've stitched.)
posted by smackfu at 5:12 PM on March 26, 2010

This is nothing new. I've been using content-aware fill to pad out MetaFilter comments that I couldn't think how to finish for ages. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Aliquam placerat faucibus urna a dapibus. Mauris non nunc massa. Ut erat ipsum, mattis vel pellentesque eu, mollis non nisl. Aliquam vehicula metus eget dolor suscipit viverra. In dapibus magna non massa accumsan sagittis. But that last one with the clouds fucking blew my mind.
posted by EndsOfInvention at 6:20 PM on March 26, 2010 [3 favorites]

My friend had this brilliant (silly) idea; he wants to write a program that takes a photograph sized array of pixels and have them as quickly as possible cycle through all the possible color combinations for each pixel and then save any interesting results.

Check out John F. Simon's Every Icon.
posted by oulipian at 6:32 PM on March 26, 2010

"Though, for example, at a rate of 100 icons per second (on a typical desktop computer), it will take only 1.36 years to display all variations of the first line of the grid, the second line takes an exponentially longer 5.85 billion years to complete." - Artist's Statement
posted by oulipian at 6:34 PM on March 26, 2010

Yep, this is what GIMP has had for a few years in the plugin Resynthesizer, just with a way way slicker user interface (much like their content aware resizing feature -- much nicer UI than the free software equivalents.)
posted by edheil at 6:42 PM on March 26, 2010

This was my take on the every icon idea. You can drill down to find any icon you want.
posted by aneel at 7:54 PM on March 26, 2010

Will I be able to do this in the future with words? I type 'future', 'do', 'words', and keep typing until I get the previous sentence? To eliminate the need for sequence in writing?
posted by niccolo at 9:10 PM on March 26, 2010

Nahhhhh. I don't trust this video shit. Pics or it didn't happen!
posted by Duke999R at 9:35 PM on March 26, 2010

cjorgensen: "via?"

Nah, I didn't see it there. It was a random video linked on VideoSift that I thought was cool. I'm sure its all over all sorts of blogs. If I had come across it as a result of a blog, I would have given credit.
posted by lazaruslong at 11:50 PM on March 26, 2010 [1 favorite]

Of course it's fake. The whole point is that it's fake. It's just a question of whether you're being fooled by man or machine. Meta!
posted by thejoshu at 8:41 AM on March 27, 2010 [2 favorites]

I believe someone did the "cycle X to copyright all Y" with notes, too, during the early days of DRM stupidity.
posted by five fresh fish at 10:07 AM on March 27, 2010

If I had come across it as a result of a blog, I would have given credit.

Cool. That's just where I saw it first, was wondering if you had as well.
posted by cjorgensen at 11:05 AM on March 27, 2010

Little known fact: It took Adobe only 2 weeks to copy this from the GIMP. It took them 2 years to figure out how to install GIMP and the plugin, then how to open up an image.
posted by chairface at 1:36 PM on March 27, 2010 [2 favorites]

Pretty cool looking ... but I'm gonna needa see what it does with an astronaut or a comrade before I make up my mind.

And maybe that grassy knoll ... gunman or trick-of-the-light?
posted by Twang at 6:29 PM on March 27, 2010

DO NOT try to content aware fill blank space. There are eldritch beasts that live in the folds of blankness, and this could bring them forth.

Well that, or a fuzzy, squint your eyes and it's sort of right, picture of Jesus.
posted by seanyboy at 3:40 AM on March 28, 2010 [1 favorite]

Did anyone else notice that Facebook crops profile pictures to the biggest face in the image (to generate the profile thumbnails)?
posted by miyabo at 8:15 AM on March 28, 2010

Haha it was only a matter of time until someone made a video about this.
posted by lazaruslong at 11:12 AM on March 29, 2010 [1 favorite]

Excellent use of the "it's made a sandwich" tag, there.
posted by ook at 11:40 AM on March 29, 2010