Moon mode is not what it seems
April 24, 2019 10:48 AM Subscribe
Have Huawei developed an AI that cleverly enhances your pictures of the moon? Or is it just adding artefactual moon-things to your shaky blurry snap cos it wants you to be happy? A user, Wang Yue, puts it to the test at Zhihu (Chinese language article), and Huawei respond to questions at Android Authority.
Spotted on twitter, thread here.
'Artificially enhances' vs. 'Adds artefacts' might not be a non-dichotomous distinction I guess?
Spotted on twitter, thread here.
'Artificially enhances' vs. 'Adds artefacts' might not be a non-dichotomous distinction I guess?
Machine learning for image enhancement is best thought of as controlled hallucination anyway. They're just being a bit more aggressive and specific here than normal.
Eventually we will have phones that will hallucinate entire faces when an image comes out blurry, because it recognizes that that's Joe's face, and you have 100 other clear pictures of Joe, so it can take a stab at guessing what the non-blurry picture might have looked like.
posted by BungaDunga at 11:11 AM on April 24, 2019 [17 favorites]
Eventually we will have phones that will hallucinate entire faces when an image comes out blurry, because it recognizes that that's Joe's face, and you have 100 other clear pictures of Joe, so it can take a stab at guessing what the non-blurry picture might have looked like.
posted by BungaDunga at 11:11 AM on April 24, 2019 [17 favorites]
does this moon mode make my butt look bigger?
posted by nikaspark at 11:14 AM on April 24, 2019 [2 favorites]
posted by nikaspark at 11:14 AM on April 24, 2019 [2 favorites]
Fizz, I came here for that and you did not disappoint.
posted by greermahoney at 11:15 AM on April 24, 2019
posted by greermahoney at 11:15 AM on April 24, 2019
>phones that will hallucinate entire faces when an image comes out blurry
which is just what your brain's doing anyway, most of the time. Imagining the world.
posted by PandaMomentum at 11:16 AM on April 24, 2019 [19 favorites]
which is just what your brain's doing anyway, most of the time. Imagining the world.
posted by PandaMomentum at 11:16 AM on April 24, 2019 [19 favorites]
Concerns about Moon Mode aside, the P30's camera has absolutely ridiculous low-light performance, and pushes the limits of what I thought was physically possible.
posted by schmod at 11:29 AM on April 24, 2019 [9 favorites]
posted by schmod at 11:29 AM on April 24, 2019 [9 favorites]
the P30's camera has absolutely ridiculous low-light performance
That.... OK, you're right, that does not seem possible.
posted by The Bellman at 11:33 AM on April 24, 2019 [3 favorites]
That.... OK, you're right, that does not seem possible.
posted by The Bellman at 11:33 AM on April 24, 2019 [3 favorites]
What's so weird about moon mode? The moon is illuminated same as the earth - so a daylight f-stop and shutter speed work. The usual rule of thumb is 1/ISO at f/8.
So if it recognizes the moon - a single white orb, generally, surrounded by relative darkness - and suggests a mode tailored to that, what's the big deal? Perhaps the *only* magic is in recognizing, despite most of the photo being dark sky, it's a moon shot, so the camera doesn't try to over-expose to force the sky to 20% grey.
posted by notsnot at 11:50 AM on April 24, 2019 [1 favorite]
So if it recognizes the moon - a single white orb, generally, surrounded by relative darkness - and suggests a mode tailored to that, what's the big deal? Perhaps the *only* magic is in recognizing, despite most of the photo being dark sky, it's a moon shot, so the camera doesn't try to over-expose to force the sky to 20% grey.
posted by notsnot at 11:50 AM on April 24, 2019 [1 favorite]
Perhaps the P30's camera is so good they have to use AI to prevent pictures of the pressurized tent city on the near side of the moon built by extraterrestrial refugees. No pics means it's not happening.
posted by otherchaz at 11:55 AM on April 24, 2019 [6 favorites]
posted by otherchaz at 11:55 AM on April 24, 2019 [6 favorites]
So if it recognizes the moon - a single white orb, generally, surrounded by relative darkness - and suggests a mode tailored to that, what's the big deal? Perhaps the *only* magic is in recognizing, despite most of the photo being dark sky, it's a moon shot, so the camera doesn't try to over-expose to force the sky to 20% grey.
Because it's pretty clearly* hallucinating the features on the moon based on reference images, not sharpening up the data that's actually there. So your picture of the moon is really only some percentage your pixels, and some other, larger percentage other people's pixels. It's not like sunset modes or backlit modes that are just trying to adjust the camera settings to something useful.
*allegedly
posted by BungaDunga at 12:10 PM on April 24, 2019
Because it's pretty clearly* hallucinating the features on the moon based on reference images, not sharpening up the data that's actually there. So your picture of the moon is really only some percentage your pixels, and some other, larger percentage other people's pixels. It's not like sunset modes or backlit modes that are just trying to adjust the camera settings to something useful.
*allegedly
posted by BungaDunga at 12:10 PM on April 24, 2019
>Machine learning for image enhancement is best thought of as controlled hallucination anyway.
>which is just what your brain's doing anyway, most of the time. Imagining the world.
Yes, both of these things! This is a very interesting little development. And objecting to machines "filling in the details" is silly in a way when our brain does the exact same thing! Where do you think your blind spot goes when you're not paying attention to it? Your brain is just making up what it thinks goes there. It's incredible!
How much we want our cameras to do this is something we're figuring out and my guess is Huawei turned the dial up a little too high here so the phone's "brain" "sees" something that the camera can't possibly resolve. That's a perfectly understandable mistake and I suspect this kind of thing will add a fascinating tension to media and cameras in the future.
I wrote a bit about the trend in computational photography (self-link!) because it's so cool and stories like this are just like popcorn to me.
posted by BlackLeotardFront at 12:13 PM on April 24, 2019 [5 favorites]
>which is just what your brain's doing anyway, most of the time. Imagining the world.
Yes, both of these things! This is a very interesting little development. And objecting to machines "filling in the details" is silly in a way when our brain does the exact same thing! Where do you think your blind spot goes when you're not paying attention to it? Your brain is just making up what it thinks goes there. It's incredible!
How much we want our cameras to do this is something we're figuring out and my guess is Huawei turned the dial up a little too high here so the phone's "brain" "sees" something that the camera can't possibly resolve. That's a perfectly understandable mistake and I suspect this kind of thing will add a fascinating tension to media and cameras in the future.
I wrote a bit about the trend in computational photography (self-link!) because it's so cool and stories like this are just like popcorn to me.
posted by BlackLeotardFront at 12:13 PM on April 24, 2019 [5 favorites]
This is an interesting use of the snapchat filter concept, that's for sure. From a PR perspective this kind of story is not exactly desirable for them, so I'm kind of surprised they didn't position it more along those lines. Instead they have a sort of weird psuedo-camera which actually takes pictures of a partially simulated augmented reality moon which they are trying to pass off as totally legit with a straight face. It could be interesting if they were above board with it.
posted by feloniousmonk at 12:13 PM on April 24, 2019 [1 favorite]
posted by feloniousmonk at 12:13 PM on April 24, 2019 [1 favorite]
It's kind of astonishing to think that the moon is the only object that everyone around the world can take clear photos of, with features and all. You can try to get a shot of the sun through some heavy medium, or careful shots of a starry night sky, but that's not the same level of photographability as THE MOON.
So yeah sure, you know what the moon should look like, so why not drop a better image in there?
posted by rum-soaked space hobo at 12:17 PM on April 24, 2019 [1 favorite]
So yeah sure, you know what the moon should look like, so why not drop a better image in there?
posted by rum-soaked space hobo at 12:17 PM on April 24, 2019 [1 favorite]
Here you are, everybody. Have a moon and be happy: ๐
posted by rum-soaked space hobo at 12:19 PM on April 24, 2019 [1 favorite]
posted by rum-soaked space hobo at 12:19 PM on April 24, 2019 [1 favorite]
some other, larger percentage other people's pixels.
Eeeeeewwwwwww, used pixels!
posted by Greg_Ace at 12:35 PM on April 24, 2019 [6 favorites]
Eeeeeewwwwwww, used pixels!
posted by Greg_Ace at 12:35 PM on April 24, 2019 [6 favorites]
Will this lead to an enhanced version of the Joo Janta 200 Super-Chromatic Peril Sensitive Sunglasses from HHGTTG? The original model just turns black/opaque to prevent you from seeing anything that might alarm you. With this technology it could instead substitute anything you don't want to see with other stuff.
posted by Hairy Lobster at 12:51 PM on April 24, 2019 [5 favorites]
posted by Hairy Lobster at 12:51 PM on April 24, 2019 [5 favorites]
With this technology it could instead substitute anything you don't want to see with other stuff.
Fox News already exists.
posted by biogeo at 12:53 PM on April 24, 2019 [8 favorites]
Fox News already exists.
posted by biogeo at 12:53 PM on April 24, 2019 [8 favorites]
the pressurized tent city on the near side of the moon built by extraterrestrial refugees.
No, John Varley taught me that the refugees who fled to the moon largely settled on Farside because they couldn't stand the sight of the Earth they'd lost to the Invaders hanging over them in the sky. Only weirdo hermits and occasional runaway kids went to Nearside.
posted by The Tensor at 1:07 PM on April 24, 2019 [1 favorite]
No, John Varley taught me that the refugees who fled to the moon largely settled on Farside because they couldn't stand the sight of the Earth they'd lost to the Invaders hanging over them in the sky. Only weirdo hermits and occasional runaway kids went to Nearside.
posted by The Tensor at 1:07 PM on April 24, 2019 [1 favorite]
A couple of details...
The person who first raised the issue is named '็่ท็จ' Wang Yuekun. Two days agao he was fired over this affair from his company 'Aifou Technology', which is a technology review company like cnet.com. Their official Weibo account is here, and Wang was one of their head writers. Their Weibo post about the firing decision was widely disliked and many suspect that Aifou was pressured into this decision. Too bad I can't seem to link to that post directly.
posted by of strange foe at 1:18 PM on April 24, 2019 [7 favorites]
The person who first raised the issue is named '็่ท็จ' Wang Yuekun. Two days agao he was fired over this affair from his company 'Aifou Technology', which is a technology review company like cnet.com. Their official Weibo account is here, and Wang was one of their head writers. Their Weibo post about the firing decision was widely disliked and many suspect that Aifou was pressured into this decision. Too bad I can't seem to link to that post directly.
posted by of strange foe at 1:18 PM on April 24, 2019 [7 favorites]
I just see a photo of my buddy's girlfriend, their breakup was rough.
posted by maxwelton at 1:49 PM on April 24, 2019 [2 favorites]
posted by maxwelton at 1:49 PM on April 24, 2019 [2 favorites]
If the phone knows the current location, orientation and time, and can call up cloud coverage maps, it could render an accurate image of what the moon would look like and substitute that in in place of the blob the sensor picks up.
Though why stop there? Why not also, if photographing a night sky, replace the urban light pollution with a jet-black sky with the stars clearly visible in their rightful places? Or, closer to earth, use mapping data we have to render buildings and streets, in a way that improves the composition (removing passersby, vehicles, and so on)? It could also work for people and animals. Replace bystanders in a photo with good-looking models of the sort someone leading a #blessed influencer life would be surrounded by. Make your dog a dragon. The possibilities are limitless.
posted by acb at 1:54 PM on April 24, 2019 [5 favorites]
Though why stop there? Why not also, if photographing a night sky, replace the urban light pollution with a jet-black sky with the stars clearly visible in their rightful places? Or, closer to earth, use mapping data we have to render buildings and streets, in a way that improves the composition (removing passersby, vehicles, and so on)? It could also work for people and animals. Replace bystanders in a photo with good-looking models of the sort someone leading a #blessed influencer life would be surrounded by. Make your dog a dragon. The possibilities are limitless.
posted by acb at 1:54 PM on April 24, 2019 [5 favorites]
Surely there's an easy way to resolve this - take a photo of the moon from the Southern Hemisphere and see if the result is the right way up ....
(for those who don't know the moon and most of the constellations are upside down in the Northern Hemisphere - for example Orion's Pot is upside down in the Northern Hemisphere)
posted by mbo at 2:46 PM on April 24, 2019 [2 favorites]
(for those who don't know the moon and most of the constellations are upside down in the Northern Hemisphere - for example Orion's Pot is upside down in the Northern Hemisphere)
posted by mbo at 2:46 PM on April 24, 2019 [2 favorites]
If you have the location tagging enabled and it sees you're in Japan it puts a rabbit moon in there instead of a cheese moon.
posted by glonous keming at 4:01 PM on April 24, 2019 [1 favorite]
posted by glonous keming at 4:01 PM on April 24, 2019 [1 favorite]
You should see what happens when I forget to turn off my Huawei's beauty face filter when I take a selfie. It's pretty horrifying to see this hobo bearded craggy faced old man with smoothed skin, enhanced eyelashes and rosy cheeks. I think blending my image with moon might turn out better.
posted by srboisvert at 4:24 PM on April 24, 2019 [2 favorites]
posted by srboisvert at 4:24 PM on April 24, 2019 [2 favorites]
Surely there's an easy way to resolve this - take a photo of the moon from the Southern Hemisphere and see if the result is the right way up ....
Couldn't the moon mode software check GPS coordinates to determine which way to render the moon?
posted by acb at 5:12 PM on April 24, 2019 [1 favorite]
Couldn't the moon mode software check GPS coordinates to determine which way to render the moon?
posted by acb at 5:12 PM on April 24, 2019 [1 favorite]
Does moon mode also make the moon appear 10x bigger so that it's as big in the picture as you had expected before you pulled out the camera?
posted by ckape at 6:01 PM on April 24, 2019 [3 favorites]
posted by ckape at 6:01 PM on April 24, 2019 [3 favorites]
Kodachrome
They give us those nice moon colors
They give us the green moon colors
Makes you think all the moon's a sunny day
I got an Android camera
I love to take a moon photograph
So mama don't take my Kodamoon away
posted by RobotVoodooPower at 7:30 PM on April 24, 2019 [3 favorites]
They give us those nice moon colors
They give us the green moon colors
Makes you think all the moon's a sunny day
I got an Android camera
I love to take a moon photograph
So mama don't take my Kodamoon away
posted by RobotVoodooPower at 7:30 PM on April 24, 2019 [3 favorites]
This is how you end up with Thomas Riker.
posted by Poogle at 2:09 AM on April 25, 2019 [1 favorite]
posted by Poogle at 2:09 AM on April 25, 2019 [1 favorite]
If you want to take a photo of the moon
Have an AI trace its craters in the afternoon
Superimpose deep shadows and light that's alien
Until you drop your Android phone on the pavement
And you want to take a photo of the moon
And you believe the PR goons
Na na na na
Yeah yeah yeah
Presto! Photo of the moon.
posted by anthom at 3:14 AM on April 25, 2019
Have an AI trace its craters in the afternoon
Superimpose deep shadows and light that's alien
Until you drop your Android phone on the pavement
And you want to take a photo of the moon
And you believe the PR goons
Na na na na
Yeah yeah yeah
Presto! Photo of the moon.
posted by anthom at 3:14 AM on April 25, 2019
Where do you think your blind spot goes when you're not paying attention to it?
I'll tell you for sure where it doesn't go, and that's into the uncanny valley.
posted by flabdablet at 5:42 AM on April 25, 2019
I'll tell you for sure where it doesn't go, and that's into the uncanny valley.
posted by flabdablet at 5:42 AM on April 25, 2019
he person who first raised the issue is named '็่ท็จ' Wang Yuekun.
I canโt tell you how bummed I am that his first name is not Yue (ๆ), as in, the moon. I went and checked the Chinese article to be sure.
posted by chainsofreedom at 6:26 AM on April 25, 2019 [2 favorites]
I canโt tell you how bummed I am that his first name is not Yue (ๆ), as in, the moon. I went and checked the Chinese article to be sure.
posted by chainsofreedom at 6:26 AM on April 25, 2019 [2 favorites]
If you said something like "standard light collection / photo enhancing algorithms can explain this" then you didn't actually read the original posts. The author designed tests for this, and (IMO) convincingly showed that something else is involved. He also suggested what it probably is, and it's very reasonable.
The p30 camera (and many others these days) has a dedicated neural network chip designed to enhance photos based on machine learning - that is, based on some training set of "good" photos. There is landscape mode, portrait, food, etc. The thing about moon photos is that the training set is really homogeneous: some features (like the crater the author spends a lot of time on) are in *every single good shot of the moon*. So the ML algorithm knows that the crater "should" be there, and further it knows where in the photo it should be. So it gets added, even if it wasn't actually there to begin with.
Thing is, it's probably not malicious. I'd bet good money nobody ever consciously decided to have moon mode add features that aren't there. This is just a great example of how machine learning can be unpredictable, and how we need to treat the output of such systems skeptically.
posted by dbx at 7:01 AM on April 25, 2019 [3 favorites]
The p30 camera (and many others these days) has a dedicated neural network chip designed to enhance photos based on machine learning - that is, based on some training set of "good" photos. There is landscape mode, portrait, food, etc. The thing about moon photos is that the training set is really homogeneous: some features (like the crater the author spends a lot of time on) are in *every single good shot of the moon*. So the ML algorithm knows that the crater "should" be there, and further it knows where in the photo it should be. So it gets added, even if it wasn't actually there to begin with.
Thing is, it's probably not malicious. I'd bet good money nobody ever consciously decided to have moon mode add features that aren't there. This is just a great example of how machine learning can be unpredictable, and how we need to treat the output of such systems skeptically.
posted by dbx at 7:01 AM on April 25, 2019 [3 favorites]
But motion is easily corrected with image stacking. Instead of a 30s exposure, you align and merge individual shots. This reduces noise, increases light gathering and incidentally corrects for motion
The article (in Chinese, so the translation is a bit wonky) involves taking pictures of a blurry picture of the moon, which is then magically enhanced to look like a nice crisp shot of the moon. No amount of focus stacking can do that: the details have been pre-smoothed away.
The article (in Chinese, so the translation is a bit wonky) involves taking pictures of a blurry picture of the moon, which is then magically enhanced to look like a nice crisp shot of the moon. No amount of focus stacking can do that: the details have been pre-smoothed away.
The first round of experiments:And then, to make things really bullet proof, the author tries adding and removing features from the blurry moon with Photoshop to see if the camera is really providing some sort of "superresolution", and finds that... surprise, nope, the removed features show back up and the added ones go away.
1. Gaussian blurring of the moon image until many details are completely invisible or invisible.
2. Use the camera to take this blurred moon picture and trigger the P month function.
3. The photo restores many details that are not visible in the paste.
If someone still wants to say that the super-resolution algorithm is restored, please tell me, if there is no "standard answer" lunar data as a reference, why is the result of the restoration not the modified moon, but the original moon?posted by BungaDunga at 7:11 AM on April 25, 2019 [3 favorites]
Does moon mode also make the moon appear 10x bigger so that it's as big in the picture as you had expected before you pulled out the camera?
If you are going to fake the moon then of course you need to do this! And you need to offer an "ET on a bike" option for people of a certain age.
posted by rongorongo at 10:05 AM on April 25, 2019 [1 favorite]
If you are going to fake the moon then of course you need to do this! And you need to offer an "ET on a bike" option for people of a certain age.
posted by rongorongo at 10:05 AM on April 25, 2019 [1 favorite]
I've got a Moon Mode, but people really don't like it when I use it in photographs.
posted by lucidium at 1:30 PM on April 25, 2019 [5 favorites]
posted by lucidium at 1:30 PM on April 25, 2019 [5 favorites]
> we need to treat the output of such systems skeptically
No, no--we need these nice, friendly AI system driving all of our automobiles for us ASAP. No need to hang around waiting for testing or proof of safety . . . if it's AI, it must be good.
posted by flug at 7:05 PM on April 25, 2019 [2 favorites]
No, no--we need these nice, friendly AI system driving all of our automobiles for us ASAP. No need to hang around waiting for testing or proof of safety . . . if it's AI, it must be good.
posted by flug at 7:05 PM on April 25, 2019 [2 favorites]
Somehow Douglas Adams got it exactly correct with his Sirius Cybernetics Corporation.
posted by BungaDunga at 8:56 PM on April 25, 2019
posted by BungaDunga at 8:56 PM on April 25, 2019
« Older Treatment was noninferior to "it's just a phase" | Enjoy your spot in antiquity! Say hi to westerns... Newer »
This thread has been archived and is closed to new comments
posted by Fizz at 10:57 AM on April 24, 2019 [15 favorites]