Smoother /= Better in Animation
September 28, 2023 7:02 PM Subscribe
(double link youtube) Noodle explains why applying smoothing and interpolation to animation ruins it. Just for good measure Knowing Better explains why you don't see in 4K. Some NSFW language
That "Knowing Better" clip repeats some pretty glaring misconceptions about whether humans can "see" higher frame rates or higher resolutions. His benchmark for "seeing" something is that if it appears on the screen for 1/72th of a second and you miss it, it's proof that you can't see more than 72 frames per second.
Human vision isn't a binary test of seeing it or not seeing it - the perception and fusion of the image by the brain is what is important. Typical (wiki link) film projection considers a 180° shutter angle the norm, which means that a 24 frames per second film is actually projected at 48 frames per second, with visible frames every odd frame and the shutter covering the light every even frame to advance the film to the next frame (otherwise you'd see the film physically move and smear across the screen). We don't perceive these blank frames directly, however they have a huge impact on how our brain fuses these images. Our brain uses these moments of blackness to interpolate motion between frames. Without the blank frames, we would perceive each frame immediately transitioning to the next, and an object moving across the screen in discrete hard steps which ends up being perceived as motion blur rather than smooth motion - see a simulated example here, I can confirm it really does look that good.
Black frame insertion has been used for years in gaming monitors (my AOC G2460PG was released in 2014), and is currently being used in VR where smooth motion is paramount - the term of art in VR is low persistence where each frame "persists" for only a fraction of time, with the rest being a full dark screen.
So the point is not whether we can see something for a fraction of a second or not - we clearly can't see these full dark frames - but rather how frame rate and persistence combine to give us better image quality. He claims almost no one can see beyond 120fps, but I can guarantee you virtually everyone who looks at a moving object on a 60fps display versus a 240fps or low persistence display will be gobsmacked at how much better it looks.
In fact, even animation heavily uses this trick, with its roots in film.
I think almost all PC monitors, phone screens and TVs are totally missing the mark. Motion smoothing is an abomination as well. If you don't have a low persistence display, it's so visibly inferior to regular displays it blows my mind people still stick with the old ones. Another way of looking at it - WE SOLVED THIS ISSUE decades ago with film projection which naturally had black frame insertion due to the film advance mechanism, even CRT monitors solved this problem by not having full persistence as the beam scanned the pixels row by row, it's only with LCD monitors that we've had to reinvent a 100 year old technology.
As for 4K... yeah he kind of has a point for most people, with a 55" TV in a regular setting the difference between 1080p and 4K isn't going to be worth it. However his video is 6 years old, and today displays are getting so cheap, I just bought a 75" 4K TV for something like AUD900 (USD 600) and I am genuinely enjoying 4K content on it, it makes a pretty visible difference. Well, for nature and documentary videos. There are films and TV series that are simply filmed and mastered in 1080p because the image clarity isn't the point - Succession and Andor are the two recent ones that come to mind where I seem to recall the creators simply saying they never aimed for 4K.
posted by xdvesper at 10:17 PM on September 28, 2023 [11 favorites]
Human vision isn't a binary test of seeing it or not seeing it - the perception and fusion of the image by the brain is what is important. Typical (wiki link) film projection considers a 180° shutter angle the norm, which means that a 24 frames per second film is actually projected at 48 frames per second, with visible frames every odd frame and the shutter covering the light every even frame to advance the film to the next frame (otherwise you'd see the film physically move and smear across the screen). We don't perceive these blank frames directly, however they have a huge impact on how our brain fuses these images. Our brain uses these moments of blackness to interpolate motion between frames. Without the blank frames, we would perceive each frame immediately transitioning to the next, and an object moving across the screen in discrete hard steps which ends up being perceived as motion blur rather than smooth motion - see a simulated example here, I can confirm it really does look that good.
Black frame insertion has been used for years in gaming monitors (my AOC G2460PG was released in 2014), and is currently being used in VR where smooth motion is paramount - the term of art in VR is low persistence where each frame "persists" for only a fraction of time, with the rest being a full dark screen.
So the point is not whether we can see something for a fraction of a second or not - we clearly can't see these full dark frames - but rather how frame rate and persistence combine to give us better image quality. He claims almost no one can see beyond 120fps, but I can guarantee you virtually everyone who looks at a moving object on a 60fps display versus a 240fps or low persistence display will be gobsmacked at how much better it looks.
In fact, even animation heavily uses this trick, with its roots in film.
I think almost all PC monitors, phone screens and TVs are totally missing the mark. Motion smoothing is an abomination as well. If you don't have a low persistence display, it's so visibly inferior to regular displays it blows my mind people still stick with the old ones. Another way of looking at it - WE SOLVED THIS ISSUE decades ago with film projection which naturally had black frame insertion due to the film advance mechanism, even CRT monitors solved this problem by not having full persistence as the beam scanned the pixels row by row, it's only with LCD monitors that we've had to reinvent a 100 year old technology.
As for 4K... yeah he kind of has a point for most people, with a 55" TV in a regular setting the difference between 1080p and 4K isn't going to be worth it. However his video is 6 years old, and today displays are getting so cheap, I just bought a 75" 4K TV for something like AUD900 (USD 600) and I am genuinely enjoying 4K content on it, it makes a pretty visible difference. Well, for nature and documentary videos. There are films and TV series that are simply filmed and mastered in 1080p because the image clarity isn't the point - Succession and Andor are the two recent ones that come to mind where I seem to recall the creators simply saying they never aimed for 4K.
posted by xdvesper at 10:17 PM on September 28, 2023 [11 favorites]
Interpolating frames is an abomination. Animation that uses 30, 15, 12 frames a second or less does so on purpose. Listen to Noodle. (And watch his follow-up video as well!)
posted by Dysk at 12:13 AM on September 29, 2023 [3 favorites]
posted by Dysk at 12:13 AM on September 29, 2023 [3 favorites]
I watched these a while back, I don't recall him saying 'photorealism' because there's a chunk of this that assumes the goal is or it's better to have life-like photo-realistic visuals. Given a tiny proportion of people have visual recall or eidetic memories, that's daft.
posted by k3ninho at 1:07 AM on September 29, 2023
posted by k3ninho at 1:07 AM on September 29, 2023
As for 4K... yeah he kind of has a point for most people, with a 55" TV in a regular setting the difference between 1080p and 4K isn't going to be worth it.
The jump to 4k is paired with a jump to hdr and even if the resolution doesn't really matter very much, hdr makes a big difference.
posted by GCU Sweet and Full of Grace at 3:09 AM on September 29, 2023 [2 favorites]
The jump to 4k is paired with a jump to hdr and even if the resolution doesn't really matter very much, hdr makes a big difference.
posted by GCU Sweet and Full of Grace at 3:09 AM on September 29, 2023 [2 favorites]
Oh man yeah motion interpolation on animation especially makes me feel like I’m experiencing vertigo. The Haus of Decline guy compares it to the feel of quitting his SSRIs cold turkey.
posted by DoctorFedora at 5:46 AM on September 29, 2023
posted by DoctorFedora at 5:46 AM on September 29, 2023
Wait—is anyone making low persistence TVs though?
posted by thecaddy at 6:11 AM on September 29, 2023
posted by thecaddy at 6:11 AM on September 29, 2023
Wait—is anyone making low persistence TVs though?
I imagine they tried it for a while but gave up quickly.
posted by paper chromatographologist at 6:18 AM on September 29, 2023 [12 favorites]
I imagine they tried it for a while but gave up quickly.
posted by paper chromatographologist at 6:18 AM on September 29, 2023 [12 favorites]
Lots of tvs nowadays are low-persistence insofar as they have black frame insertion. It's mostly for improved motion clarity in gaming.
posted by GCU Sweet and Full of Grace at 6:27 AM on September 29, 2023 [1 favorite]
posted by GCU Sweet and Full of Grace at 6:27 AM on September 29, 2023 [1 favorite]
A flash in the pan, if you will…
posted by blue_beetle at 6:30 AM on September 29, 2023
posted by blue_beetle at 6:30 AM on September 29, 2023
Ah, cool. Something I’ll need to keep an eye out for if it’s a better solution than motion interpolation.
posted by thecaddy at 7:13 AM on September 29, 2023
posted by thecaddy at 7:13 AM on September 29, 2023
Spider-Man: Into the Spiderverse touches on this effect in one of the behind-the-scenes clips. Despite being largely animated in 3D, the animators deliberately kept frame rates lower to make the film feel more alive.
posted by Eikonaut at 8:37 AM on September 29, 2023
posted by Eikonaut at 8:37 AM on September 29, 2023
It's kind of insane how little people realize animated motion works best when it's just as stylized as the drawings. Recently I was playing with Moho (an animation program) and it felt revolutionary that it acknowledges that animators usually don't want things to move on ones all the time, and lets you set a framerate for its auto-tweening. I still remember all the bullshit I had to do in Flash to make its movement "less Flash-y", one of the last steps of any scene was usually to manually insert keyframes on every other frame of every damn tween, then convert them all to untweened frames. Moho lets you just say "tween this on 2s" and be done with it.
posted by egypturnash at 9:09 AM on September 29, 2023 [3 favorites]
posted by egypturnash at 9:09 AM on September 29, 2023 [3 favorites]
I found I literally could not see differences in his examples, so it was really hard to follow the argument. Maybe they went by too quickly?
posted by mittens at 11:52 AM on September 29, 2023 [1 favorite]
posted by mittens at 11:52 AM on September 29, 2023 [1 favorite]
Oh, but Into the Spider-Verse absolutely made me feel horrible--something in that jerky frame-rate just killed my eyes (and stomach). So it's interesting to hear so many people say the opposite.
posted by mittens at 12:19 PM on September 29, 2023 [1 favorite]
posted by mittens at 12:19 PM on September 29, 2023 [1 favorite]
Moho lets you just say "tween this on 2s" and be done with it.
OMG egypturnash yes I have also manually “baked” flash animation on twos and this feature seems so immensely thoughtful that the mere idea of it made me experience true love for the first time just now
posted by TangoCharlie at 12:37 PM on September 29, 2023 [1 favorite]
OMG egypturnash yes I have also manually “baked” flash animation on twos and this feature seems so immensely thoughtful that the mere idea of it made me experience true love for the first time just now
posted by TangoCharlie at 12:37 PM on September 29, 2023 [1 favorite]
i have a question about refresh rates on TVs used for computer displays, that in turn mostly get used to watch video.
i have a TCL 55" that natively runs at 60Hz (and ofc i have all the interpolation fuckery disabled). it is hooked up to a desktop PC that is my primary computing station and also my primary living-room media player box. i recently have been manually setting the computer's output resolution to 23.976 instead of the usual 60 (and it appears supported by the display) and i think this looks better when watching shows (played from local mkv files, not streamed from the internet) because there is no longer noticeable judder from doing the 3:2 pulldown. i usually check the codec info in VLC to confirm that the embedded stream's framerate is 23.976 but since i started doing this i haven't happened on one that's anything other than 23.976. presumable if i had one that was different i'd adjust my GPU's output Hz to match the stream's FPS.
do i have this correctly figured or am i just placebo-ing myself?
posted by glonous keming at 1:16 PM on September 29, 2023
i have a TCL 55" that natively runs at 60Hz (and ofc i have all the interpolation fuckery disabled). it is hooked up to a desktop PC that is my primary computing station and also my primary living-room media player box. i recently have been manually setting the computer's output resolution to 23.976 instead of the usual 60 (and it appears supported by the display) and i think this looks better when watching shows (played from local mkv files, not streamed from the internet) because there is no longer noticeable judder from doing the 3:2 pulldown. i usually check the codec info in VLC to confirm that the embedded stream's framerate is 23.976 but since i started doing this i haven't happened on one that's anything other than 23.976. presumable if i had one that was different i'd adjust my GPU's output Hz to match the stream's FPS.
do i have this correctly figured or am i just placebo-ing myself?
posted by glonous keming at 1:16 PM on September 29, 2023
Glonous Keming, I personally think for movies, using a HDMI source looks bad and introduces stuttering and juddering on the TVs I've had. So I pretty quickly stopped doing that.
I get much better results by playing the media file natively on the TV itself using the TVs own media player via USB.
posted by xdvesper at 4:03 PM on September 29, 2023
I get much better results by playing the media file natively on the TV itself using the TVs own media player via USB.
posted by xdvesper at 4:03 PM on September 29, 2023
Glorious Kerning, you have it correctly figured.
If your tv and computer both support it at your target resolution, you might prefer switching to 120hz which also avoids pulldown. For watching local files, you can also avoid some of this by using kodi and telling it to switch to the file's frame rate... which *mostly* works.
It's frustrating. We mostly stream using a shield tv, and I ended up setting it to 1080p 120hz instead of 4k60 to avoid pulldown, which... whatever, like I said above having the hdr matters more than 2k/4k. I gather appletv does a good job of using the framerate of the file being watched/streamed.
posted by GCU Sweet and Full of Grace at 4:49 PM on September 29, 2023
If your tv and computer both support it at your target resolution, you might prefer switching to 120hz which also avoids pulldown. For watching local files, you can also avoid some of this by using kodi and telling it to switch to the file's frame rate... which *mostly* works.
It's frustrating. We mostly stream using a shield tv, and I ended up setting it to 1080p 120hz instead of 4k60 to avoid pulldown, which... whatever, like I said above having the hdr matters more than 2k/4k. I gather appletv does a good job of using the framerate of the file being watched/streamed.
posted by GCU Sweet and Full of Grace at 4:49 PM on September 29, 2023
Wait—is anyone making low persistence TVs though?
On the surface, some claim to, but that's more a gaming thing where you pipe 120 frames per second from your gaming PC to the TV and it does low persistence on those frames.
I haven't owned one of those advanced TVs yet so I haven't had the chance to play with the settings, but on the surface none of them seem to say this implementation works for movie playback in particular.
Frustratingly even on PC the implementation doesn't really accommodate movies - variable frame rate mode can accommodate 24fps, but variable frame rate mode is mutually exclusive with black frame insertion, so when I'm in black frame insertion mode the lowest I can do is 60fps. I suppose I could do 120fps and black frame insertion on a 24fps media file to avoid 3:2 pulldown but that seems like the low persistence part of it wouldn't be ideal - what you really want is 24fps with black frame insertion.
posted by xdvesper at 8:00 PM on September 29, 2023
On the surface, some claim to, but that's more a gaming thing where you pipe 120 frames per second from your gaming PC to the TV and it does low persistence on those frames.
I haven't owned one of those advanced TVs yet so I haven't had the chance to play with the settings, but on the surface none of them seem to say this implementation works for movie playback in particular.
Frustratingly even on PC the implementation doesn't really accommodate movies - variable frame rate mode can accommodate 24fps, but variable frame rate mode is mutually exclusive with black frame insertion, so when I'm in black frame insertion mode the lowest I can do is 60fps. I suppose I could do 120fps and black frame insertion on a 24fps media file to avoid 3:2 pulldown but that seems like the low persistence part of it wouldn't be ideal - what you really want is 24fps with black frame insertion.
posted by xdvesper at 8:00 PM on September 29, 2023
mittens > something in (Spiderverse’s) jerky frame-rate just killed my eyes (and stomach)
There’s a lot of places in Spiderverse where the animation is on like threes or fours, while the camera is moving around on ones. I could really see it causing queasiness in people who are more sensitive to visual weirdness. There’s also really just too damn much information in every frame, I trained to be an animator so I can pay attention to every frame in a way most people can’t; when I saw Spiderverse 2 I started to feel kind of exhausted halfway through because I was trying to appreciate a *very* long series of nice paintings being flashed before my eyes at high speed. Really I think that style of animation needs a lot of holds and slow motions, and a huge percentage of the film was high speed action scenes.
posted by egypturnash at 3:16 PM on September 30, 2023 [2 favorites]
There’s a lot of places in Spiderverse where the animation is on like threes or fours, while the camera is moving around on ones. I could really see it causing queasiness in people who are more sensitive to visual weirdness. There’s also really just too damn much information in every frame, I trained to be an animator so I can pay attention to every frame in a way most people can’t; when I saw Spiderverse 2 I started to feel kind of exhausted halfway through because I was trying to appreciate a *very* long series of nice paintings being flashed before my eyes at high speed. Really I think that style of animation needs a lot of holds and slow motions, and a huge percentage of the film was high speed action scenes.
posted by egypturnash at 3:16 PM on September 30, 2023 [2 favorites]
when I saw Spiderverse 2 I started to feel kind of exhausted halfway through because I was trying to appreciate a *very* long series of nice paintings being flashed before my eyes at high speed.
This explains why it took me 3 tries to get through the whole film. I watched an hour, then had to step away and come back the next day for the next hour, and then a 3rd time.
posted by mikelieman at 4:01 PM on September 30, 2023
This explains why it took me 3 tries to get through the whole film. I watched an hour, then had to step away and come back the next day for the next hour, and then a 3rd time.
posted by mikelieman at 4:01 PM on September 30, 2023
I don't think I fully appreciated the way the Spiderverse films juxtapose different frame rates till reading these comments. To the point, it's still quite a sensation to see it done well on purpose, as opposed to watching your TV do it haphazardly to a studio Ghibli film and ruin it. I tend to love the way stop motion looks like a series of beautiful photographs. I could totally understand the overload one might experience during Spiderverse, but I tend to like it. The only real analogue that comes to mind is Waking Life, which always pulls me in and out of passive watching with its varied approach to figure and ground through rotoscoping.
posted by es_de_bah at 7:58 PM on October 6, 2023
posted by es_de_bah at 7:58 PM on October 6, 2023
The other touchstone for me, here, is Futurama using 3D modeling in a jarring way against the 'normal' 2d animation. Kinda worked because of the subject matter, so it didn't seem out of place. Currently watching Chainsaw Man and it's so weird when the texture-painted-on-model shots come up. But again, it's not awful, given the sureallity of the show. Being wrenched around fits for the tone.
posted by es_de_bah at 8:04 PM on October 6, 2023
posted by es_de_bah at 8:04 PM on October 6, 2023
« Older It’s like stealing joy. | Sitteeem Newer »
This thread has been archived and is closed to new comments
posted by ChurchHatesTucker at 10:13 PM on September 28, 2023 [12 favorites]