Cinematic Realism Considered - Part 3

Cinematic Realism Considered - Part 3

Cinematic Realism Considered - Parts 1 and 2 focused on the digital revolution that's been ongoing within the feature film Industry for over a decade. Digital Cinema has eliminated most of the constraints on moviemakers that analog film imposed on the Industry for almost 100 years.  This lack of constraints has created uncharted territory for the filmmaker, turning high frame rate (HFR) or temporal resolution and ultra high definition (UHD) or spatial resolution into creative tools that are only now undergoing study and experimentation.

The Hobbit has shown us that high frame rate can change the look and feel of a film from a low resolution cinematic story to high resolution cinematic realism that is intended to be an accurate representation of reality.  There's little doubt that over time, both UHD and HFR will undergo a considerable amount of refinement as moviemakers gain a better understanding of these new creative tools and how best to use them.  For instance, we've seen that HFR works best on wide shots but when a shot is character oriented in tight compositions, HFR can look somewhat uncomfortable.  In fact, legendary filmmaker and visual effects pioneer, Douglas Trumbull has stated that HFR is probably not appropriate for period or dialog driven films.  With HFR and UHD much of the movie making magic such as props, makeup and even acting undergoes audience scrutiny that if improperly composed and lensed can take moviegoers out of their sense of suspended disbelief.

What Role Does 3D Play in This Equation?

Is 3D somehow different than temporal or spatial resolution as a creative tool or is it a key ingredient for pushing the movie going experience over the top, from the realm of cinematic story telling to cinematic realism?

I believe that 3D is consistent with both experiences. In a 3D movie our brain actively creates a unique visual data set resulting from the horizontal displacement or disparity of the images projected onto our left and right retinae regardless of the temporal or spatial resolution. Our conscious perception of that unique data set is called stereopsis.

Of course, data from 2D images projected onto our retinas via HFR and/or UHD is also processed by our visual system but in a different manner.  In a 2D movie experience, temporal and/or spatial information hitting our left and right retinae are precisely identical in every way and void of any horizontal disparity. Consequently the brain will not transform HFR and UHD qualitatively into a new and different data set and therefore our perception of that visual data is not significantly different from the source.

It is well established in neuroscience that disparity resulting from the separation of our two eyes has a profound effect on the brain that is uniquely different than what is experienced with one eye.  In my blog, Engaged In 2D and Immersed in 3D, I point out that the lack of disparity in a 2D movie actually forces the audience to perceive that all the action is being presented on the movie screen and only on the movie screen, mitigating any significant semblance of immersion whether the movie is presented at 24fps or 48fps.  On the other hand, the horizontal displacement of images hitting our retinae due to the separation of our left and right eyes creates a sense of depth and volume that brings the action from the screen to the audience in a very personal manner that can infringe on each audience member's personal space... i.e. their comfort zone.  In fact, because our individual self-perceptions and personal spaces are so variable, no two people in the audience of a 3D movie will have the same subjective reaction to the same stereo visuals.  Unlike UHD and HFR, a 3D movie tears down the second, third and fourth wall of the cinematic experience, enveloping the entire theater in a unique manner for each audience member. And while the effectiveness of 3D to enhance a story is independent of HFR and UHD the opposite is not true. It's generally agreed that HFR and UHD are largely dependent on 3D to achieve the most convincing sense of cinematic realism and immersion.

Cinematic Realism and the Uncanny Valley:

Those filmmakers attempting to create an accurate sense of cinematic realism face a daunting task because any errors in the simulation of reality on the screen, no matter how slight and seemingly irrelevant will likely be noticed by the audience.  That's because the human brain has been hard wired through evolution to anticipate a world that visually follows a set of rules.

When those rules are broken a phenomenon called the  'uncanny valley' is often the result. The uncanny valley was first coined by the robotics professor Masahiro Mori in 1970 to describe how we perceive humanoid constructs like puppets, cartoon characters, robots, clowns, etc that are clearly not intended to approach reality in appearance relative to how we perceive puppets, robots and cartoon characters that are intended to accurately simulate reality.

The closer that these artificial constructs come to appear as accurate replications of reality/realism the more our sense of empathy increases. However, as these human approximations reach a point closest to realism, visual nuances of imperfection can cause us to become uncomfortable with the image. Indeed, we often find even minutely flawed human simulations as grotesque.

I believe, the uncanny valley applies to many forms of cinematic realism and it may explain why the Hobbit at 48 fps had mixed reviews.  Most of the time our brains can automatically fill in missing data and/or accept errors in low-resolution data. However if realism is clearly the end goal of a filmmaker then every detail of that simulated reality must be void of visual errors or the opposite effect will be achieved. It's because HD or 2K projected at 24 frames per second (even in the presence of 3D) is a lower resolution experience than HFR and UHD that audiences tends to be much more forgiving when it comes to inaccuracies, intentional cheats, etc in filming.

One thing is for certain, the lack of technical constrains from digital cinema will greatly complicate the art of moviemaking in the future. Some cinematographers and directors will likely embrace the creative options this new era in filmmaking presents while others will avoid or reject them in favor of what they consider traditional approaches to storytelling.

Regardless of how the Industry approaches these advancements in filmmaking one thing is for certain... It's going to get very interesting. 

Cinematic Realism Considered - Part 2

Cinematic Realism Considered - Part 2

In Part 1 of CinematicRealism Considered, I discussed both ultra high definition (UHD) and high frame rate (HFR) as signaling the latest and most radical revolution in cinema technology in the last half century. These two recent advances in movie making and theatrical exhibition together with the advent of digital 3D can create a movie going experience that blurs the separation between fantasy and reality, fiction from non-fiction. Indeed the expressed purpose of these three technologies in combination is to simulate reality as closely as possible... creating cinematic realism.

To be clear, none of these advances would have been possible without the introduction and current ubiquitous status of digital filmmaking and exhibition. Digital Cinema became a disruptive technology only during the last decade, completely changing the way we think about moviemaking and removing most of the constraints that have been inherent in the use of physical film for almost a century. 

As a result, filmmakers like Peter Jackson have begun pushing the boundaries of this new medium in an attempt to dramatically change the movie going experience, proclaiming cinematic realism the future of cinema. However, to some, these changes call into question the very heart and definition of the cinematic experience.

Is There an Ideal Cinematic Experience?

Why do we go to the theater in the first place? I believe that most of us go to the theater to escape reality via visuals and audio that depict a story. But as camera, cinematic projection and screen technology continue to advance one has to wonder whether the movie going experience will continue to offer an escape from reality or is it destined to become an accurate simulation of reality framed within a story. This is an important distinction because we know from recent HFR releases that there are some movie goers who consider cinematic realism a radical if not uncomfortable departure from the familiar cinematic experience. 

The Digital Revolution in Cinema:

But lets back up a bit and put HFR in historical perspective.  In the absence of significant advances in digital camera and digital projection technology over the last decade this blog on cinematic realism would have little relevance.

Today the majority of movie theaters around the world are fully digital. In fact, digital technology has for the most part completely eliminated analog film for theatrical exhibition. This has had a profound effect on film printing and film distribution, which was the cash cow of the major film labs. Today, filmmaking and exhibition has entered a whole new era in which postproduction houses must compete with smaller, more agile finishing houses and VFX facilities must gear up with software and capex requirements to handle higher resolutions and higher frames rates. 

The 24 Frame Rate Standard:

Prior to the introduction of digital cinema and for nearly the past 90 years, the standard frame rate in the U.S. for exhibiting feature films has been 24 frames per second (fps). To the theater going audience today and in the past, the 24 fps experience is what cinema has been and what it's supposed to be.

So what is it about 24 fps that makes it so unique?  Nothing really. The 24 fps standard was an arbitrary number introduced to feature films in 1926 because it was considered the slowest film speed that could produce the illusion of smooth movement when coupled with audio while at the same time conserving the cost of film.  While 24 frames remains the current standard in digital cinema today, both analog film and digital projectors are typically double flashed or triple flashed meaning that each frame is shown twice or three times as they are projected. This is done to reduce the transition time between frames thereby mitigating flicker. So while we've always experienced 48 or 72 frames per second in the theater, only 24 frames of actual visual information has been displayed each second. I believe audiences tend to perceive today’s feature film experience as something other than reality because of the relatively low resolution that we get from 24 fps delivered in HD or 2K. This is true whether or not 3D is part of the equation. 

Deviating From The Norm:

To deviate from 24 fps can impose a different experience on the audience that may not be considered an enhancement for some films. Both HFR and UHD dramatically increase the amount of visual data delivered to our eyes and brain. High frame rate delivers more images per second, which increases the amount of visual information hitting our retinas per unit of time.  This is called 'temporal resolution' or time based resolution. Ultra high definition on the other hand increases the pixel resolution of each frame. It's going from HD at 1920 by 1080 to UHD at 3840 by 2160, which is 4 times the resolution.  It's basically the difference between a 2.07 megapixel image and an 8.29 megapixel image. This is called 'spatial resolution’.

It's our persistence of vision or the amount of time an image remains on the retina after removal of that image that melds 48 fps into a continuous visual experience with high data rate.  So if you combine higher temporal resolution with higher spatial resolution the amount of data reaching the retina goes up geometrically and when shot correctly, the screen literally appears to be approaching reality. Add quality 3D to this mix and you have a unique cinematic experience that appears to be an extension of reality.  

So it's easy to understand that 24 fps at 1920 x 1080 or 2K is, by comparison a low-resolution experience that delivers a 'dreamy' state clearly separate from reality. Some filmmakers and moviegoers will undoubtedly prefer lower resolution over UHD and HFR for some films and/or genre.  On the other hand, a variable frame rate approach might satisfy both desired experiences. HFR is most effective for wide shots and vista panoramas whereas a lower frame rate might be used for close ups and other tighter shots. Regardless of the genre of a movie, if a filmmaker uses variable frame rate creatively the movie going audience will likely to be more accepting of the enhanced visuals. 

Whether it's cinematic storytelling or cinematic realism I believe both have legitimacy in filmmaking and exhibition. It really comes down to a creative call for the filmmaker and a budgetary call for the studio.

In Part 3 of Cinematic Realism Considered we'll look at the relative contribution of 3D to the UHD and HFR equation and discuss cinematic realism within the context of the 'uncanny valley'.

Cinematic Realism Considered - Part 1

Cinematic Realism Considered - Part 1

There are some directors in Hollywood who have declared, a priori that the end game of feature filmmaking is to create an experience for the audience that is as close to reality as possible.  Toward that end, there has been a great deal of talk recently about the introduction of high frame rates (HFR), ultra high resolution (UHR) and more effective 3D, preferably without glasses to create an experience so close to reality that the lines between cinema and real life become not just blurred but invisible.

The Hobbit:

When people were exposed to Peter Jackson’s 48 frames per second (fps) 3D release of The Hobbit there was a fairly significant negative reaction. Exhibited at twice the normal rate of 24 fps, the lack of motion blur was unsettling to many moviegoers as was the enhanced clarity of the image.   The increased frame rate delivered more visual information to the audience per unit time but it also exposed a good deal of the movie making magic to the moviegoers, particularly the makeup and consumes.  Indeed, the heightened visuals apparently even influenced some audience members' reaction to the acting in a negative way.

Does this mean that HFR is something to avoid in filmmaking and theatrical exhibition?  Of course it doesn’t, it simply means that Peter Jackson pioneered a new paradigm in filmmaking that will over time achieve greater maturity and acceptance for both the filmmaker and the audience.

Several weeks ago I was at the International Broadcast Conference (IBC) in Amsterdam and went to a special 48 fps screening of the new trailer from the next Hobbit, Desolation of Smaug. I must admit that it was a unique visual experience for me but certainly not one that I would consider negative. In fact I was very impressed.  The only time that my suspended disbelief was disrupted is when a foreground character rapidly traversed the screen from left to right in front of the subject of focus. That character appeared to have no motion blur so it seemed to me part of the production crew walking by… and there in lies the problem. We, as moviegoers have a set of expectations that have been established since the first movie we ever sat through and much of those expectations are violated when watching a feature film at twice the speed that we consider normal. Is that bad? Well yes and no. Yes, in that it can take you out of the moment if you're not used to it. No, in that once the movie going audience is acclimated to this uniquely different cinematic experience they undoubtedly will learn to accept it.

Variable Frame Rates Will Likely Be The Norm:

Today most 3D converted films are of exceptional quality but the Industry has regrettably not explored or experimented with more diverse genre for the medium.  Instead we seem to have settled on a narrow sample that many creative and production people in the Industry consider ‘appropriate’ for 3D. Unfortunately the same fate could happen to HFR in that it might be considered appropriate for a subclass of films. Fortunately, the effectiveness of HFR appears to be  shot specific rather than genre specific. Consequently selected shots like wide-angle vistas will be sped up to 48, 60, or even 120 fps while close-ups might be exhibited at slower frame rates, etc.  I'm hopeful that variable frame rates will make the HFR applicable to a wide range of genres.

I’m not going to make a case for or against the latest trend toward cinematic realism nor would I ever criticize advances in film production and/or projection technology regardless of the stated end goal. However, there are questions that this movement toward realism evokes that go to the very heart and definition of cinema.

What exactly is the ideal cinematic experience? That question will be addressed in:

Cinematic Realism Considered – Part 2.