Tuesday, June 26, 2012

The Third Dimension, Higher Resolution and Higher Frame Rates, Oh My!

How Advances in Digital Film Making and Exhibition
Are Destined to Radically Alter Your Movie Going Experience

The feature film industry is entering a new era of technological advances that are challenging many of the old ways of thinking about filmmaking.  More significant, these advances promise to dramatically change the expectations and experiences of movie going audiences.  Some technological changes are responsible for the emergence of contemporary high-quality 3D movies while other innovations have been triggered by the popularity of 3D movies.

Until the last decade, little has changed in the manner in which feature films are made nor how feature films are exhibited to audiences.  Film formats have come and gone and color became ubiquitous in the U.S. during the late 60’s. However, in the last decade we’ve seen what is arguably the most significant advancement in feature film technology.  Sometimes referred to as the “Digital Age of Cinema,” the early 2000’s saw Hollywood accepting the fact that film may no longer be the dominant medium for making and exhibiting movies, paving the way for digital cinema to become the default standard.

The Transition to Digital Was Not Universally Accepted
Yes, the dawn of digital cinema marked the end of an era in film-making. I remember in the late ‘90s when a few film purists, archivists and directors bemoaned the fact that the rich texture of film grain and the ‘breathing’ of film as it passed through the projection shutters gives a feature film an organic feel that should never be lost.  They considered it an essential part of the art of film-making.

 Indeed to some, it was part of the art back in the day. Today—with the use of digital cameras, the digital intermediate process and digital cinema exhibition—the nuances of natural film grain and film weave are gone.  When grain is desired, it is typically a visual effect rather than a natural aspect of the medium.

The Digital Intermediate
Of course, many directors still shoot on film stock. When Legend3D converted the majority of Transformers: Dark of the Moon 2D-to-3D shots, we accommodated Michael Bay’s preference for shooting actors on anamorphic 35mm film.  Since film is a single camera, 2D process there was no other way to inter-cut those shots within the 3D film without meticulously executed 2D to 3D conversion of each frame of the film. There are many other directors who, like Michael, prefer the look and feel of film stock for certain type of shots. But regardless, how images are captured, the common denominator today and one of the most significant advances in the history of post-production is the Digital Intermediate (DI) process where all forms of media within a feature film are converted to a digital conform.  Whether a title is shot on film stock, shot with digital cameras, created entirely in CG or converted from 2D-to-3D, DI has become the norm in post-production and as far as feature film exhibitors are concerned 35mm distribution prints are all but being replaced by digital projection systems worldwide.

Digital Film Production Makes 3D Possible.
Most contemporary creative and technical filmmakers have come to accept the inevitable; “digital is here to stay.”  Actually, I don’t think there are many legacy creative types who miss the tactile sensation of celluloid or polyester running through their hands, or the technical skills and patience that accompany the use of splicers and glue and the periscopic experience of viewing edits on a Moviola or on the later flatbeds.   

Those days are largely gone and today’s film professionals are more comfortable with a keyboard and a mouse.  In addition to all the superior visual effects, compositing, digital color grading, etc. that DI has given us, one of the most important advantages of digital film production is that it has ushered in the age of stereo theatrical exhibition.  Without digital projection, 3D would have an insurmountable hill to climb to gain acceptance.  As recently as two years ago, there were attempts to employ analog film for 3D exhibition, but those attempts were met with criticism that analog 3D film projection—even though it was not the old fashioned red/blue or cyan anaglyph—represented a step backward. Analog film formats for stereo, often called “over and under” are still around though most professionals do not consider it a lasting format.

Higher Spatial Resolution
Due to the life and death implications of digital radiology not to mention the huge revenue opportunities of that industry to the medical community, the giant radiology equipment manufacturers like Siemens, GE, etc have historically partnered with the major teaching hospitals around the country, allocating enormous R&D budgets in an effort to push the state of the art as far as conceivably possible.   As a result, medical imaging has always been considered the pinnacle of imaging quality and resolution demands.  That is until now.  Today, digital cinema is considering the introduction of spatial resolution and dynamic range solutions for digital cinema that actually come close to exceeding the gold standard of medical imaging, diagnostic mammography. Before my entry into visual effects, my focus was on neurobiological and medical research and my career spanned the pioneering eras of neuroscience and digital medical imaging.  In fact medical imaging was the original foundation of the Legend3D process and I frequently refer back to that technology when assessing current and future trends in feature film capture and exhibition.  Back in the 80’s and 90’s the Holy Grail of diagnostic medical imaging was 4K, 10bits for mammography.  That was valued as the closest approximation to analog x-ray film and was the lowest resolution considered optimal for the clinical detection of minute calcifications and early stage cancerous breast anomalies.  However, the capture, display and storage technology at the time was not sufficiently advanced to handle that kind of data throughput.  Today mammography remains in the 2K to 4K range with some scanning sensors delivering up to 6K at 14 bits.   However, the 4K standard remains the minimum resolution for much of diagnostic radiology.  It took a long time for diagnostic radiology to move up to 4K and it’s an even bigger step for digital cinema. Today filmmakers have 8K cameras in their toolbox as well as super resolution scanners that subsequently down-sample to 4K. Some people call some of the higher resolution solutions above 4K “faux resolution” because the full capture resolution is not realized on the output but instead is used to calculate a lower resolution of 4K image.  However, it’s been demonstrated that down sampling does appear to improve resolution as well as contrast and will undoubtedly be a giant step for theatrical exhibition.  However, getting these advances installed world wide will be a significant task because the vast majority of theaters around the world currently exhibit feature films at a maximum resolution of 2K, so to exhibit 4K movies there has to be an upgrade in projectors and also an upgrade in servers due to the added bandwidth.  The real benefit of higher spatial resolution is the ability to install much larger screens.  However, once again we have a cost benefit ratio that the exhibitors will have to tackle and one that does not necessarily come out positive on the side of further advancement.

Interest in Higher Frame Rates For 3D
There is another contribution to higher spatial resolution in feature film commonly referred to as temporal resolution. Whether scanned from film or directly captured from digital cameras, 24 frames per second creates an additional dimension of “time” which significantly increases the effective spatial resolution on the big screen; offering a perceptual resolution actually greater than 2K or 4K.  This coupled with down sampling from 8K and higher physical spatial resolutions could eventually produce super high-resolution feature film exhibition.  But it doesn’t stop there.  Temporal resolution is about to have an even greater influence on the movie going experience with the latest interest in introducing high frame rates that are well above 24 frames per second.

Recently we have been hearing a good deal about higher frame rates, particularly from Peter Jackson who is shooting The Hobbit in 3D at 48 frames per second.  

It’s well known in the Industry that 24 frames per second originally became the standard for film because 24 frames was the slowest that film could run through the projector with reasonable fidelity for the first sound on film formats from Western Electric.  It was an arbitrary number that was intended to save money on film stock yet it was fast enough to allow for sound. For an in-depth commentary on frame rates I’d like to direct you to the excellent Scott Squires’ blog, “The Effects Corner”. 

In today’s digital world we are close to no longer have restrictions on frame rate for exhibition and in addition to Peter Jackson’s use of 48 frames per second on The Hobbit there is also talk of 60 or even 120 frames per second in an attempt to give the audience a more realistic 3D visual experience.  One advantage of the higher frame rates for 3D is that it removes a great deal of motion-blur, which tends to cause visual stuttering when an object is rapidly moving horizontally across the screen.  However, one consequence of higher frame rates sometimes considered negative is the loss of grain, even artificially inserted. Some say that 48 frames per second and higher comes across as “live” video.  I believe that any objections to the look created by higher frame rates are simply due to the fact that we are not yet used to it.  Certainly, you can have an immersive 3D experience with 24 frames per second or any of the higher frame rates being proposed. I believe that in the right hands, the different frame rates above 24 frames per second can profoundly change the 3D visual experience for the better, particularly when considering advances in higher physical spatial resolution mentioned above.  How VFX pipelines will handle these higher frame rates and their massive increase in data has yet to be determined as well as how the resulting increase in bandwidth requirements will be accommodated within the Blu-ray format.  However one thing is certain— increased frame rates will create a whole new, more vivid visual experience for the movie going audience.

So will 3D, higher spatial resolution and higher frame rates become the new standard? Or should they be considered simply another part of the director’s tool-box or toy-box? Up for consideration is whether 3D is appropriate for all genres? To me, the fact that four out of five of the top-grossing movies of all time are 3D makes that a moot question. Aside from its obvious appeal to audiences, I personally, believe that proper creative execution of 3D can enhance any feature film genre. However, it’s highly questionable whether increased frame rates are appropriate for all films or film genre.  Budgetary and technical considerations aside, 3D and higher frame rates, like film grain and black and white, should probably be a creative consideration by the filmmaker and there is little doubt that many filmmakers will continue to prefer shooting and exhibiting their feature films in 2D at 24 frames per second.  As for higher spatial resolution, it certainly would seem to make sense for significantly bigger screens than we are used to at our local Cineplex, but does it add enough to warrant the additional production expense considering amount of data already being generated by 3D and higher frame rates?   With exhibitors just recovering from conversion to digital and 3D, will they see the cost benefit of the necessary build out and retrofits as well as the cost of new projectors?  I believe it is not likely for a long time.  As far as the home theater industry is concerned, we are already working with 240hz and higher frame rates in HDTV 3D sets that greatly increases the perceptual resolution but I don’t believe that higher resolution will motivate the purchase of new TVs.  In fact, until you get to 55” displays, resolution above HD is probably not a significant differentiator.  On the other hand, I believe quality 3D at higher frame rates will motivate consumers to upgrade.

As a visual effects innovator for the past 25 plus years, I’m the last person to join some critics suggesting that particular technological advancements in moviemaking and exhibition should be discouraged, avoided or not considered at all.  Regardless of the technology employed, it’s the quality of the story and the talent of the creative and technical filmmakers that make or break a movie.  If new and even radical technological advances can help filmmakers create more compelling visual stories then I say, bring it on.  Just remember to consider the enormous bandwidth and storage demands on production from the beginning to the end of their pipelines and the consequent cost to the studios, visual effects houses, conversion vendors, post production houses and last but certainly not least, the exhibitors.

I’d like to thank my colleagues, Garrett Smith and Rob Hummel for reviewing this blog and the invaluable suggestions and edits offered:

Garrett J. Smith is currently CTO of Ha Productions based in Santa Barbara. He previously served as vice president, production technology and digital mastering operations at Paramount Pictures. During his 24-year tenure at Paramount, Garrett participated in the development of DVD, HDTV and Digital Cinema Initiatives. Prior to Paramount Pictures, he worked in various post-production positions including; post-production supervisor for “Ripley’s Believe It or Not”; director, post-production for Columbia Pictures Television; and manager, film services at CBS Network Television. Garrett is a member of the Academy of Motion Picture Arts and Sciences and serves on the Science and Technology Council. He is also an associate member of the American Society of Cinematographers and an adjunct associate professor at the USC School of Cinematic Arts.

Rob Hummel is the President of Group 47, Inc. Formed to acquire and improve upon the technology behind DOTS; the advanced digital archival storage media originally developed by Eastman Kodak. Previously, Rob was President of Legend3D, the leading 2D to 3D conversion services company for major motion pictures and television.  He was the chief liaison between Legend and film studios, using innovative solutions to enable studios to convert movies into 3D efficiently and economically. He began his career as the director of production services for the Technicolor Laboratories, and then moved on to Douglas Trumbull’s visual effects company during the making of Blade Runner (1982) and to post-production work on Tron (1982).  A former president of DALSA Digital Cinema, Hummel has also served as senior vice president of production technology at Warner Bros., where he oversaw digital post-production (mastering films for digital cinema, HDTV, DVD, etc.) and digital restoration work on such classics as “Gone with the Wind” and “The Wizard of Oz.”   He previously worked in post-production, animation and Imagineering at Walt Disney Studios, headed animation technology at DreamWorks, and helped launch digital cinema units at Technicolor and Sony.   Hummel currently serves as the Chair of the Public Programs Committee of Academy of Motion Picture Arts & Sciences’ Science and Technology Council and sits on the Scientific and Technical Awards Committee.  Rob has hosted several programs at the Academy on Film Formats, Film Technology, and 3D Stereoscopic Imaging.  He is also an associate member of the American Society of Cinematographers and editor of the 8th edition of the American Cinematographer, and authored most of its articles. Rob has taught classes at the USC and the UCLA and is an Honorary Visiting Professor at the Kanazawa Institute of Technology in Japan.