Autostereo (Glasses Free) 3DTV In 2014: Sizzle or Fizzle?

Autostereo (Glasses-Free) 3D

Is 10 Years Away and Always Will Be?

That phrase caught my attention in a Variety article last January published by Valentina Valentini following a less than stellar debut of autostereo devices at the 2013 Consumer Electronic Show (CES) in Las Vegas. It reflects the thinking among many professionals 'in the know' within the 3D industry, but certainly not by all. Will the story be any different at CES 2014? With the mammoth Vegas convention rapidly approaching in less than a month I thought this would be a good time to take a look at where things stand in the world of glasses-free 3D.

The Holy Grail of 3D TV:

First, I should express my bias. I consider today’s passive 3D HDTVs ideal for achieving a very high quality stereo experience and wearing polarized glasses doesn't bother me at all. However, the consumer electronics industry and the media in general seem to have determined that the only way to make 3D welcome in the home is to get rid of the 3D glasses all together. Indeed, the amount of attention devoted to 4K or ultra high definition (UHD) TV at CES 2013 was seen as a less than positive forecast for 3D. In fact many proponents of 3D consider the inordinant amount of attention paid to UHD a wake up call that there needs to be greater attention devoted to making 3D more 'consumer friendly'. And there's little doubt that consumer friendly means no 3D glasses.

Unfortunately that perception has resulted in consumers being caught up in the industry hype that autostereo TVs are at best, already here or at worse, imminent.  As a consequence some have delayed their 3DTV purchases in anticipation of the ‘new generation’ of glasses-free TVs rather than taking advantage of the exceptional quality that HD and UHD passive 3D systems deliver today, albeit with glasses.

The Technology Behind Glasses Free Display Devices:

Currently glasses free 3D display devices are dominated by lenticular and parallax barrier technology, a variation of which has been around for over a century. Prior to being applied to these devices most of us have experienced the technology in novelty items and postcards. Sometimes called flicker or wiggle pictures, the cards are printed with two images on one sheet.  When the cards are wiggled horizontally they display a simple animation between the two images or alternately expose a second, entirely different image. In its more subtle form, this lenticular technique can simulate parallax, producing a fairly high quality static stereo image. This is accomplished by precisely placing a corrugated screen over a stereo pair that has been sliced into alternating strips.  Reflected light off of the alternating striped images is refracted via the corrugated ridges to the viewers corresponding left and right eyes.

This works fine for print signage such as point of purchase advertising as well as for other static displays such as theater lobby posters.  However, the technology requires viewers to position themselves precisely within a sweet spot in front of the printed image so that the corrugated ridges accurately refract reflected light to the appropriate eye. A moviegoer attracted to a lenticular lobby poster quickly understands that they must position themselves to an optimal location directly in front of and at a particular distance from the image to fully appreciate the 3D effect; a place we call the sweetspot. If they move to either side of the sweet spot they experience a 'dead zone' where the image becomes extremely confused and uncomfortable. 

While this limitation might be considered a marketing advantage for autostereo signage because of its attention grabbing potential, the same viewing restrictions are a significant disadvantage when applied to autostereo HDTVs that have emerged over the past three years. Lenticular and parallax barrier HDTV systems are similar in concept to the print technology described above in that they typically involve slicing stereo pairs into alternating strips.

However, in the case of 3DTV, the refracted light is not reflected off of the stereo pair strips.  Instead the light is generated from a backlight behind the LCD screen and a second LCD screen, imbedded with micro lenses is overlaid in such a manner that the pixels of the source image line up precisely with the lenses in the overlay. It’s these micro lenses that act to refract the light from each pixel of the underlying stereo pair to the appropriate left or right eye of the observer, producing the illusion of parallax without glasses.  In the case of parallax barrier systems, micro slits are added into the screen to mitigate cross talk between the refracted light, producing what some consider a higher quality stereo experience than lenticular alone.  However, regardless which autostereo display technology one uses, the viewing angle or sweet spot is a serious limitation.  If the viewer moves their head vertically or to one side or another, the parallax effect falls apart entirely.

Autostereo Mobile Devices:

When applied to single user devices like smart phones, tablets and even laptops, the sweet spot limitation is not as much of an issue because the user typically positions their head in front of the display and can move the device backward and forward from their eyes to get an optimized stereo effect. I’ve seen some decent quality autostereo video and images on smart phones such as the discontinued HTC 3D phone and I've seen adequate autostereo on a handful of tablets that are just making their debut in the market.

However it's clear that the technology has a long way to go before it can be considered a solid consumer product. One improvement on single user autostereo devices is the addition of eye tracking that takes advantage of front facing cameras. Eye tracking has the advantage of adjusting the viewing sweet spot based on the location of the viewer's head.  These devices can achieve a good parallax illusion for a single person within a viewing angle of as much as 170 degrees. However, apparently the eye tracking approach will not work with current parallax barrier systems because the micro barriers are a limitation in manipulating the sweet spot for wider viewing angles. 

Autostereo HDTV:

In the case of autostereo 3DTV where multiple people would typically watch a movie together in a family room, the process of converting stereo pairs that can be refracted through micro lenses and barriers is the same as described above for single user devices.  However, with more than one viewer there is a requirement for multiple stereo pairs to be displayed on the screen simultaneously.  While this allows for several viewers to watch 3D on the same TV screen, each person still must locate and then sit within their specific sweet spot.  If they get up from their seat or move their head too far  to either side, the illusion of parallax not only goes away but there is often a reversal of left and right eye projection making further viewing very uncomfortable.  On top of this, the use of as many as 9 or more stereo pairs being displayed simultaneously significantly reduces the overall resolution of the experience. I believe that attempts to improve lenticular and parallax barrier systems for TV viewing have been largely unsuccessful.  4K systems partially satisfy the resolution issue from displaying multiple images to accommodate several viewers, but the sweet spot limitation is still very much evident.

A Promising Variation of This Technology Was Demonstrated at CES 2013:

Dolby and Phillips have been promoting a 4K autostereo TV that some have called lenticular technology on steroids. It has greater pixel density due to the higher resolution and it uses lenticular lenses that I’m told are extremely high precision, down to the micron.

Instead of having 9 to 16 views, this technology accommodates 28 views which Dolby and Phillips claim mitigates dead spots and ghosting.  However, in reality the system actually has many more sweet spots and dead spots than other systems but because of the greater number of views being displayed, the transitions between each sweet spot are smaller and therefore less bothersome.  In addition, I'm told that this lenticular device uses an algorithm that further reduces the negative visual impact of the dead zones between sweet spots. The Dolby/Phillips system seems to be a step in the right direction but I believe it requires a great deal of improvement before it can become a consumer commodity. The two companies claim that in addition to planned algorithmic improvements to their lenticular system, an 8K resolution TV will eventually deliver 56 views, further limiting the dead zones between the sweet spots. Of course that product is very much in the future.

Will this be the autostereo solution? In Valentina Valentini's CES 2013 Variety article she quotes Lenny Lipton, the godfather of 3D who, reflecting on Dolby's autostereo display said, "the best I've ever seen, but he added "I'll be damned if I know whether the public will accept it." 

So Where Do We Go From Here?

Scientists at NHK (Japanese Broadcasting Corporation) consider 8K the maximum spatial resolution that 2D TV will ultimately achieve. They claim that anything above 8K produces deminishing returns because the added resolution is imperceptable to the human eye.

They speculate that once 8K 2D TV has reached an installed base, 3D will become the highest priority in the advancement of TV technology. Those evangelizing autostereo TV have expressed hope that 8K will provide the necessary spatial resolution to host 56 or more sweetspots.  Will that do the trick? We'll have to wait and see.  I wouldn't be surprised if one or more of the major manufacturers debuts such an autostereo prototype at CES 2014.

If they do, I suggest asking them to show the same content next to a current passive HD 3DTV. Then you can compare both the quality and accuracy of the stereo between the two devices. To become successful, in addition to tackling reduced dead zones the device has to display accurate stereo information, something that has been sorely lacking in the current line up of lenticular and parallax barrier displays.

Conclusion:

It's quite apparent that Ultra High Definition will continue to be one of the hottest buzzwords for the consumer electronics industry this year and it will likely become a driving force among many of the largest TV manufacturers.  However I firmly believe that the industry as a whole will come to understand that 3D and UHD are not mutually exclusive.  Instead, I see 3D technology essential to the near term adoption of UHD and vice versa. The two technologies are in a sense, mutually beneficial killer apps.

So if you're planning to attend CES in January, I don't think you'll find a UHD solution to the autostereo puzzle. Near term, autostereo technology will likely progress along the lines of the Dolby/Phillips approach.  No one will be happier than me if they or others succeed in creating an autostereo experience that is equal in quality to the 3DTVs currently available to consumers.  If we don't see that level of quality on the horizon, autostereo technology might have to take on a uniquely different technological paradigm if glasses free 3D TV in the home is going to become a viable consumer product.  I'll be attending CES this year and hope to explore the latest and greatest in autostereo systems.  If I find that developments in autostereo technology have exceeded my expectations I will discuss those findings in a future blog post.

In the meantime, I'll be content wearing my passive 3D glasses at home, watching absolutely stunning 3D on the latest TV displays whether they be HD or UHD.

Cinematic Realism Considered - Part 3

Cinematic Realism Considered - Part 3

Cinematic Realism Considered - Parts 1 and 2 focused on the digital revolution that's been ongoing within the feature film Industry for over a decade. Digital Cinema has eliminated most of the constraints on moviemakers that analog film imposed on the Industry for almost 100 years.  This lack of constraints has created uncharted territory for the filmmaker, turning high frame rate (HFR) or temporal resolution and ultra high definition (UHD) or spatial resolution into creative tools that are only now undergoing study and experimentation.

The Hobbit has shown us that high frame rate can change the look and feel of a film from a low resolution cinematic story to high resolution cinematic realism that is intended to be an accurate representation of reality.  There's little doubt that over time, both UHD and HFR will undergo a considerable amount of refinement as moviemakers gain a better understanding of these new creative tools and how best to use them.  For instance, we've seen that HFR works best on wide shots but when a shot is character oriented in tight compositions, HFR can look somewhat uncomfortable.  In fact, legendary filmmaker and visual effects pioneer, Douglas Trumbull has stated that HFR is probably not appropriate for period or dialog driven films.  With HFR and UHD much of the movie making magic such as props, makeup and even acting undergoes audience scrutiny that if improperly composed and lensed can take moviegoers out of their sense of suspended disbelief.

What Role Does 3D Play in This Equation?

Is 3D somehow different than temporal or spatial resolution as a creative tool or is it a key ingredient for pushing the movie going experience over the top, from the realm of cinematic story telling to cinematic realism?

I believe that 3D is consistent with both experiences. In a 3D movie our brain actively creates a unique visual data set resulting from the horizontal displacement or disparity of the images projected onto our left and right retinae regardless of the temporal or spatial resolution. Our conscious perception of that unique data set is called stereopsis.

Of course, data from 2D images projected onto our retinas via HFR and/or UHD is also processed by our visual system but in a different manner.  In a 2D movie experience, temporal and/or spatial information hitting our left and right retinae are precisely identical in every way and void of any horizontal disparity. Consequently the brain will not transform HFR and UHD qualitatively into a new and different data set and therefore our perception of that visual data is not significantly different from the source.

It is well established in neuroscience that disparity resulting from the separation of our two eyes has a profound effect on the brain that is uniquely different than what is experienced with one eye.  In my blog, Engaged In 2D and Immersed in 3D, I point out that the lack of disparity in a 2D movie actually forces the audience to perceive that all the action is being presented on the movie screen and only on the movie screen, mitigating any significant semblance of immersion whether the movie is presented at 24fps or 48fps.  On the other hand, the horizontal displacement of images hitting our retinae due to the separation of our left and right eyes creates a sense of depth and volume that brings the action from the screen to the audience in a very personal manner that can infringe on each audience member's personal space... i.e. their comfort zone.  In fact, because our individual self-perceptions and personal spaces are so variable, no two people in the audience of a 3D movie will have the same subjective reaction to the same stereo visuals.  Unlike UHD and HFR, a 3D movie tears down the second, third and fourth wall of the cinematic experience, enveloping the entire theater in a unique manner for each audience member. And while the effectiveness of 3D to enhance a story is independent of HFR and UHD the opposite is not true. It's generally agreed that HFR and UHD are largely dependent on 3D to achieve the most convincing sense of cinematic realism and immersion.

Cinematic Realism and the Uncanny Valley:

Those filmmakers attempting to create an accurate sense of cinematic realism face a daunting task because any errors in the simulation of reality on the screen, no matter how slight and seemingly irrelevant will likely be noticed by the audience.  That's because the human brain has been hard wired through evolution to anticipate a world that visually follows a set of rules.

When those rules are broken a phenomenon called the  'uncanny valley' is often the result. The uncanny valley was first coined by the robotics professor Masahiro Mori in 1970 to describe how we perceive humanoid constructs like puppets, cartoon characters, robots, clowns, etc that are clearly not intended to approach reality in appearance relative to how we perceive puppets, robots and cartoon characters that are intended to accurately simulate reality.

The closer that these artificial constructs come to appear as accurate replications of reality/realism the more our sense of empathy increases. However, as these human approximations reach a point closest to realism, visual nuances of imperfection can cause us to become uncomfortable with the image. Indeed, we often find even minutely flawed human simulations as grotesque.

I believe, the uncanny valley applies to many forms of cinematic realism and it may explain why the Hobbit at 48 fps had mixed reviews.  Most of the time our brains can automatically fill in missing data and/or accept errors in low-resolution data. However if realism is clearly the end goal of a filmmaker then every detail of that simulated reality must be void of visual errors or the opposite effect will be achieved. It's because HD or 2K projected at 24 frames per second (even in the presence of 3D) is a lower resolution experience than HFR and UHD that audiences tends to be much more forgiving when it comes to inaccuracies, intentional cheats, etc in filming.

One thing is for certain, the lack of technical constrains from digital cinema will greatly complicate the art of moviemaking in the future. Some cinematographers and directors will likely embrace the creative options this new era in filmmaking presents while others will avoid or reject them in favor of what they consider traditional approaches to storytelling.

Regardless of how the Industry approaches these advancements in filmmaking one thing is for certain... It's going to get very interesting. 

Cinematic Realism Considered - Part 2

Cinematic Realism Considered - Part 2

In Part 1 of CinematicRealism Considered, I discussed both ultra high definition (UHD) and high frame rate (HFR) as signaling the latest and most radical revolution in cinema technology in the last half century. These two recent advances in movie making and theatrical exhibition together with the advent of digital 3D can create a movie going experience that blurs the separation between fantasy and reality, fiction from non-fiction. Indeed the expressed purpose of these three technologies in combination is to simulate reality as closely as possible... creating cinematic realism.

To be clear, none of these advances would have been possible without the introduction and current ubiquitous status of digital filmmaking and exhibition. Digital Cinema became a disruptive technology only during the last decade, completely changing the way we think about moviemaking and removing most of the constraints that have been inherent in the use of physical film for almost a century. 

As a result, filmmakers like Peter Jackson have begun pushing the boundaries of this new medium in an attempt to dramatically change the movie going experience, proclaiming cinematic realism the future of cinema. However, to some, these changes call into question the very heart and definition of the cinematic experience.

Is There an Ideal Cinematic Experience?

Why do we go to the theater in the first place? I believe that most of us go to the theater to escape reality via visuals and audio that depict a story. But as camera, cinematic projection and screen technology continue to advance one has to wonder whether the movie going experience will continue to offer an escape from reality or is it destined to become an accurate simulation of reality framed within a story. This is an important distinction because we know from recent HFR releases that there are some movie goers who consider cinematic realism a radical if not uncomfortable departure from the familiar cinematic experience. 

The Digital Revolution in Cinema:

But lets back up a bit and put HFR in historical perspective.  In the absence of significant advances in digital camera and digital projection technology over the last decade this blog on cinematic realism would have little relevance.

Today the majority of movie theaters around the world are fully digital. In fact, digital technology has for the most part completely eliminated analog film for theatrical exhibition. This has had a profound effect on film printing and film distribution, which was the cash cow of the major film labs. Today, filmmaking and exhibition has entered a whole new era in which postproduction houses must compete with smaller, more agile finishing houses and VFX facilities must gear up with software and capex requirements to handle higher resolutions and higher frames rates. 

The 24 Frame Rate Standard:

Prior to the introduction of digital cinema and for nearly the past 90 years, the standard frame rate in the U.S. for exhibiting feature films has been 24 frames per second (fps). To the theater going audience today and in the past, the 24 fps experience is what cinema has been and what it's supposed to be.

So what is it about 24 fps that makes it so unique?  Nothing really. The 24 fps standard was an arbitrary number introduced to feature films in 1926 because it was considered the slowest film speed that could produce the illusion of smooth movement when coupled with audio while at the same time conserving the cost of film.  While 24 frames remains the current standard in digital cinema today, both analog film and digital projectors are typically double flashed or triple flashed meaning that each frame is shown twice or three times as they are projected. This is done to reduce the transition time between frames thereby mitigating flicker. So while we've always experienced 48 or 72 frames per second in the theater, only 24 frames of actual visual information has been displayed each second. I believe audiences tend to perceive today’s feature film experience as something other than reality because of the relatively low resolution that we get from 24 fps delivered in HD or 2K. This is true whether or not 3D is part of the equation. 

Deviating From The Norm:

To deviate from 24 fps can impose a different experience on the audience that may not be considered an enhancement for some films. Both HFR and UHD dramatically increase the amount of visual data delivered to our eyes and brain. High frame rate delivers more images per second, which increases the amount of visual information hitting our retinas per unit of time.  This is called 'temporal resolution' or time based resolution. Ultra high definition on the other hand increases the pixel resolution of each frame. It's going from HD at 1920 by 1080 to UHD at 3840 by 2160, which is 4 times the resolution.  It's basically the difference between a 2.07 megapixel image and an 8.29 megapixel image. This is called 'spatial resolution’.

It's our persistence of vision or the amount of time an image remains on the retina after removal of that image that melds 48 fps into a continuous visual experience with high data rate.  So if you combine higher temporal resolution with higher spatial resolution the amount of data reaching the retina goes up geometrically and when shot correctly, the screen literally appears to be approaching reality. Add quality 3D to this mix and you have a unique cinematic experience that appears to be an extension of reality.  

So it's easy to understand that 24 fps at 1920 x 1080 or 2K is, by comparison a low-resolution experience that delivers a 'dreamy' state clearly separate from reality. Some filmmakers and moviegoers will undoubtedly prefer lower resolution over UHD and HFR for some films and/or genre.  On the other hand, a variable frame rate approach might satisfy both desired experiences. HFR is most effective for wide shots and vista panoramas whereas a lower frame rate might be used for close ups and other tighter shots. Regardless of the genre of a movie, if a filmmaker uses variable frame rate creatively the movie going audience will likely to be more accepting of the enhanced visuals. 

Whether it's cinematic storytelling or cinematic realism I believe both have legitimacy in filmmaking and exhibition. It really comes down to a creative call for the filmmaker and a budgetary call for the studio.

In Part 3 of Cinematic Realism Considered we'll look at the relative contribution of 3D to the UHD and HFR equation and discuss cinematic realism within the context of the 'uncanny valley'.