This
month, the multiple 3D films being recognized by the 84th Academy Awards cement
the fact that 3D is here to stay. On the heels of the Best Visual Effects
nominations for Hugo and Transformers: Dark of the Moon, Legend3D Founder,
President, CCO/CTO Barry Sandrew dissects the appeal of 3D for the moviegoer in
this two-part blog series on the neuroscience of 2D and 3D feature films.
Relying on our current understanding of stereopsis, as described in scientific
literature, Sandrew contemplates how our brains differentially perceive images
and activate emotional responses when viewing a 2D movie versus a 3D movie.
The group dynamics of
the 2D experience
When
we attend a 2D movie, the group dynamics of the audience come into play as
we direct our collective attention as observers to the images on the screen.
Our personal space is infringed upon only tangentially by the other audience
members that are sitting on either side of us, as well as by those behind and
in front of us. The tangential influence upon us of the surrounding audience is
a reminder that we are in a theater with strangers, all watching images being
projected on the screen. However, in a 2D movie, our fellow audience members typically
do not detract from the experience. In
fact, the audience in a 2D movie is intended to be an integral part of the total
feature film experience and has been presented as one of the selling points of
attending a 2D move in a theater rather than watching it on DVD or Blu-ray at home.
As we become engaged in the movie, we can introspectively sense surprise,
anger, sadness and happiness based on the perceived action on the screen and to
a lesser degree, through the reactions of the audience members around us. We
can empathize and sympathize with the characters being portrayed and we can
experience anxiety and fear as we follow the storyline.
Disparity—A key element in the perception of stereo
Our
brains receive 2D information in a theater as images presented sequentially at
24 frames per second. In a 2D theater experience, motion is perceived as
changes in the up, down and sideways position of one object in relation to
other 2D objects. We perceive three
dimensions in a 2D movie via single eye or monocular cues such as occlusion, motion,
shading, size, parallax, texture gradient, perspective, saturation and
brightness. However, lacking in a 2D movie is the most powerful 3D influence on
the brain—disparity. Due to the separation of our eyes and the distance between
them, we see a slightly different image from each eye. This separation causes a horizontal
displacement between the images, which is referred to as disparity. The best
way to test this is to put your thumb in front of your eyes and alternately close
one eye and then the other. You will see two separate perspectives of your
thumb displaced horizontally and your thumb will appear to jump from side to
side. When these separate images from our left and right eye reach our visual
cortex, they are directed to highly specialized neurons that are tuned to
critical parameters of disparity. As a
consequence, the two images are fused into one uniquely different image that
exhibits the appearance of depth and volume. The result is 3D perception.
A lack of disparity
contradicts monocular cues
In
a 2D movie, each of our eyes receives precisely identical images from the movie
screen with no horizontal displacement. When the two sets of identical images
reach the visual cortex of our brain, they are directed primarily to neurons
tuned to zero disparity. There, they are “fused” together into a single image
that is exactly identical to each of the two images that originally comprised
it. Other than the existence of monocular stereo cues, mentioned above, the
audience does not perceive depth and volume due to the absence of disparity. This
lack of disparity in a 2D movie creates a cue conflict situation that prevents
us from being fully immersed in the story in the same manner as a 3D movie. The
monocular cues in a 2D movie are telling us there is depth, but the lack of
disparity contradicts the monocular cues—forcing the brain to try and reconcile
the conflict.
The absence of
disparity “pushes” the screen away from us
For
some monocular cues, like motion and occlusion, there is a further complication
that acts to distance us from the movie screen. As mentioned above, in a 2D movie
the relative motion of objects on the screen (one 2D object moving relative to and
possibly occluding another 2D object) can indicate depth in the absence of
disparity. However, in the real world when there is this kind of relative
motion in the absence of disparity, our brains normally interpret that
information as indicating that the objects are far away. That’s because
disparity in human vision drops off significantly at distances of several
hundred yards. You can prove this if you
try the thumb trick above, alternately closing one eye and then the other, but
this time, try the trick looking at a tree that is several hundred yards away. You’ll see that the tree will not appear to
jump from side to side like your thumb did. This is because the perspectives
from our two eyes at that distance are identical. Our eyes do not converge on
the tree, but rather are set to infinity.
The father away things are from us the less horizontal displacement we
experience. So our brain “comes up with”
a solution in which we are literally “distanced” from the action on the screen.
This distancing and lack of disparity in a 2D movie renders the spatial areas
in front of and behind the theater screen irrelevant to the story and therefore
irrelevant to each member of the audience. The storytelling, in its entirety,
happens on the white screen at the end of the theater.
Skillful
use of disparity in a 3D movie will always enhance the experience
In spite of the contradictions our
brain must reconcile, a 2D movie is almost always comfortable to watch and with
proper directing, cinematography and storytelling, it can evoke very strong emotions.
Indeed, many of us remember the outpouring of emotions while watching the
sinking of the Titanic or the
building terror of T-Rex in Jurassic Park,
as both movie experiences were presented in 2D. However, the influence of
disparity on the audience in a 3D movie cannot be trivialized as a fad or
unnecessary. Whether a 3D movie is captured with two cameras or converted from
2D-to-3D, disparity that is properly and skillfully stereographed will always
enhance the storytelling experience by significantly amplifying the responses
of disparity-tuned neurons and therefore more closely simulating reality.
3D
a
higher resolution medium
In
the visual system, the absolute number of neurons activated in the brain is not
as significant as the ratio of activation between subpopulations of neurons
responding to zero disparity and those responding to a wide spectrum of tuned disparities,
including zero. This is analogous to an
audio amplifier where the ratio between subpopulations of audio frequencies can
vary and increasing the gain on the system increases the overall fidelity of
the audio experience, meaning it becomes closer to the original source in
resolution. In the same manner, the ratio
of activated zero disparity and disparity-tuned subpopulations of neurons in a
3D movie can vary, but it’s the intensity of the relative responses of those subpopulations
that create a more accurate and higher resolution stereo image. It appears that our brains interpret this
higher resolution information as more closely simulating reality (“fidelity”) and
therefore it immerses us in the movie to a greater degree, creating a
heightened emotional investment in the story.
This is, of course, an oversimplification of an exquisitely complex process
that has evolved from the earliest primates. However, the theater going
experience is something we can all relate to and many of us recognize the
profound difference between being engaged
in a 2D movie versus being immersed
in a 3D movie.
3D movies are a uniquely
personal experience
Dynamic
disparity (i.e. the relative amount of disparity, changes in disparity over
time and the rate of disparity change) transforms the movie screen from a
projection screen into a window that has both an interior and an exterior.
Consequently, the storytelling environment is projected throughout the entire theater,
actively becoming a unique part of the personal space of each member of the
audience and resulting in the space both in front of and behind the screen
becoming an integral part of the story. For example, an object that might fly
out of the screen at us, traveling through what is called negative parallax,
becomes very personal for each member of the audience because it affects and/or
“intrudes” each audience members’ personal space equally. Surround sound has an analogous effect in
that we are “in it” rather than simply listening to stereo audio in front of us. As a result, unlike the 2D movie experience,
a 3D movie is more personal and the group dynamics of the larger audience are
no longer in play to the same extent. In
fact, it’s my opinion that 3D glasses actually have the positive effect of helping
to separate us from the other audience members, containing the experience as a
more personal one for each of us, thereby potentially amplifying the immersive
nature of the experience.
The Bottom Line
When
we are engaged in a 2D movie, we are doing so as passive observers, watching
the story take place “over there”, on the screen in front of us. However, when
we are immersed in a 3D movie, we are doing so as active participants and the
action can be happening behind and in front of us—as far as infinity. This is
where the central difference lies. We cannot discount the uniquely active
physical and emotional reactions that we have in the 3D feature film experience
which are not experienced in our more passive reception of 2D movies. I believe,
this is one of the reasons why we have seen 3D continue to flourish and be
adopted so enthusiastically by both national and international movie exhibitors,
as well as by the entertainment and consumer electronics industries at large.
Consumers and moviegoers are hungry for visuals that offer a significantly greater
sense of engagement and for visuals that spark an immersive, emotional response
that has the ability to transport them into the heart of the action. This year, we will have the opportunity to
test the differential 2D and 3D theatrical experiences with three iconic movies
that were originally released in 2D, being released in 3D. Top Gun has been converted by Legend3D and will be re-released on its 25th anniversary by
Paramount on the heels of 3D converted re-releases of Titanic and Star Wars. I am confident that all three films will be
very successful at the box office, as each was already a proven success in 2D. Depending
on how skillfully each of these films was converted, now in 3D they will give
those who saw them originally in 2D an immersive and refreshingly unique
experience; an experience that will hopefully help to solidify 3D as an
essential part of filmmaking.
Next,
we will continue to look at the neuroscience of 3D movies and further explore
the uniquely tuned structures and neurons in the brain that respond selectively
to dynamic disparity. We’ll look at concepts of visual processing that remain
contentious within the scientific community, involving differential pathways
within the brain that signal separately “where something is” and “what
something is.” The evolutionary survival value of binocular vision will be
discussed in the context of 3D movies and we will touch upon more primitive
structures in the brain that are likely triggered by disparity to elicit
powerful physical emotions and ‘flight’ or ‘fight’ reactions. For more
detailed information and technical reviews on stereopsis, as well as in-depth
information on the uniquely tuned disparity neurons in many parts of the brain,
please see the following references:
(1) Cumming
BG, De Angeles GC. (2001). The Physiology
of Stereopsis. Annual Review of Neuroscience. 24, 203-238.
(2) Howard,
IP, Rogers, BJ. (1995). Binocular Vision
and Stereopsis. New York: Oxford University Press.
(3) Poggio,
GF, Poggio, T. (1984). The Analysis of
Stereopsis. Annual Review of Neuroscience. 7, 379-412.
(4)
Born, R., Bradley, D. (2005). Structure
and Function of Visual Area MT. Annual Review of Neuroscience. 28, 157-189.
I wish to acknowledge
the review and insightful suggestions of my neuroscience colleague, David
Heeger, Ph.D., Professor at New York University (http://www.cns.nyu.edu/~david/) where he is a member
of the Center for Brain Imaging. The son of Nobel laureate and chemist, Alan J.
Heeger, David Heeger is a contemporary neuroscientist who has been at the
forefront in the field of functional magnetic resonance imaging (fMRI). Dr. Heeger
was the first to bring together two separate and largely unrelated disciplines—cognitive
neuroscience and film studies, opening
the way for the exciting new interdisciplinary field of “Neurocinematic” Studies.
