Master of conversions: Legend3D’s Barry Sandrew goes deep
Aug 18, 2014
'The Lego Movie' is one of this year's hits that turned to Legend3D.
The Hollywood career of Dr. Barry Sandrew, who founded 3D-conversion and visual-effects giant Legend3D, moves from black-and-white to color and now to state-of-the-art 3D, as if mimicking the evolution of the movie industry itself.
Carlsbad, Calif.-based Legend3D, leveraging Sandrew’s many patents and inventions, specializes in 2D-to-3D conversion technology and services. Working at the cutting edge, the company has been integral to bringing that additional third dimension to many features over the past five or six years, including such recent and current international blockbusters as Transformers: Age of Extinction, The Amazing Spider-Man 2,The Lego Movie and Maleficent. Older sparkling jewels in the Legend crown include Hugo and Life of Pi.
Sandrew is also Legend’s chief technology officer and chief creative officer. Indicative of his importance in the 3D tech world, he was inducted last March into the International 3D & Advanced Imaging Society board of governors. The organization is the 3D industry’s largest and most important group. It represents the interests and goals of the 3D business internationally with a mission to advance the creative arts and sciences of stereoscopic 3D and 4K. (Disney, DreamWorks, Sony, Paramount and IMAX are just a few of the Society’s big-name founders.)
Sandrew’s journey to 3D is Hollywood-worthy. A PhD neuroscientist, he invented Legend’s proprietary 3D software after more erudite beginnings as an East Coast academic on staffs at Harvard, Boston’s famed Massachusetts General Hospital, and Columbia’s College of Physicians and Surgeons.
His career first took a hairpin plot turn in the late ’80s that even a Hollywood script couldn’t imagine: He devised the proprietary software for the controversial but jaw-dropping colorization process and made an extraordinary leap to the West Coast to build it. His inventions for Hollywood emerged from work done from 1978 to 1987 at Harvard Mass General Hospital. There he established neuroscience labs including an imaging lab centered on MRI, CAT and PET scans of the brain.
Although at the hot center of the 3D frenzy powering blockbusters worldwide, Sandrew remains more soft-spoken scientist than Hollywood player. “I’m immersed in science, but left [Boston] in 1987 to come to California to invent colorization,” he recounts. “Back then, some entrepreneurs approached me who wanted to digitally colorize black-and-white movies. The colorization process at that time was analog, producing extremely poor color quality. They understood that if you colorize a public-domain film, the colorized version is eligible for a new copyright that they would own. They knew about me and my medical imaging work, which is how I was brought into the colorization business.”
Out West, Sandrew founded American Film Technologies (AFT) and served as chief technology officer as inventor and patent-holder of the first all-digital colorization process that was used to colorize more than 250 motion pictures, including “essentially all of [Ted] Turner’s work.” The process most memorably brightened (literally) Ted Turner’s cable channels but frightened, even horrified, film purists. Deemed by many as tampering with an auteur’s vision, colorization faced a considerable backlash, but Sandrew—Our Man from Academe—weathered the storm: “I was naive about all [the controversy] because I was from such a different field, so it didn’t affect me. The irony is that the more critics complained, the more successful colorization became.”
After leaving AFT, Sandrew was a founder of Lightspan, Inc. in 1993, one of the largest educational software companies in the U.S. selling K-6 “edutainment” DVDs meeting all state standards to entire school districts. Then came Legend Films, which he founded in 2000 when he reinvented the colorization process with digital tweaks made possible by significant advances in computer technology.
Says Sandrew, “I invented colorization in the early ’90s and was approached in 2000 by a friend, Jeff Yapp, formerly president of Fox International Home Video, who wanted me to invent colorization again but in a more refined manner producing far superior color using the latest technology. So I invented a new process for colorization and we did a lot of work on public-domain and major studio catalog titles. Then in 2006 when my technology partner Greg Passmore showed me 3D technology for the first time, I was blown away and I immediately had a sense of where it was going. So I diverted my R&D away from colorization to 3D.”
The advanced colorization work, in fact, enabled the platform from which he created Legend3D’s proprietary 3D conversion software. Simply put, the jiggling of pixels enables both platforms. Colorization and 3D processes are, in fact, related and indeed “similar in the early stages.”
Legend was renamed Legend3D in 2010 when, again empowered by his proprietary software, Sandrew shifted gears from colorization to 2D-to-3D conversion.
The majority of Legend3D clients are Hollywood studios and their first-run features. But things old could be new again. Legend’s proprietary 3D conversion makes it possible to convert any 2D film to 3D, including the hundreds or maybe thousands of musty or newer features that sit in distributors’ libraries and comprise their backlist. If content is king, Legend3D’s software could enable any feature to re-enter the marketplace as 3D royalty.
But such a possibility raises the question of what kinds of films—what genres—are appropriate for conversion, assuming the 3D conversions would make these films more appealing to audiences. Asked this inevitable question, Sandrew ventures, “We don’t really know yet which are best for the format, because to date the industry has been focused on limited genres and maybe just about anything will be enhanced with 3D. Look at [the 3D conversion of] Top Gun, which is really more of a love story than anything and it’s great in 3D.” He adds that he’s a fan of catalog titles and believes that many franchises like the Matrix, Pirates of the Caribbean and Harry Potter films would work as conversions. In fact and no surprise, Sandrew doesn’t name any film or kinds of films that shouldn’t be considered good candidates for 3D conversion.
Mainly, the company is after the big fish: “Our sales force,” says Sandrew, “is focused on the new studio films emerging, not so much the catalog titles.”
Legend plays well with IMAX: “In Transformers [Age of Extinction], Michael Bay used many digital camera formats including IMAX and we had to match the IMAX look, but there was no problem. IMAX is not an issue for us al all.”
Sandrew cites IMAX as a company that is aggressively moving toward leading-edge display technologies. “They were already delivering the brightness required for theatre audiences and now they’re installing laser projectors that will improve the theatre experience even more. This is happening in China and South America and in five years will be ubiquitous.”
According to Sandrew, Legend has a “very limited number of competitors” and, like the competition, the company finds its clients through “a very short list. We all know what films are coming out, so we’re all bidding against each other on the same titles. We’re often brought in at the end after the 2D shoot.”
While all company projects involve Sandrew’s proprietary software for the conversion of 2D material to 3D, work varies according to whether features are “native” 3D films (begun as 3D camera-captured productions) like Hugo and the first Amazing Spider-Man, “hybrids” (a combo of material captured during production as either 2D or 3D) like Transformers: Age of Extinction, or full 3D conversions of original 2D films like The Amazing Spider-Man 2, Man of Steel and Top Gun.
Without understanding the mechanics, many of us take 3D for granted, already knowing that each eye captures a slightly different perspective or angle that the brain next processes into a single image. Or is that it? Sandrew helps give a little more, er, depth to the 3D magic:
As he explains it, stereophonic, so familiar to music lovers, enables an individual to hear two audio feeds but experience these as one. Similarly, stereoscopic is the underlying quality of 3D, the experiencing of two visual feeds as one. The eyes converge to see the stereoscopic effect of two distinct visual captures so that the actual perception of 3D only happens inside the brain. The eyes do see two different objects, but the brain takes over creating one additionally endowed with depth, volume and solidity. In 3D, the whole is definitely greater than the sum of its parts.
If perception is at the heart of 3D, misperception is at the heart of the notion that 3D popularity is sagging, Sandrew contends. These notions are floated in the media, with blame placed on fickle audiences, ticket upcharges, inadequate screen brightness or even the productions themselves.
“[Declining interest] is more misperception than reality,” he continues, reminding that 3D grosses have hit an all-time high and that of the top 15 performing films of all time, ten were in 3D, and that 2013 was a stellar year when 13 of the 15 top-grossing films were 3D. He predicts that this year and 2015 will mark the biggest performances ever of 3D product.
And not to be forgotten, he adds, is that “while 3D continues to grow here, it is even much more popular internationally, especially in China.”
Sandrew continues: “‘Jaded’ is a good word for what might be a cooling of audience enthusiasm, but look at China, where the upcharge and glasses are signs of a premium experience instead of what some in North America deem an annoyance. But in other parts of the world, it’s that coveted premium experience.”
But 3D is not just for theatres. 3D TV has hit the home front, although not yet claiming any victory. Sandrew is also a big believer in 3D TV for couch potatoes and sports-bar fans and does not buy into the perception that 3D at home is “dead in the water.” The stalled status of 3D for smaller screens doesn’t bother him: “The consumer-electronics industry introduced 3D TV too fast and in too grand a fashion—consumers weren’t ready for it. But today it’s difficult to purchase a high-end flat-screen TV that is not 3D-enabled. So, as people buy 4K TVs, the installed base for 3D TV will grow. Once these are in homes and the high dynamic range is available, people will use them [for 3D viewing].”
Technology keeps roaring along, but exhibitors too can get active and need not just be vigilant on the sidelines awaiting the latest and greatest. They have options today, says Sandrew, for improving the cinematic experience for their audiences. Eventually, he notes, 4K will be further enhanced by the introduction of high dynamic range (HDR) both in and out of the home. “At NAB 2014, the mantra was: We don’t want more pixels, we want better pixels, brighter pixels with a greater color gamut which more precisely captures what the eye sees.”
But, he cautions, “these advances will ultimately make TV a superior viewing experience than what audiences will be able to get in theatres.” He reminds that “unlike IMAX, traditional theatre screens are typically not bright enough and 3D glasses further darken the movie screen.” But as home viewing improves, he speculates, “this will force exhibitors to upgrade to laser projectors which raise the resolution bar, improve brightness and introduce high dynamic range. Right now, theatres can offer filmgoers passive glasses and make sure the venues are clean. Perhaps number one is cranking up the projector bulbs to a minimum of six foot-lamberts so that the images get brighter. They now tend to be dull.”
Filmmakers too are doing their part by "being less shy about 3D,” he says. “In the early days, there was the belief that pushing 3D would make movies appear too gimmicky, but look how well 3D works with so many films. The Amazing Spider-Man 2 is a good example of how far 3D can be pushed without appearing gimmicky.”
As for that all-important question of when 3D viewing without glasses might be possible for cinema audiences, Sandrew suggests that “glasses-free is somewhere out there but not imminent. It will eventually happen, but not with the technology that exists now.”
Taking a more in-depth look at Sandrew’s conversion technology, it comprises three steps: separating the different elements (or planes) in a shot, giving these depth and volume, and painting in the gaps between the objects and the backgrounds from which they were pulled forward. Rotoscoping (manipulating individual frames) is common in the first step, separating the elements, but Sandrew also applies his pixel-specific process developed from the company’s colorization work, which entails his creative staff determining a sampling of pixels in specific objects to be separated. The tedious frame-by-frame work is then outsourced to one of Legend3D’s India facilities.
Early last year, Legend moved into 60,000-square-foot state-of-the-art studio headquarters in Carlsbad and the company recently announced the opening of their new 3D L.A. Hub in Hollywood, which Sandrew describes as “a one-stop finishing solution for the stereo conform and post needs of our clients. We’ll be state-of-the-art with our color-grading capabilities.”
Known in-house as the Legend3D Hub, the facility will offer a superior screening facility and allow filmmakers to review their 3D conversions in continuity on the latest RealD screen technology, and do conversion adjustments in real time. The Hub, still awaiting its formal name, will be equipped with a full editorial infrastructure and a 10-gig pipeline to the Carlsbad headquarters.
So 3D is thriving, but wait a second—this is 2014 and technology advances every second. When talking 3D and immersive images, the subject of holograms inevitably arises. Might these ever become part of the film production or conversion process? “A year ago I put it in the sci-fi category,” says Sandrew, “but I have associates who have worked out some interesting possibilities. Maybe it’s ten years off before that technology can be done, but there’s some potential.”
Let viewers and the industry beware: Blade Runner, 2001: A Space Odyssey and Dick Tracy were also once in “the sci-fi category."
