Geek Gestalt

ILM visual effects and animation teams had to employ some of the latest tricks in the industry to make the new "Star Trek" film feel both realistic and true to the classic franchise. Among the biggest challenges was updating the Starship Enterprise for 2009.

(Credit: Industrial Light & Magic)

SAN FRANCISCO--For Paul Kavanagh, the animation supervisor on the new "Star Trek" movie, one technical element of the film was particularly challenging.

During live-action filming, director J.J. Abrams had done something unusual: In a bid to incorporate a shaky, handheld effect, Abrams would frequently sit behind the camera and literally tap on the back of it with his fingers. But "Star Trek" is jam-packed with computer graphics, and for Kavanagh, it was imperative to find a way to replicate the effect of that finger tapping, even in the purely digital sequences. Not to do so, he said, would have created a visual inconsistency that threatened to disrupt the audience's experience.

Back at Industrial Light & Magic, where Kavanagh works, he considered several ways to solve the problem. He talked to the people in ILM's motion-capture department, who showed him a number of 3D mo-cap cameras and techniques, but he felt those were too time-consuming and expensive.

Still, the mo-cap folks had another technology that was both simple and cheap: an orientation sensor that could be plugged into a computer with a simple USB connection and used to record motion. So Kavanagh and his animation team figured out that if they tapped on a desk while filming scenes with CG cameras--on-screen camera viewers that incorporate realistic lenses--and layered the motion from the orientation sensors underneath, they could get the same effect as Abrams got with live-action.

"J.J. did come down to visit us, and he loved it," Kavanagh recalled. "He definitely wanted the same kind of handheld look, but (what we did) was a big surprise for him. He loved that the look carried across the shots."

As you might imagine, "Star Trek" is a feast of effects and animation. According to ILM's Roger Guyett, the film's visual effects supervisor, it has a full hour of visual effects in all. "Every aspect of (the effects has) to be planned and thought through," Guyett said. "It's easy to underestimate the amount of work that goes into creating" an entire world.

Yet Abrams wanted a very tactile feel to the movie, Guyett said, and that meant filming as much as possible and adding in visual effects, rather than relying entirely on CG. "It was closer to the model of the original 'Star Wars' movie"--building actual sets that audiences can react to--"not filling in all the blanks (with CG) later on."

For example, when considering how to create a shuttle hangar, Abrams decided he wanted an actual set, rather than crafting it digitally. That meant finding a suitable space and then lighting it to match the look and feel of the rest of the film.

One benefit of that, Guyett said, is that it helped the actors to have a real set to work on, because they had to imagine less. "You've got actual wind blowing in your face," he said, rather than having to act like there's wind.

For Guyett and his team, another big challenge was figuring out how to handle a massive amount of destruction in the film.

For example, he said, they had to bring photo-realism to the way two colliding spaceships would fall apart. But the physics involved in something like that happening in space are far different than they would be inside the Earth's atmosphere. Similarly, the team needed to figure out how to realistically show what the explosion from a missile hitting the Enterprise would look like.

"The rules of physics aren't the same" in outer space, Guyett said. "Explosions behave in a different way."

Making the physics of an explosion in space look right was no easy task. But Guyett said one of the biggest advantages of working at ILM is that the company is rife with "geniuses" who he can consult with on just about any kind of scientific conundrum.

"You can e-mail a guy," Guyett said, "and say, 'When a ship explodes in space, what actually happens?'"

Then, because of ILM's latest tools--which accurately model the way gravity, or the lack of it, would affect an explosion in space--the filmmakers can find a way to make it look as close as possible to what the in-house science experts say it should.

Guyett explained that ILM's computers allow teams like his to simulate happenings like a nuclear explosion on film and not have it be prohibitively expensive. Just four or five years ago, he said, such a thing wouldn't have been possible. As an example, he said that creating a crash sequence in "Men in Black" had been very expensive because it involved breaking up a costly model. On top of that, they'd had only one chance at getting the shot. But back then, he added, doing it in CG wouldn't have worked because the technology didn't yet exist to get the physics right.

Another challenge, Guyett said, was finding a way to update iconic "Star Trek" elements for a 2009 film without upsetting hard-core Trekkies.

For example, he said that he and Abrams had labored endlessly to try to create a transporter effect. "It's a very iconic thing in the 'Star Trek' world," Guyett said. "It's a sound that everyone knows."

One problem they had to solve was that the transporter ended up looking different on each of the different sets were used in the film. "So we'd just have to adjust it (each time)," Guyett said. "The seemingly smaller challenges can take the longest to figure out."

In animating the new "Star Trek" film, animation supervisor Paul Kavanagh crafted a unique hybrid team of animators interested in camera work and camera department people interested in animation. Each member of the group would be given responsibility for working on individual shots.

(Credit: Industrial Light & Magic)

For animation supervisor Kavanagh, working on "Star Trek" presented the chance to do something he'd never done before: create a single working group of animators interested in camera work and people from the camera department interested in animation, and let individuals take responsibility for individual shots.

"We haven't tried that before at ILM," Kavanagh said.

He explained that for his eventual team, "Star Trek" was start-to-finish crunch time. They had to work on 860 shots in less than six months, and sometimes Abrams would toss in wild cards by deciding to change the story during sequences, and ask the animation department to do their own pre-visualization, something the director is usually in charge of.

In the past, it would have taken too much time, but because Kavanagh had created his hybrid working group, they were up to the task. "The benefits that came from it is that we came up with new camera techniques for all-CG shots," he said.

One of Kavanagh's favorite sequences is one in which Captain Kirk is banished to an ice planet and ends up in a battle with a beast known as a polarilla.

Crafted in CG and meant to be a hybrid of a polar bear and a gorilla, the polarilla was the animation team's responsibility, and Kavanagh said it was up to them to find a way to both breathe life into the creature and give it character.

He said they did a number of animation tests on the polarilla, trying to find the best creatures to base it on from a series of reference sources, including the BBC's Motion Gallery, YouTube, and visits to the San Francisco Zoo. In the end, they decided it would run like a polar bear, but have the rear quarters and hanging knuckles of a gorilla. It would also feature the weight of a grizzly bear.

In the sequence, however, they had to animate another creature, known as Big Red, a lobster/crab hybrid that jumps up through the ice to challenge the polarilla for the chance to attack Kirk.

Big Red "was fantastically fun to animate," Kavanagh said of the beast, which has 120 eyes in the back of its head.

As the chase sequence evolves, he recalled, they had to figure out how Big Red would reach out to grab Kirk's leg, as spelled out in the script. But because the creature's mouth was "so long," the animation team felt it didn't work to have it grab Kirk with its arm.

"We thought, what if its tongue is what grabs Kirk's leg?" Kavanagh said. "We had to figure out how that creatively looks. And that's really the fun part of the job."

They decided to have it slip and slide, Kavanagh said, but no so much "that it looks comical.

It seems that in the end, that was a challenge that both Guyett's visual effects team and Kavanagh's hybrid animation team had to tackle. But in updating "Star Trek" for 2009, will true Trekkies recognize the latest iteration of the franchise?

Judging by the mostly enthusiastic reviews, the answer seems to be yes. But Guyett's less interested in reviews than whether he did his job.

"Oh yeah," he said. "There are nods to the history of the series, what has happened and what will happen....But we just made it contemporary."

On June 22, Geek Gestalt will kick off Road Trip 2009. After driving more than 12,000 miles in the Pacific Northwest, the Southwest and the Southeast over the last three years, I'll be looking for the best in technology, science, military, nature, aviation and more in Colorado, Utah, Idaho, Wyoming, Montana and South and North Dakota. If you have a suggestion for someplace to visit, drop me a line. And in the meantime, join the Road Trip 2009 Facebook page and follow my Twitter feed.

While wearing a motion-capture suit, CNET reporter Kara Tsuboi shows how her movements are translated instantly to an 'Iron Man' character on the screen behind her. The technique is used in an increasing number of films to mix live-action footage with digital, 3D sets.

(Credit: Daniel Terdiman/CNET News)

SAN FRANCISCO--On Tuesday, the DVD version of the mega-hit film Iron Man will be released, and to celebrate, the visual effects superstars at Industrial Light & Magic decided to show off just a little bit more magic behind the movie.

Back in April, ILM invited me and a couple of my colleagues to their fantastic facilities here for a look at the technology behind the famous suit used in Iron Man. Recently, they invited us back to see how the seamless animation in some of the film's scenes--such as one famous shot involving the throwing of an Audi--was produced.

In particular, they wanted to give us the inside scoop on the motion-capture technology used to create a number of the film's scenes, a technology that is increasingly being used today that allows directors to see, in real time, while the actors are acting, what animated sequences will look like.

That's why we--myself, CNET reporter Kara Tsuboi, and a cameraman--spent several hours on an ILM image capture stage last week: So that Tsuboi could don a motion-capture suit and we could all see how footage of her would translate instantly into an animated Iron Man scene.

The idea is that George Lucas--who owns the effects studio--wants to give filmmakers advanced technological tools that provide them with flexibility and efficiency. And so he staffs ILM with the kinds of people who can make that happen.

"We understand the entire process," said , "from writing code to animating creatures to even shooting live elements. So we know each layer in the process. We understand the vision of the key creatives and understand" what the actors are going to do.

The technology used at ILM--and elsewhere, as well--allows directors to mix real filmmaking and virtual spaces, but with full camera control, depth of field, tracking, and panning. The upshot? A filmmaker can have an entire digital set created, then have an actor perform on the image capture stage wearing the motion-capture suit, and see, as the filming is happening, how the actor's character looks superimposed on the digital background.

This poster of Iron Man flying is from a scene that mixes computer graphics--the Iron Man character--with real footage of a cloud-filled sky.

(Credit: Daniel Terdiman/CNET News.com)

When the visual effects and animation wizards at Industrial Light & Magic started working on Paramount Pictures film Iron Man, their biggest challenge was creating a suit for the title character that was part CGI (computer-generated imagery) and part real costumery.

If you're not familiar with Iron Man, it's the story of Tony Stark, a genius billionaire industrialist who's also a bit of a jerk and who designs and sells weapons. In the film's opening sequences, Stark is demonstrating one of his weapons in some unspecified country near Afghanistan when he is captured by terrorists who demand that he craft a weapon for them. In the scuffle that ensues, he ends up wounded, with shrapnel lodged near his heart.

To make a long story short, Stark ends up making an iron full-body suit that protects him and his wounded heart, and along the way, he ends up going through a personality transplant and becoming a superhero instead of a force for evil.

But it all comes back to Iron Man's suit--a technical marvel that allows him to fly, shoot missiles, be impervious to many conventional forms of attack, and more. To watch video from the film, you see that the suit has no end of little flaps and compartments that all seem to operate independently and which are all essential to giving the Iron Man character his fully teched-out superhero flavor.

The problem was, according to ILM visual effects supervisor Ben Snow, that the traditional options for creating a suit like Iron Man's--either making it fully CGI or making it a fully real-life, physical, or "practical," suit--weren't going to work for this film.

"The suit has to do a lot more than just a suit of armor can do," Snow said when I visited ILM recently.

To begin with, the producers commissioned a practical suit for Iron Man, something that lead actor Robert Downey Jr. would have to wear on the set. But as the production advanced, the visual effects team became more confident with how they could influence how the suit was integrated into the film--and Snow and his team began to make more and more suggestions of how to blend CGI with the practical suit.

"One of the fun things for ILM," Snow said, "was that we got to contribute to the ideas of what makes the suit tick."

At ILM, the visual effects and animations teams were tasked with finding ways to make it hard to tell when a scene is mainly computer graphics and when it's real footage.

(Credit: Daniel Terdiman/CNET News.com)

That meant, Snow added, that the film's director, Jon Favreau, encouraged him and his team to get creative in the ways they planned to blend CGI with the practical suit Downey wore on the set.

One issue that made it difficult to rely too much on the practical suit was that it was heavy and therefore a burden for Downey to wear. The weight of the real suit made it hard for the actor to do the full range of motions that the script called for.

So as the team figured out ways to do more and more elements of the suit with computer graphics, Downey and others who wear iron suits in the film began to discover that they didn't always have to be wearing so much weight.

"When the actors and the stunts realized they could pull parts off (of) the suit," Snow said, "they were trying to take parts off all the time."

The solution? Snow and his team began to offer joking bribes to the actors not to remove the pieces.

"We'd offer them half the cost of doing the scene in CG," he joked, "and we'd pocket the other half."

Still, the job required a great deal of CG--but graphics work that had to be painstakingly accurate.

"The hardest part about this," Snow said, "(is) sometimes the real guy is inside (the suit, so) you have to make sure it moves the same way as the real guy."

Favreau wanted realism in the CG work, Snow added, and didn't want the CG work to make Iron Man's motions look too "martial arts."

But after some time, it seemed like the visual effects and animation specialists were getting really good at what they were doing. So good, in fact, that sometimes when the dailies came in, the lines between where the real suit ended and the CGI began wasn't clear.

"At some point," Snow said, "Jon Favreau started asking questions about the suit (thinking it was real). At this point, we realized that they'd stopped being able to tell what was CG and what was real and that was really great. Jon Favreau rang me up and said, 'Look, I've got a problem with this shot. It's very CG looking. Can you do something about it?' I said, 'Jon, that's actually a practical suit. It's not CG.'"

That was certainly an important moment for everyone because, Snow said, Favreau wanted things in the film to look as realistic as possible.

For example, in a widely-seen sequence in the film in which Iron Man is flying through the air, pursued by fighter jets, most of the action in the foreground is CG. But in order to get the clouds to come across as authentic, Favreau used real footage of clouds taken from an airplane above them. And then Snow and his team blended the two elements.

ILM Animation supervisor Hal Hickel, who won an Oscar for his work on 'Pirates of the Caribbean: Dead Man's Chest,' talks about his work on 'Iron Man.'

(Credit: Daniel Terdiman/CNET News.com)

While Snow and his team were working on the visual effects of Iron Man's suit, ILM animation supervisor Hal Hickel--who won an Oscar for visual effects for his work on Pirates of the Caribbean: Dead Man's Chest--was dealing with things like figuring out how to make Iron Man's flying motions feel right to the audience.

"It ended up being really challenging, the flying," Hickel said, "figuring out how he should fly. Iron Man takes off slowly like a heavy object, and then lands very fast because he weighs a lot."

Another big challenge was working out the size scale differences in fighting sequences between Iron Man--a normal human-size--and his enemy, Iron Monger, who is meant to be 15 feet tall.

But the scale difference was too great to do with live action, Hickel said.

"We backed the camera off and reconstructed the set digitally," he said.

Another big challenge for Hickel and his team was "trying to remind the audience that there's a man in (the suit) and that he's in jeopardy."

That meant trying to figure out ways to show Downey's face, even when he is in the suit.

"We tried to have little performance nuances that signaled to you that he's not a robot," Hickel said, adding that one way of achieving that was to film Downey in a motion capture suit and incorporating that footage into the animation process.

"It was a great help," Hickel said, "and he was excited that he didn't have to wear the full practical armored suit."

ILM's John Knoll and Hal Hickel were nominated for the best visual effects Oscar for their work on Pirates of the Caribbean: At World's End, the third film in the Pirates franchise. Knoll and Hickel have been nominated for each of the three films, and they won the Oscar last year.

(Credit: Industrial Light & Magic)

As you probably have heard by now, it's Oscar nominations day, and Hollywood is all a-buzz.

But up here in the Bay Area, there are some pretty happy people as well.

That's because the folks at Lucasfilm divisions Industrial Light & Magic and Skywalker Sound were honored with a total of six nominations Tuesday.

ILM was nominated for its visual effects work on both Transformers and Pirates of the Caribbean: At World's End.

ILM's John Knoll and Hal Hickel, who were nominated for their work on both of the two previous Pirates films, and who won the Oscar last year, were nominated again for the third Pirates movie.

And ILM visual effects supervisor Scott Farrar, animation supervisor Scott Benza, and associate visual effects supervisor Russell Earl were nominated for their visual effects work on Transformers.

ILM animation supervisor Scott Benza and associate visual effects supervisor Russell Earl earned their first Academy Award visual effects nominations for their work on Transformers. Visual effects supervisor Scott Farrar was also nominated for Transformers, his fifth overall nomination. He previously won the Oscar for his work on Cocoon.

(Credit: Industrial Light & Magic)

Over at Skywalker Sound, Randy Thom was nominated for best sound editing and best sound mixing for his work on Pixar's Ratatouille, which was also nominated for best animated feature. In addition, Michael Semanick, a two-time Oscar winner, was nominated for sound mixing and previous Oscar winner Michael Silvers earned a sound editing nomination.

Skywalker Sound's Randy Thom, Michael Semanick and Michael Silvers earned Oscar nominations for best sound editing and best sound mixing for their work on Pixar Animation's Ratatouille. The film also was nominated for best animated feature.

(Credit: Pixar)

Finally, Skywalker Sound's Matthew Wood also got a best sound editing nomination, as did two-time Oscar winner Ethan Van der Ryn for his work on Transformers.