Trevor Hogg chats with production visual effects supervisor James Madigan; visual effects supervisors Bill George, Thad Beier, Ollie Rankin, Vince Cirelli, Mark Larranaga, and Zachary Kinnery; executive visual effects supervisor Payam Shohadai, visual effects producer Steven Swanson, and digital production manager Michael Perdew about bringing a gritty realism to G.I. Joe: Retaliation. Beware there are spoilers...
At the end of making Red (2010), producer Lorenzo di Bonaventura (Stardust) presented his visual effects supervisor for the cinematic adaptation with a script intended to be the sequel to G.I. Joe: The Rise of Cobra (2009). “I loved it,” recalls James Madigan (Iron Man 2) who went on to have conservations with the producers and Paramount about G.I. Joe: Retaliation (2013). “The first question I asked them was, ‘Being this is a franchise is the look and feel of the first Joe which made the film a bit cartoony something we’re looking to repeat because it feels like this one would be well served by a rough and tumble, Black Hawk Down [2001], and The Hurt Locker [2009] feel to it.’ Everyone jumped out of their seats and said, ‘Exactly.’ Right from that point I liked the idea of where they wanted to take it and when Jon Chu [Justin Bieber: Never Say Never] was brought on board I was brought in to have a meeting with him and we clicked. We were throwing out ideas that were very much on the same page. G.I. Joe is one of those things that have not so much fantastical elements but next generation gadgets. These guys are capable of impossible feats but we were trying to ground the physics of that and the feel of the CG in a world that was heavier and rougher.”
Different eras of the iconic toy action figure were blended together in the second instalment. “Before the whole Cobra and the 6 inch dolls there was the 12 inch doll which had the kung fu grip and the real hair,” says James Madigan. “If you’re only into that set of toys you’re in a straightforward military movie; there’s a lot you can do with that but it’s a different feel. When you’re brought into the later 1980s comic book side of it there’s a lot of fun that you can have. I love how Jon wanted to pull that into a real world thing as though this is a takeover that happens, Cobra is like a special forces and the country gets duped into the thing happening. That was all interesting because it took the fun aspects of it and grounded them in a place that wasn’t too over-the-top or fantastical. There are a few sequences in this that I saw right away which were the trickiest in terms of jumping the shark, that could be over-the-top or too impossible if we didn’t ground it as much as we can in real world physics. One of the main ones was the scene in the Himalayas with the Ninjas.” The original screenplay idea for the mountain cliff battle is not what appears in the final cut of the movie. “Lorenzo is a rock climber and he hooked me and Jon up with a good friend named Paul Borne [The Matrix Reloaded] who does this kind of stuff for real. We researched how that would actually go. Obviously, the things they’re doing are over-the-top but we wanted to make sure that the speeds we were travelling at were somewhat relevant for something you could do for real.”
“Jon is amazing,” remarks James Madigan. “I have never worked with anybody who didn’t matter day, night, weekends, if he was in the middle of a crazy situation on-set shooting, if someone had a question for him you got his time. We would get together rough edits and show him things. I know at one point I handed something over to him at 10 o’clock at night right before I left work and when I came in the next day Jon was like, ‘I want to show you something.’ He had taken the edit and put in speed ramps, music and sound through the whole thing. I was like, ‘When did you do this?’ His assistant said, ‘He was up until four in the morning.’ Jon was showing it to me at eight in the morning. The guy was like a machine and very creative.” The man behind the camera is familiar with the latest technological tools. “He used to sit on-set with his iPhone. We would be shooting five or six takes of each set up and while they would be lighting he would say to the video playback, ‘Let me have take two of this set up and take four of this set up.’ Jon would film them with his phone and then with iMovie he would do a little cut and go, ‘Okay. That works. Let’s move on.’” Madigan notes, “It is the first big visual effects movie he has done.” The inexperience was not a hindrance because the director who is better known for helming dance films is a quick study and multi-faceted. “Those are the movies Jon wound up getting while waiting for other things. I wouldn’t classify him as a guy who is limited to that at all. He does do that stuff because he has a passion for that type of creativity. Jon was straight in there with the actors and in order to get a movie shot if you don’t have the trust of the actors it is all going to fall apart.”
“The way I felt about Joe in general was that we didn’t want to have the look and feel necessarily from the first one,” states James Madigan when discussing how he chose the various visual effects vendors for the project. “The biggest opportunity we could wind-up having that CG feel would have been the Himalayas. So right off the bat I was, ‘Guys, we can’t mess around with this sequence. We want to make sure that we’re going to one of the top places.’ ILM [Industrial Light & Magic] did that sequence. I didn’t think we could get them budgetary and be able to afford the rest of the movie. We started slowly and bid everything out to a lot of companies. As you know in the visual effects world some companies were coming 20 per cent or double or four times higher than other facilities and there are reasons for that. Some facilities are more expensive and others are cheaper. There were some sequences which weren’t as difficult we made sure we saved money on those. I’ve got some in-house guys who can churn through stuff well that helped us to save some money on some other things. ILM was great. They wanted to work with us and came to the table with something we could work with. My visual effects producer Pablo Molles [The Incredible Hulk] hammered out a deal with them.”
“There was a lot of bidding and planning done at ILM before I came on the show,” states ILM Visual Effects Supervisor Bill George. “The studio knew that the Himalayan sequence was going to be one of the ‘key’ sequences in the film and one that had a number of technical challenges. I think the studio felt that the ‘casting’ of that sequence was good for ILM. There were discussions about the project and its budget that went on up until the very last minute. I found out about the project on a Tuesday, it awarded Friday and I was on a plane to New Orleans on Sunday.” The vertigo inducing ninja fight was carefully prepared. “Jim and his crew had been working for a year on the previs for the shot design of the Himalayan sequence. The plan had already been formulated as far as how the green screen elements would be lit and shot, so it was very clear to me from the beginning what the intention of the shots was. Jim was there while we were shooting the green screen elements on stage to make sure we didn’t stray too far from the intended design. Since Jim knew the sequence design better than anyone he was the go-to guy for questions. In post-production Jim was a great wing man for me since he was down in LA working with Jon and had access to him.”
“I spent about two weeks in a helicopter up in B.C. at 11,000 feet filming all of the backgrounds for that,” explains James Madigan. “It could have become the end Underwater Sequence in Joe 1 where it is 100 per cent CG and people in front of green screens. I pushed to say, ‘We want to do a shoot in the mountains.’ The visual effects supervisor had the support of his producer and director. “We made it CG where we absolutely had to but what was great was that we did that first. Another thing I wanted to stay away from was too much digital doubles. I had broke down all of the shots so that we could have stuntmen doing it for real and obviously some of it you can’t do. When we got into the CG side of it I had real footage of real people at least attempting to do it. We could refer back to it and keep it grounded in something that a human being could do or at least attempt to do. Where you see the moments where they get a bit clumsy we tried to use that wherever we could to keep that feel. I mapped out the environment and the footage we were going to use so I knew what the backgrounds were going to be so we could work with the DP who was incredibly collaborative. Patrick Loungway [The Lone Ranger] would study the plates and give us lighting for the stuntmen who would match into it. All of these glued together to make it not have this overly video game CG look to it.”
“Most of the plates that were turned over to us were of the stunt ninjas and Joes hanging on ropes against the huge green screen,” remarks Bill George. “Our job was to place them in a digital Himalayan environment. This set-up reminded me a lot of my work on the Quidditch games from Potter 2 and 3 where all we started with was a kid on a broom against a blue screen. Working on those films certainly influenced my approach to the shots on Joe. Because of the fact that the green screen elements were shots as if they were in the shadow side of the hill, I was really worried about the shots looking very flat. I talked to my Diggimatte supervisor Johan Thorngren about how we could ‘jazz’ up the cliff. I have spent a lot of time in Yosemite; I have encountered granite faces there that are so polished they appear slightly reflective. Johan came up with a way to render a reflection pass on the geometry of our cliff face that we mixed slightly onto the painted surface. The subtle reflection played across the surface adding a lot of life to it. We also added in some wet areas that would have been caused by melting snow, where the reflection was even greater. We did match-a-mations of the characters on the wall to generate subtle shadows to help connect them to the wall when they touched it. Since we had done these match-a-mations for the shadows, we could also use them to render the character's reflections in our wet surfaces. All of these subtle cues helped to integrate our green screen characters into our shadow side of the cliff. In our backgrounds, I cheated sunlight as close to the characters as I could. The direct light adds so much dimensionality and the presence of even a small sliver of blown-out sky makes the shots feel more real. There were a lot of shots that had complex blending of live-action and full CG characters as well as some that were all digital. Our Animation director Paul Kavanagh did an amazing job keeping the movement of the characters within the real world limits but pushing them to do the impossible. In almost every shot we added camera float to make it feel as if the cameramen were hanging from ropes along with the ninjas.”
“Midway through the post-production it was decided that we destroy all of London,” states James Madigan. “People were referencing different movies. I was pointing out that they spent a year working on that shot and at that point we didn’t have very long. That was a curveball. One thing everyone was keen on was not to have it be like the usual thing that you see with a flash, shock wave, and the usual wave of destruction and mushroom cloud. Everyone was like, ‘What’s a way we could do this that is new and feeling like a different kind of weapon made this?’ We looked for a series of aerial shots of London which could go together in away that related to the story.” The capital of the United Kingdom was destroyed by incorporating the concept of plate tectonics. “My take on it was, ‘What if this is something where the weapon hit and it broke up the earth deep below it so that we had a bunch of hard solid plates which were suddenly sitting on softer ground that was all shifting out in concentric circles. The ground does not completely liquefy. You have these huge chunks of earth that still have buildings, buses, rivers, subway trains on them but they are wholesale moving up and outwards, and smashing into the next plate and crunching that up. When we started extrapolating the idea into something that was happening through the Thames, all of the bridges and parliament it turned into something which was cool.” The tight time constraints did not allow much room for error. “We threw it at ILM; five shots at total and they turned out cool. It is one of those things they would play the shot in a loop about a thousand times and you would have to watch the bottom 10 per cent corner over and over to make sure everything is okay there and then move over. We would go through the entire screen scanning especially in the final shot because there is so much going on in every single pixel. Every now and then you would be, ‘Wait a minute that building is going behind the one it is actually in front of.’ There are about a million little mattes in there. They were so much fun."
“The destruction of London in G.I. Joe: Retaliation is caused by the dropping of a man-made meteor,” explains Bill George. “When the filmmakers turned over the shots to us they explained that they didn't want to see the typical ‘nuclear blast’ type shots. They wanted to surface of the earth to ‘shatter’. We explored the idea of treating the ground plane as a thick ‘shell’ that would break like glass when the impact happens. Jon also wanted the energy from the blast to move out in a wave rather than a ring. Editorial in LA had provided to us helicopter footage over London to use as the plates. These plates were used to set the camera and the lighting, but since SO much destruction was going to occur over the course of the shots we completely replaced the plate with our digital London. Once the ground plane was constructed, Paul Kavanagh oversaw the fracturing of the surface and how it moved after the impact. We had some very specific movement we wanted to see, like some of the plates under the Thames River tilting towards us so we could see the water slide towards camera. This animation of the plates drove ALL the destruction and simulations that would follow. We broke down the buildings and bridges into foreground and background assets with the foreground models being more finely detailed and the many background ones being more procedural. We benefited greatly in the fact that many of the artists who came onto our show to work on the London destruction had just come off of Battleship [2012] and Avengers [2012]. They were working at a very high level of speed and efficiency right from the start. The work was very complex and interconnected and everyone on the team used their superpowers to make the shots look amazing.”
“The CG White House was in the script so I did a quick previs of it,” remembers James Madigan who wanted to have the scene of the Cobra soldiers raising their flag to be reminiscent of the iconic image captured at Iwo Jima during World War II. “Marketing saw it and said, ‘We’d like to have that in a month.’” Madigan initially wanted to do a miniature of the famous building but the deadline did not allow enough time. “I flew to D.C. and was going around with a location scout. It’s interesting. With all of the pictures you see of the White House that’s not by accident. They’re all from the same place because they want to funnel you into certain areas where they can keep an eye on you. To get an angle on the roof is hard to do and we were starting on the roof. I noticed this restaurant right next to it that was on a rooftop and I asked, ‘Can we go up there?’ The location scout was like, ‘I know the people up there.’ She was in touch with the secret service and I had a big camera with a 400mm lens which is a paparazzi long lens. When we went up there they said no pictures of the White House.” The viewpoint was exactly what was needed to provide ILM with the right geometry and textures. “The location scout talked to someone and they said, ‘Okay, you sound like you’re approved to take pictures.’ I picked up my camera and started taking long lens pictures of the whole roof when suddenly the manager came running in and said, ‘What are you doing? The secret service just called us and they want to know who you are and why are taking pictures?’ We were like, ‘Oh, God!’ They had to make a call. We were held there. We explained it was for a movie and they were fine with it. It was funny because there are two guys on the roof of the White House with sniper rifles and when I looked through the [zoomed in] pictures it is full framed of these guys. The first guy there is looking through his binoculars and the next one his binoculars are pointed straight at me. Within seconds they knew someone was on a roof taking pictures of them! We immediately went back. It was helpful to have it just to see the texture of the roof. How things are laid out. Where there are chimneys. We made sure that we didn’t put anything else that we saw there on the roof. I circled around it and took a lot of high resolution stills of the White House. Unfortunately, it is pretty featureless. That is the trick when trying to make CG things look real. You put little bits of weathering on it and ageing; that building is in really good shape. It is pure white.”
“The White House pull-back was created by our digital matte department,” remarks Bill George. “We had a plate that Jim had shot of the Cobra soldiers raising the flag at the head of the shot. This element was combined into our CG model of the White House. Jim had risked being thrown in jail to get us really good texture reference of the real White House with his still camera in Washington D.C. In fact if you look to the sides of the soldiers in the pull-back you will see telescopes on the roof. Security men at these telescopes ‘busted’ Jim while he was taking photos for us so we thought it was only fair to include them in our shot. The dropping cobra banners were CG cloth simulations.” The risk taken by James Madigan was worthwhile. “We always start with ‘reality’ as a baseline but as I like to say, ‘Reality is boring!’ You always want to push the limits of making a shot in a film like this as dynamic as you can without going over the line. I find that it mostly comes down to speed. When designing a previs sequence to make it exciting things tend to be very quick. Translating those 1 to 1 to shots would make them feel very false and video-gamey. We had a great advantage on this show in that the stunt performers really hung from ropes and swung through frame. This showed exactly the limits of what makes this action “real”. Right from the start though there were changes made to make things more exciting like shooting slightly undercranked to speed up the action. It was a great benefit though to know exactly what a real swing looks like.” George adds, “Although I am really happy with the London destruction scene, I am proudest of what we created for the Himalayan sequence. We used every creative trick we had to give the shots depth and contrast. Knowing what the sequence looked like when it was turned over to us, you really see the vast scale of our contribution. Both Jon and Jim seemed to have a good time directing and refining our shots and all of their feedback made the shots better. It was the type of working relationship that, as a supervisor, you really hope for.”
“Digital Domain was amazing,” says James Madigan. “DD was doing work straight out of the box version one that looked great; they’re one of the best and was able to come in really reasonable. There’s a whole sequence with HISS Tanks which are these Cobra machines that are like these giant tanks that articulate and can do all kinds of things and a lot of it had to be CG. There was a big battle that we figured out. There is this mini tank which is a real thing. You see it in the trailer attacking them and that needed a lot of work. There were a lot of shots which got made late in the day. They had already been building the satellite and then the satellite needed to do a different thing. DD did it in a manner of weeks, especially, at the end there was a set of shots, then we reordered things, and suddenly with weeks to go we had major shots they were just starting and those guys hit it so hard. Anything in space is DD. Anything with the HISS Tanks is DD. There is a bit where we have a character’s face change into another character which is a fully CG face that DD did. Those were the big tickets items. They did the Archangel. There is this special Joe plane. It’s a real thing but basically the engines turn up so it can hover like helicopter. There’s a real one in the military but it has propellers whereas this has jets. They did all of that. It was a lot of DD in London [Reliance MediaWorks] that did a good bit of it. They did really good work.”
“James was a wonderful, articulate person who had no problem telling us what he wanted,” states Digital Domain Visual Effects Supervisor Thad Beier. “He was astonishingly well-prepared; he does more planning for the visual effects work than I have seen in a lot of people. For the tank battle James had gone out there and surveyed the whole environment himself; he basically published a book which had every shot in it, where the tanks should be and what the shots should be.” Not everything went accordingly to plan as is normally the case once filming commences. “In the HISS Tank Sequence there were these giant tanks that they actually built and moved around. They were difficult to move and had limited action capabilities.” The mechanical props gave a “tremendous performance” but kept on breaking down and had to be continually rebuilt. “Because of previs we were able to take care of those situations as they came up. As far as the camera work goes the second unit director did vary quite a bit from the previs as they often do. He had the previs as a side point for the camera work but that is something the effects unit director often wants to change, and the second unit DP as well. They want to put their own look on the thing. Previs helps to figure out what the actions should be; as far as the camera moves they were more often up to the day.”
Digital HISS Tanks were utilized for the close-up shots. “In a lot of cases the CG tank is a lot more detailed than the physical one,” states Thad Beier. “The CG tanks could go much faster and move a lot better so in those cases where the tanks had to move quickly they’re all CG. The big challenges were when we were blowing up tanks. They only had one of these physical tanks and didn’t want to blow that up. They built a box of a tank out of the exact same scale out of metal and they would blow that up. We would have to put the CG tank in the middle and fire. Trying to simulate the effects of the lighting of the fire with the tank inside of it as all those surfaces would be lit was a lot of fun. You usually think about lighting things more from sources which are faraway like the sun or sky but here you’re getting the tank in the light source. It was an interesting lighting challenge to try to make it look like it would be affected by light and not lighting itself.”
“That was by far the biggest technical and artistic challenge,” remarks Thad Beier upon being asked about the brief moment where it is revealed that the leader of the United States of America has been replaced by someone else. “You see the evil Zartan president only in our few shots where he transforms. We had to make sure that he was recognizable for those four or five seconds that he is in character. We had to transform him completely from the Jonathan Pryce [Brazil] character to Arnold Vosloo [The Mummy] and back in the limited window that we had in a way that was clear to everyone watching that there is a different person under there. We shot both characters. We shot Jonathan Pryce. We shot Arnold with the same lighting conditions, place and camera moves.” Despite having matched filming conditions for both actors the face of Arnold had to be recreated digitally. “We had hoped to use some of that photography directly but in the end the motion of his performance was so different than the motion of the president there was no way to use the image directly. We ended up using it as a reference.” The physical appearance of the two performers caused complications. It is surprising that Jonathan Pryce’s head is actually bigger than Arnold’s.”
The transition between the two men had to appear physical rather than morphing. “The idea was as the president potentially injures himself, the nanomites,” explains Thad Beier, “the micro machines that make up his mask rush to heal him and when they did that they would reveal the character underneath. Once he was healed within a few seconds they would rush back to recreate the mask.” No previs was produced to orchestrate the shot. “That was a shot we did on the day as we shot it. We had several hundred visuals of that shot which were composited for only a few seconds of the movie. It was by far our most complex shot. It was a lot of fun.” The final result was accomplished without the aid of facial capture. “It came down to the last two or three weeks where I was sitting in a chair with the animator everyday going over it,” remarks James Madigan. “Faces are all about the subtleties. What you think your face would do to convey a certain thing it is not really what it does.” A lot of effort was made to avoid the ‘uncanny valley’ effect which results in lifeless eyes. “It was a lot of work of me and the animator days on end looking in a mirror filming ourselves trying to figure out exactly what we needed to do.”
The concept art for the Cobra Commander Helicopter was supplied by the Art Department for the movie. “It had some features which were like a Comanche Helicopter which was never put into production,” states Thad Beier. “We had our team in London build a [digital] model helicopter based on those drawings.” A physical replica was built and used for one shot. “The real thing defined what the look had to be. You had to match that thing exactly.” Lots of photography was taken along with LiDAR scans of the aircraft to be used as reference material. “There is one point where the helicopter crashes in the movie and there were spectacular helicopter crash footage we were looking at but in the end they changed the script so that the Cobra commander escapes and we didn’t end up using that. For a lot of the shots there was a real helicopter which was about the same size and scale as the Cobra Commander Helicopter.” The aircraft supplied essential environmental subtleties such as those caused by it hovering over the water and grass. “The Archangles were a late edition for us and we had to scramble to get those done,” recalls Thad Beier. “They were giant vertical take off and landing airplanes. We only see them at a dark night. It is one of things where what you don’t see is more important. You have to image what the shape is and let your imagination fill in everything you can’t see. We had a single drawing to work from to build it and we built it. As you move around the object the flaws in the original drawing became apparent so we had to remodel it so it could look good from all points of view. The biggest challenge with that by far was the lighting. Trying to make it look like a dark night but to be able to see everything you wanted in the Archangel. They shot the environment flying over day for night but on a cloudy day so there was no direct moonlight even and to design a balanced lighting of the Archangles.” An edge light was devised to help give definition to the black aircraft which allowed the viewer to be teased as it quickly flies by.
“The big shot where Firefly jumps the motorbike, and it breaks apart into pieces and those pieces turn into rockets which fire and explode at the prison; that was a complicated sequence indeed,” states Thad Beier. “Madigan shot the real motorcycle which was a spectacular reference to work from but as it is a slow-motion shot you got to examine it in detail.” The sequence which featured practical explosions was heavily art directed with almost every frame being separately lit. “The interaction of the pieces and explosions was a real challenge. We tried severely to get the motion of the Firefly motorcycle to work with a camera move so it looked like a natural move that could happen.” Animation Supervisor Erik Gamache was given the task of portraying the imaginary high tech vehicle as if it was something that existed. “The Firefly character our main villain for the piece had to be animated as well and had to have natural motions. Erik is a motorcycle rider and is a fanatic about motorcycles; he was engaged with the thought and decided to make it as realistic as possible. As Firefly is jumping he has to hit various controls and pull a ripcord. Erik was concerned that a motorcycle rider would see all of the motions he was doing as a reasonable thing to do. He would never let go of the handlebars at that point of the jump. He would do it here. Erik was the right guy for the job. It’s a shot that has been in every trailer for the movie and we are pleased with the way it turned out.”
Production provided the concept art for the Zeus Satellite. “Its profile in the movie grew as the movie went on. Erik and I looked at and said, ‘That looks cool. We can use that here.’ We ended up with a lot more close-up shots and shots of the satellite with the Earth behind it,” remembers Thad Beier. “Later on in the production they decided to blow up the satellite. Explosions in space are a real challenge. What do explosions in space look like? We wanted to come up with something physically plausible yet still interesting looking. The idea is the explosion in space doesn’t have an atmosphere so it keeps on growing and would never stop. It would never billow in the way explosions do on Earth. We had an idea that you would see the fuel burning and continuing to expand, and as it expands the force of it would split the satellite apart. A lot of the simulations of the satellite blowing up we had expected the need to do a lot of keyframe animation of the satellite parts breaking apart.” The simulation turned out better than expected. “We were surprised by the time the simulation was finished all hundred thousand parts were breaking apart and ripping and tumbling. It looked fabulous. We looked at it when all of the hero animation was done except for one shot of the Cobra logo coming at the camera, the explosion itself worked fabulous. Jeremy Hampton was our main simulation effects guy and did a spectacular job.” There were only a few weeks left in the project which meant there would not be many opportunities to run the giant simulations to see how they looked. It was a nice destruction of the Cobra satellite system in the end.”
Digital Domain was also responsible for the destruction of the ICBM (Intercontinental Ballistic Missile) fleet. “I was surprised when we were doing the ICBM shot that none of my younger animators were familiar with the term ICBM,” notes Thad Beier. “It surprised me how the missile fleets of the US or the Soviet Union had faded from memory to the extent that kids don’t even know what ICBM means anymore. We liked the look of the MX Peacekeeper Missile, the last ICBM ever built.” The weapon requires solid rather liquid fuel which causes to blow up in way that resembles giant fireworks. “Tens of hundreds of thousand of little particles are burning and flying away from the explosion. Again since this is in space these wouldn’t explode down the atmosphere, they would explode out and continue to move outward as the shot progressed. Jeremy Hampton our effects supervisor had a lot to simulate there. The volume that would contain this explosion would have to be large in the end but still have a lot of resolution to capture all of the details of the explosion. At the end the explosion simulation volume had 10 billion voxels in it. By far the biggest one we had ever attempted. It was a relatively realistic explosion of a solid fuel rocket and it was choreographed well.” Each explosion was positioned on the screen so to assist the storytelling as well to maximize its visual impact. “The explosions we had done low resolution simulations so we only had one chance to run the final explosion before we had to deliver.” Good fortune prevailed. “The first time we saw it everyone loved it. ‘That’s exactly right. That’s the explosion we want.’ I was pleased that it worked out.”
“Some of the work on the Archangel sequence came from a different facility and we took that over,” states Thad Beier. “Digital Domain spread the work out between three facilities. We have our group in Venice which did a lot of work. We had a group in London that did most of the tank battle sequence and in India.” In regards to maintaining a unified look, Beier remarks, “That’s absolutely the supervisor’s job to make that happen. We’re viewing things from all the facilities everyday and making sure that they’re on the same page. The producer and I flew out to London a few times to meet him on the ground and talk to him in a way you can’t do over the phone or video link or cyber.” The decision for G.I. Joe: Retaliation to be screened in 3D resulted in a minor complication as a strict back to front layer policy was not maintained. “The visual effects shots can be better if they’re broken into parts into layers and the conversion company can use the layers separately. It makes their job much easier than if they’re presented with the final composite.” The fantastical elements were handled realistically. “This movie does a lot better job of that than the first movie did. It is much more plausible. It is much more about the characters and how people react to plausible things. The viewer is engaged with the story better because nothing magical happens. These guys have to work for it given the tools that they have. They have real things to work against. The HISS Tanks are wonderfully physical objects and the Joe tank is a beautiful piece of machinery. Those are real physical things that the viewers get in engaged with and understand on a visceral level which will be a great part of this movie. They will get into it in a way than a more fantastical movie.” Jon Chu made the effort worthwhile with his positive attitude. “I’d like to say that I thought that the director provided an inspiration and an enthusiasm throughout the process that affected everybody.”
“Method did a lot of the stuff in the bunker,” states James Madigan who had the visual effects company handle the sequences involving the explosive fireflies which are part of an arsenal belonging to the villain Firefly played by Ray Stevenson (Jayne Mansfield’s Car). “He releases these little mechanical fireflies which are like small little bombs that can fly at someone who thinks they’re looking at an insect so they can get close and blow them up. It had to sell at the beginning before we first see them operate that it was like a real firefly.” The Vancouver facility for Method Studios was responsible for the inciting incident which propels the G.I. Joes into action when their base camp is attacked. “They did the whole desert battle which is all CG Apaches in the night scene. We had some special effects on the ground blowing things up. Jon wanted all the bullets being fired to be tracers and tracers are one of those things that they always wind up looking like Star Wars [1977]. Depending on your exposure, time of night, how much light, sometimes they’re yellow or red. It’s a tricky thing. They did all of that plus missiles firing with the jet trails behind them. They made that sequence hum and cranked out a lot of that stuff early on for the trailers. We need to believe that the Joes are not spooked by the approaching helicopters. ‘Oh, here comes the evac team.’ Then all of the sudden it goes horribly wrong. There was a careful way Method had the helicopters marching at us from shot to shot until their behaviour immediately changes. We had to go through the continuity of each shot. How the first wave happens. How the second wave happens. Where they are coming from? How distant they should feel when they’re firing missiles. There was a lot of technical geometry we had to work out in order for those shots to feel right. They did an amazing job with that stuff.”
“We had worked with Jim Madigan a couple of times in the past,” states Method Studios Visual Effects Supervisor Ollie Rankin. “Prior to G.I. Joe we had done work for him on Red and also he got involved with the tail end of Man on a Ledge [2012]. We had established a relationship with him through those previous shows and when this show came up it certainly was an exciting body of work which we were quite keen to get involved in.” Rankin observes, “Jim is a hands on supervisor and believes very much in previs; he goes through and does quite a rigorous previs of all the action sequences. That is the first line of communication to us. We know in advance the way the story plays out across the shots and when we would do a turnover with him he would talk to us in-depth about any particular requirements the director had for a shot.” Previs was an indispensable tool. “The turnaround in changes to animation and editorial that previs allows are enormous. The time to take to do it is minuscule compared to how long it takes to put those changes through a fully rendered and composited shot. They can try out different things a lot more easily in previs and reduce the number of iterations that we would need to go through in post-production.” The film editors combined footage from other films and the Internet to be used as a creative point of reference. “It was cobbling together an edit that gave the level of energy and the timing of the sequence. One by one we would go through and replace those shots with either fully CG or integrated plates with CG versions of the shot as they were related to the G.I. Joe assets.” The digital versions were not completely the same. “You don’t want your shot to be exactly like they were in some other movie so there is some creative license in there to push the shots in a different direction. For instance, the Helicopter Attack Sequence that we worked on in G.I. Joe was a good example of that where before we even got started on it we were able to have mapped out for us the crescendo of violence, chaos and pyrotechnics over the course of the sequence. It starts out quiet and tame by comparison to how it ends up. It is full hellfire and brimstone at the end.”
“It was almost a reboot of the franchise because a lot of people were disappointed with the first G.I. Joe film,” observes Ollie Rankin. “This director is himself a bit of a fanboy so he came into it with a lot of history and familiarity with the characters and the universe of G.I. Joe; he wanted to give it a grittier more realistic feel than the first one had.” A movie made by British filmmaker Ridley Scott which depicted an ill-fated US military extraction mission in Somalia was a major source of inspiration. “One of the keystones we had been given for the Helicopter Attack Sequence was Black Hawk Down. There was a desire to try and evoke some of the realism of military combat that had been conveyed succinctly in that film. Beyond that one of the only upsides of their being active wars in the Middle East is that YouTube is full of footage of attack helicopters and explosions; we certainly spent quite a lot of time watching footage of these actual machines in action and of the various weaponry systems that they have. There was a certain point where in order to tell a story you need to diverge a bit from reality. Some of the weapons systems were slightly modified from how they would be in reality in order to stage the combat scene as it was intended to play out.” Rankin remarks, “One of the biggest challenges was that Apache helicopters are black and stealth aircraft designed to be invisible but because they play such a pivotal role we needed to show them up. It was finding that balance between making them look like invisible stealth aircraft and yet to still have them feature visually.” It all came down to lighting. “That was a lot of tricky lighting involved in making that happen. Some of the most dramatic shots in that sequence are where the helicopters are almost invisible against the night sky and then is lit up by the flame of a rocket being launched and the explosions would highlight them as well.”
During the Helicopter Attack Sequence, Roadblock (The Rock) drops down into a deep well and bullets are fired into the water after him. “Method did this too,” states James Madigan. “They’re tricky. That was one of those ones that went back and forth. Anytime you see that kind of thing for real in a movie that’s a bullet going 20 times slower than a real one would. It’s usually special effects firing something into the water. We wanted that to feel real, and it’s all digital, dynamics and bubbles. You’ve got to vary the size of the bubbles, and have a big meaty bubble trail that goes through the middle and then more peripheral ones that are small bubbles. Different size bubbles rise at different rates. It looks like a quick little shot we went round and round on that one. There is also the focus. Half the bubbles would be in focus and the other half which are closer to the camera are not. There was a lot of finessing which had to happen there to make you buy that.”
“One of the characters is named Firefly and he carries with him a canister containing a swarm of robotic fireflies,” states Ollie Rankin. “In one sequence he opens that canister up and deploys an entire swarm of several hundred fireflies with each one having its own intelligence.” The exploding robots had to behave realistically enough to believably fool a distant observer. “We looked at a lot of reference of actual fireflies and the insectoid design of the robots was certainly inspired by that. Around the time we were researching this there were also a number of videos on the Internet of little hovering drones being programmed at various educational institutions around the world so we used that as well as a guide that they might move.” Method Studios was in charge of all the sequences involving the lethal technology. “The fireflies feature throughout the movie so there is one firefly in the Apache Sequence and then two or three other sequences that were all about the fireflies being used as weapon to achieve certain ends. At one point the character Firefly is going to break two of the other Cobra agents out of an underground prison and uses the swarm of fireflies to kill all of the guards outside the prison and make his entry into the underground part of it. There is another sequence later on in the movie where the fireflies are being used as a weapon to take out the character Roadblock, and there was a final sequence that is a showdown between Roadblock and Firefly.”
The Prison Break Scene which happens at night is the first time the fireflies have a prominent role to play. “The sequence was much about fooling people into believing that these could be perceived as real fireflies,” states Ollie Rankin. “They had this fluid illuminated canister that provided a lot of the illumination for them. We tuned their appearance to take advantage of that. A couple of the other sequences were daylight sequences; they required different lighting characteristics in order for them to look visually appealing.” The change in lighting conditions led to other creative opportunities. “On the one hand night is definitely more forgiving but daylight on metallic and glass based objects give you nice opportunities for reflecting an environment dome in the surfaces. It gives a lot more subtlety to the metal without that much more complexity in the approach.” Rankin remarks, “They just used one at the beginning of the Helicopter Attack Sequence to tip the audience off to the fact that Cobra was about to get involved. The swarming of the fireflies was quite an interesting challenge. One of our animation TDs [Technical Director] wrote a custom swarming solution in Maya to provide for that. It was quite fun. These fireflies started off mounted on this contraption and were packed in tightly together; you couldn’t tell what they were. It had the look almost of a metallic ear of corn. There is quite a nice transition as individual fireflies come to life and deploy their wings and then take off from the mounting.” To make the fireflies appear to be realistic the tiny robots could not all be doing the same thing. “Part of that was done by randomly offsetting and varying the speed they were moving their wings, and also their legs which are hanging below them would move in an organic fashion. This was not manually keyframed; it was done algorithmically rather than having a large live intelligence swarming approach. The approach taken was to pre-calculate all of the motion paths of the fireflies. What that meant was that we got this great initial first pass at the dispersal, and the fireflies consolidating into this swarm and heading down the road in the direction that they’re all guided to go. Because the motion paths were live in Maya we were able to manipulate individuals which were doing the wrong thing and were able to affect the choreography on a minute level if we needed to.”
“Later on in the Prison Escape Sequence, once the Cobra cohorts have been broken out of the glass cylinders they have been imprisoned in, they need to escape from the prison complex,” says Ollie Rankin. “The warden chooses to rather than trying to shoot one of the escapees to shoot a part of the cooling system which triggers a chain reaction and sets off a bunch of explosions as the cooling system overheats. This culminates in a huge sequence of pyro and destruction as bits of masonry gets blown up, pipes explode and doors get flung around. One of the fleeing characters gets engulfed completely in flame as he is running away from the explosions.” When it is suggested that the judgement of the prison warden was rather questionable, the native of New Zealand laughs, “It seemed like he wasn’t thinking very far into the future.” Practical explosions were incorporated into the scene. “Jim Madigan is a big fan of real pyro so it was quite a hybrid of real pyro and CG destruction going on in those shots. Over four shots that are in the movie there is this one character who flees through the explosions. They filmed the character running through the environment for each of these shots and then they blew up that environment with gaseous pyro explosions for multiple takes from multiple camera angles. But of course none of the structure exploded in that case. We would take those plates of the explosions and extract the character running from the plates that had been shot of him, and track him into those exploding pyro plates. Because there was no motion-control camera used that was quite often a tricky task getting his feet to stick to the ground as he runs through. Once we had the shot laid out with the explosions that were in there we would time the CG destruction to coincide with gaseous explosions that were there, and would tear up bits of concrete and piping as they would be affected. We also added additional 2D explosion elements, lots of smoke and small debris, tiling being ripped out of the floor, and a fully CG ventilation fan as a 2D explosion placed behind it which sends the fan spiralling off into the room. It was fun to pull these shots together because a lot of people get into visual effects for an opportunity to blow shit up. When you can put four or five of them together on a shot and make each of them responsible for a different kind of destruction [it is a dream come true].”
“There is a scene where The Rock’s [Dwayne Johnson] character Roadblock is being pursued by a smaller swarm of fireflies but this time probably less than 50,” recalls Ollie Rankin. “They chase him into and through a warehouse each trying in turn to get closer enough to him to blown themselves up and kill him but he manages to escape unharmed. In that scene the set special effects guys had done a good first round of setting off explosions in the practical photography. Roadblock was running through this warehouse pushing over shelving and knocking boxes out of the way as little explosions were going off around him. The director seemed to think that the explosions were big enough and certainly with the other sequences with the fireflies the explosions were much bigger. We found ourselves match-moving the cameras and all of the environment geometry, and choreographed the paths of the fireflies through the warehouse and timed them to coincide with the explosions that were going off. We then added additional explosions where there weren’t enough already in the plate or where the explosions in the plate were not dramatic enough.”
A resurrected scene turned out to be the biggest challenge which was completed Method New York. “There was a sequence which was originally planned to be in the film and as we were partway through developing it, it was dropped,” explains Ollie Rankin. “But then during the 3D conversion and re-edit of the film they decided to reintroduce that particular sequence. It was a bit of technology that the screenwriters had come up with where Roadblock was wearing a pair of fingerless gloves that were superheated. They had these heating coils on them so hot that he was able to use them to melt his way through a wire mesh fence. That posed a particularly interesting technical challenge for us because the way it was shot was not practically. He was wearing these gloves and they had cut a big hole in the fence. What we needed to do was replace the hole in the fence and then simulate the softening, bending, melting and dropping of these bits of metal fence which was a challenge we tackle in Houdini. We created a system where we match-moved the gloves he was wearing, simulated the heat emission from the element on those gloves, and distributed that heat into the fence and used it to drive the fluidity of the metal and viscosity of it as it liquefied. We were able to burn it from deforming metal into liquid, and generate sparks and smoke as appropriate.”
“That came about as the body of work grew at a time when our facility here in Vancouver was busy with other projects as well and we were fortunate enough that our facility in New York had some additional capacity at the time,” states Ollie Rankin. “We were able to package up a couple of sequences of shots and a couple of groups of shots within other sequences that we internally outsourced to our sister studio. That was a successful model as we were concerned. For instance, once we had established the look of the trace fire here in Vancouver, we were able to outsource a lot of those shots to them to complete. We tried to keep all of the things here in Vancouver which required complex CG to complete but anything that was primarily 2D we were able to package up and get them to do it on our behalf. From our client’s point of view that was transparent and seamless because we remained their primary point of contact.” The three hour time zone discrepancy did not complicate matters. “That’s one instance where the time difference is actually advantageous. They would have a chance to submit work to us in morning before we arrived. We would come in and give them feedback quickly which allowed them to carry on working for the day.”
“There didn’t end up being as much background replacement as we had originally planned for,” reveals Ollie Rankin. “They did quite a good job of designing the set in a way that they were fully encompassing. There were a couple of cases where we needed to do matte paintings to extend the background beyond either the set or the location that they used. There was one sequence where the surviving G.I. Joes are walking through the desert hear and the sound of aircraft. They duck behind a sand dune to avoid detection and combat crawl back up to the top of the sand dune only to find that there’s civil cargo airport beyond it; that was a combination of matte paintings, CG projections and a CG airplane.” The 3D conversion of the movie was not problematic. “It wasn’t so much of an issue. We provided our Nuke scripts in many cases to the conversion facility so they would have the flexibility to separate out the elements that they needed. Some of our shots they were able to convert without any help from us. We certainly didn’t have any major challenges in helping them to complete that.”
Two California based visual effects facilities were contracted. “Luma and a company called SaintsLA which is like my in-house team,” states James Madigan. “Zachary Kinney, who I have had with me since Red, enabled us to use ILM. They do the smaller shots in-between, things we could get done cheaply like paint outs or muzzle flashes. Those are Luma who I love. They did a lot of the stuff like throwing stars which you see right before the Himalayas start where there is fight in the hallway between Snake Eyes [Ray Park] and Storm Shadow [Byung-hun Lee] and we go into a Bullet Time mode; this big fight happening with Storm Shadow throwing all kinds of throwing stars and Snake Eyes shooting them out of the air. They did everything there. They did all of the stuff with Bruce [Willis] and Adrianne Palicki [Red Dawn] in the car when Bruce kicks down the tailgate and starts firing the machine gun. All that stuff was done on green screen which is in the trailer a lot. It’s the whole, ‘Are you okay? My cholesterol is a little high.’ There are a bunch of scenes because we had a limited time with Bruce they were locations we had already left or hadn’t gotten to yet that we shot a lot of him against green screen that they put together. They were great and also right here in L.A. so it was nice that we could pop by; Luma was an easy facility to deal with.”
“We've had a long-standing relationship with Paramount and were introduced to VFX Supervisor, Jim Madigan, and VFX Producer, Pablo Molles, with whom we hit it off right away,” states Luma Pictures Visual Effects Producer Steven Swanson. “We started with a handful of trailer shots, which, after quickly turning around finals, ballooned to over 220 shots.” Visual Effects Supervisor Vince Cirelli, who works for the VFX company which has facilities in Santa Monica, California and Melbourne, Australia, remarks, “James is very hands on, and was excellent at giving us very clear examples of what he wanted a specific effect to look like. Sometimes he would send us reference, or draw over the top of clips to illustrate a given motion or timing.” There was room for creative freedom. “Jim was very open to ideas, and was focused on making each shot as great as it could be, no matter where the ideas came from.” A prominent fight sequence needed to be produced. “Our main responsibility was the battle between Storm Shadow and Snake Eyes, where ninja stars were being shot out of the air,” says Luma Pictures Digital Production Manager Michael Perdew. “We also were required to spruce up the missile near the beginning of the film, augmented Storm Shadow’s escape from prison, and added plenty of chaos and mayhem to the end Gun-Fu Sequence.”
“For one of the shots, we realized that there was a very specific glass shatter that wouldn’t be easily simulated because of how it needed to interact with the environment,” explains Vince Cirelli. “So rather than simulating it, we bought sugar glass and a crossbow, yes I said crossbow, and captured the fracturing using our RED. It turned out to be the perfect element, and the shot finaled within the first or second submission.” Luma Pictures Executive Visual Effects Supervisor Payam Shohadai remarks, “We’re happy with the slow-mo CG throwing stars effect. The hallway is a CG recreation, and has throwing stars travelling down it, while they are being shot at. It was a fun shot to execute, and ended up being used in most of the promotional trailers.” A crucial element was the ability to incorporate a speed ramp. “If we had done that in real time you wouldn’t see anything,” observes James Madigan. “At this moment we’re illustrating one what a great shot Storm Shadow is and his martial arts abilities and two what a great shot Snake Eyes is. We knew from the beginning exactly how we were going to do that shot so I’m shooting the pieces accordingly at the right frame rates. I have to build it in a way that everything stays true to it. For example, on that shot I set it up where I made a template where I animated CG throwing stars going through the air at the speed they would really go. So if you threw them and they went 40 feet and would go SHABAAM! They would hit in five or ten frames. When the throwing stars go out of real time that’s not fake; they’re travelling at the right speed. And from there I figure out how much I would need to crank them down to have them behave in the right way. To have it go at the slow speed I would want I know how to shoot the photography. I know when I try to put the shot together how much I’m ramping. It’s almost like you do it in real time to make sure that you’re staying true to all the physics of it so when the slowed down happens it is not made up. A lot of the times those specials we were planning on them from the beginning and we’re building it accordingly. It just becomes part of the shot.” Cirelli enjoyed the time spent on G.I. Joe: Retaliation. “The project was great to work on. The film is going to be a lot of fun to watch.” Steven Swanson concurs with his Luma Pictures colleague. “It was a real pleasure working with professionals who are passionate about their work and sensitive to the needs of VFX facilities.”
“We’ve been in the trenches with Bernardo Jauregui since the old Digital Domain days,” states Saints LA Visual Effects Supervisor Mark Larranaga. “He brought us on to Red to help out and that’s where we met James Madigan. They are a great team to work with so this was just a continuation of that relationship.” There was no trouble understanding the needs of the production. “An eye for perfection and the ability to articulate the correct direction in a shot is a skill that James possesses. And working closely with Jon Chu on this job; he was always on the same page. The best visual effects blend in seamlessly into a scene without drawing attention to them. In a show like this you’re always going to have shots that are part of the original scope of work and shots that are created after primary photography.” Visual Effects Supervisor Zachary Kinnery who also works for Saints LA explains, “Our work started with James in the postvis process, so I had a desk at their offices at the new Technicolor building on the Paramount lot for a good portion of the project. There was obviously a pretty direct line of communication for that. Once we started in on doing finals, I moved back to our studio in downtown LA so I could work more closely with Mark and our artists. Communication basically consisted of Cinesync sessions, and the occasional review screenings back at the Technicolor theatres. James is a very direct person, and he usually knows exactly what he wants which is a godsend. Whether we were on the phone or there in person, there was never really a question of what he was looking for.”
“G.I. Joe in general, from my standpoint, has always been about big and bold, so that’s what we were aiming for on our shots,” states Zachary Kinnery. “A difference between this film and the last one is this one has a bit more of that gritty realism. We never felt like we necessarily had to refer to it for too much. Even in the cartoons, this type of G.I. Joe had not really been done before, as far as the gritty aspect of it, so we never felt too tied down to anything. It was just, ‘Let’s make this cool!’ Mark Larranaga believes, “With a palette like Retaliation, especially a sequel, creative freedom is a necessity. SaintsLA thrives on design and shots that need creative input.” The California facility tackled more of the 2D side of things. “While PLF [Pixel Liberation Front] which was also working out of our offices at Paramount handled more of the 3D shots,” says Kinnery. “There was a lot of collaboration. We post vis’d, I think, every FX shot in the movie. It was a lot of fun, and we helped design some pretty crazy sequences. Back in our office at SaintsLA, Mark and I supervised a small crew to finish off about 145 shots for final.” Some background replacements and CG augmentation was required. “We did a little bit of both. We did a beautiful 2.5D mountain vista matte painting which we tracked into a flashback scene, some minor set extensions, and we added a small fleet of CG helicopters to another.” Managing the workload within the required time frame was the biggest accomplishment. “We are more proud of the sheer amount of shots we did as such a small company and on schedule and budget.”
Cobra operative Firefly has a million different ways to blow people up such as the weapon he utilizes to free his imprisoned collaborators. “The idea on them was that they were glass grenades that would explode shards of glass which would cut human beings but not blow up the environment,” states James Madigan. “What we wanted was something that looked like glass but had some kind of tech inside of it; Jon designed that with the props. There were parts where we had to take them over with CG. What we had them do was roll them in, cut, setoff the special effects and of course there is never a point where it links up perfectly so we have to CG them in. The special effects that went off had some projectiles in them but we had to amp them up greatly so a lot of the glass shards you see flying around are CG. It wasn’t a major creation by us. We were going with a prop that Jon had designed and dreamt up and enhanced it.” Madigan is strong believer in practical effects. “The main approach I took with everything and tried to push production on was, ‘Lets go CG where we have to but it is important to shoot a lot of elements and to shoot as much as you can on-set.’ Even if they’re not going to be in the right place just to see how it would be in that environment with that lighting if you did it for real. Even when we have to replace things and make them CG which we had to do a lot with the HISS Tank fight you have a real reference you can refer back to.”
“They [Paramount] were looking at a lot of other films coming through and people enjoying them in 3D,” explains James Madigan. “We were looking at the film and this thing is screaming for it. It was one of those things where it needed it but there was no way it could fit into the schedule. The only solution was to move the June 29th release date and it is one of those movies you want to have it.” Madigan adds, “To be honest it was always coming up and we talked about it in the beginning and then it came up again. Jon had done Step Up 3D [2010]. Any kind of new technology Jon is a real master of. It was something he had in mind. I’m not saying that he set up all of his shots with an eye towards conversion but he thinks that way. When we were reviewing and creating things he was like, ‘This would be great in 3D.’ Sort of lamenting it because we weren’t doing it. No one wants to push the release date but when the producers offered to put the money on the table to do it, it was like, ‘Great!’ It was more Paramount seeing what they had and saying, ‘This is worth it.’” Madigan was pleased with the decision. “It looks great in 3D. It does feel like a movie that was always intended for it.”
“I lived in London for five years and blowing that up was fun. My old apartment is in one of those shots,” states James Madigan when reflecting on his favourite moments in the $180 million production of G.I. Joe: Retaliation. “And having previs and coming up with the Firefly bike scene breaking up. It was always in the script but it was how it does and how the shot goes is great. It was great to see those come to life. The Himalayas were a blast because we did go up there and we did shoot it for real. It was an idea of me, Jon and Paul Borne sitting down months before having so much fun creating this thing and then shooting it. Paul Borne was the stunt coordinator on that sequence. Having a real world guy in the room and being up on the side of an 11,000 foot high mountain shooting the actual thing and being in the helicopter shooting all of that stuff and having it as real a feel as it could have gotten. That was exhilarating.”
Production stills © 2011 Paramount Pictures. All Rights Reserved.
VFX shots © 2011 Paramount Pictures. All Rights Reserved. Images supplied by Digital Domain and Saints LA Visual Effects.
Many thanks to James Madigan, Bill George, Thad Beier, Ollie Rankin, Vince Cirelli, Mark Larranaga, Zachary Kinnery, Payam Shohadai, Steven Swanson, and Michael Perdew for taking the time to be interviewed.
Make sure to visit the official websites for G.I. Joe: Retaliation, ILM, Digital Domain, Method Studios, Luma Pictures, and Saints LA Visual Effects.
Trevor Hogg is a freelance video editor and writer who currently resides in Canada.
Lorenzo di Bonaventura |
Lorenzo di Bonaventura and Jon M. Chu |
“Jon is amazing,” remarks James Madigan. “I have never worked with anybody who didn’t matter day, night, weekends, if he was in the middle of a crazy situation on-set shooting, if someone had a question for him you got his time. We would get together rough edits and show him things. I know at one point I handed something over to him at 10 o’clock at night right before I left work and when I came in the next day Jon was like, ‘I want to show you something.’ He had taken the edit and put in speed ramps, music and sound through the whole thing. I was like, ‘When did you do this?’ His assistant said, ‘He was up until four in the morning.’ Jon was showing it to me at eight in the morning. The guy was like a machine and very creative.” The man behind the camera is familiar with the latest technological tools. “He used to sit on-set with his iPhone. We would be shooting five or six takes of each set up and while they would be lighting he would say to the video playback, ‘Let me have take two of this set up and take four of this set up.’ Jon would film them with his phone and then with iMovie he would do a little cut and go, ‘Okay. That works. Let’s move on.’” Madigan notes, “It is the first big visual effects movie he has done.” The inexperience was not a hindrance because the director who is better known for helming dance films is a quick study and multi-faceted. “Those are the movies Jon wound up getting while waiting for other things. I wouldn’t classify him as a guy who is limited to that at all. He does do that stuff because he has a passion for that type of creativity. Jon was straight in there with the actors and in order to get a movie shot if you don’t have the trust of the actors it is all going to fall apart.”
“The way I felt about Joe in general was that we didn’t want to have the look and feel necessarily from the first one,” states James Madigan when discussing how he chose the various visual effects vendors for the project. “The biggest opportunity we could wind-up having that CG feel would have been the Himalayas. So right off the bat I was, ‘Guys, we can’t mess around with this sequence. We want to make sure that we’re going to one of the top places.’ ILM [Industrial Light & Magic] did that sequence. I didn’t think we could get them budgetary and be able to afford the rest of the movie. We started slowly and bid everything out to a lot of companies. As you know in the visual effects world some companies were coming 20 per cent or double or four times higher than other facilities and there are reasons for that. Some facilities are more expensive and others are cheaper. There were some sequences which weren’t as difficult we made sure we saved money on those. I’ve got some in-house guys who can churn through stuff well that helped us to save some money on some other things. ILM was great. They wanted to work with us and came to the table with something we could work with. My visual effects producer Pablo Molles [The Incredible Hulk] hammered out a deal with them.”
“There was a lot of bidding and planning done at ILM before I came on the show,” states ILM Visual Effects Supervisor Bill George. “The studio knew that the Himalayan sequence was going to be one of the ‘key’ sequences in the film and one that had a number of technical challenges. I think the studio felt that the ‘casting’ of that sequence was good for ILM. There were discussions about the project and its budget that went on up until the very last minute. I found out about the project on a Tuesday, it awarded Friday and I was on a plane to New Orleans on Sunday.” The vertigo inducing ninja fight was carefully prepared. “Jim and his crew had been working for a year on the previs for the shot design of the Himalayan sequence. The plan had already been formulated as far as how the green screen elements would be lit and shot, so it was very clear to me from the beginning what the intention of the shots was. Jim was there while we were shooting the green screen elements on stage to make sure we didn’t stray too far from the intended design. Since Jim knew the sequence design better than anyone he was the go-to guy for questions. In post-production Jim was a great wing man for me since he was down in LA working with Jon and had access to him.”
“I spent about two weeks in a helicopter up in B.C. at 11,000 feet filming all of the backgrounds for that,” explains James Madigan. “It could have become the end Underwater Sequence in Joe 1 where it is 100 per cent CG and people in front of green screens. I pushed to say, ‘We want to do a shoot in the mountains.’ The visual effects supervisor had the support of his producer and director. “We made it CG where we absolutely had to but what was great was that we did that first. Another thing I wanted to stay away from was too much digital doubles. I had broke down all of the shots so that we could have stuntmen doing it for real and obviously some of it you can’t do. When we got into the CG side of it I had real footage of real people at least attempting to do it. We could refer back to it and keep it grounded in something that a human being could do or at least attempt to do. Where you see the moments where they get a bit clumsy we tried to use that wherever we could to keep that feel. I mapped out the environment and the footage we were going to use so I knew what the backgrounds were going to be so we could work with the DP who was incredibly collaborative. Patrick Loungway [The Lone Ranger] would study the plates and give us lighting for the stuntmen who would match into it. All of these glued together to make it not have this overly video game CG look to it.”
“Most of the plates that were turned over to us were of the stunt ninjas and Joes hanging on ropes against the huge green screen,” remarks Bill George. “Our job was to place them in a digital Himalayan environment. This set-up reminded me a lot of my work on the Quidditch games from Potter 2 and 3 where all we started with was a kid on a broom against a blue screen. Working on those films certainly influenced my approach to the shots on Joe. Because of the fact that the green screen elements were shots as if they were in the shadow side of the hill, I was really worried about the shots looking very flat. I talked to my Diggimatte supervisor Johan Thorngren about how we could ‘jazz’ up the cliff. I have spent a lot of time in Yosemite; I have encountered granite faces there that are so polished they appear slightly reflective. Johan came up with a way to render a reflection pass on the geometry of our cliff face that we mixed slightly onto the painted surface. The subtle reflection played across the surface adding a lot of life to it. We also added in some wet areas that would have been caused by melting snow, where the reflection was even greater. We did match-a-mations of the characters on the wall to generate subtle shadows to help connect them to the wall when they touched it. Since we had done these match-a-mations for the shadows, we could also use them to render the character's reflections in our wet surfaces. All of these subtle cues helped to integrate our green screen characters into our shadow side of the cliff. In our backgrounds, I cheated sunlight as close to the characters as I could. The direct light adds so much dimensionality and the presence of even a small sliver of blown-out sky makes the shots feel more real. There were a lot of shots that had complex blending of live-action and full CG characters as well as some that were all digital. Our Animation director Paul Kavanagh did an amazing job keeping the movement of the characters within the real world limits but pushing them to do the impossible. In almost every shot we added camera float to make it feel as if the cameramen were hanging from ropes along with the ninjas.”
“The destruction of London in G.I. Joe: Retaliation is caused by the dropping of a man-made meteor,” explains Bill George. “When the filmmakers turned over the shots to us they explained that they didn't want to see the typical ‘nuclear blast’ type shots. They wanted to surface of the earth to ‘shatter’. We explored the idea of treating the ground plane as a thick ‘shell’ that would break like glass when the impact happens. Jon also wanted the energy from the blast to move out in a wave rather than a ring. Editorial in LA had provided to us helicopter footage over London to use as the plates. These plates were used to set the camera and the lighting, but since SO much destruction was going to occur over the course of the shots we completely replaced the plate with our digital London. Once the ground plane was constructed, Paul Kavanagh oversaw the fracturing of the surface and how it moved after the impact. We had some very specific movement we wanted to see, like some of the plates under the Thames River tilting towards us so we could see the water slide towards camera. This animation of the plates drove ALL the destruction and simulations that would follow. We broke down the buildings and bridges into foreground and background assets with the foreground models being more finely detailed and the many background ones being more procedural. We benefited greatly in the fact that many of the artists who came onto our show to work on the London destruction had just come off of Battleship [2012] and Avengers [2012]. They were working at a very high level of speed and efficiency right from the start. The work was very complex and interconnected and everyone on the team used their superpowers to make the shots look amazing.”
“The CG White House was in the script so I did a quick previs of it,” remembers James Madigan who wanted to have the scene of the Cobra soldiers raising their flag to be reminiscent of the iconic image captured at Iwo Jima during World War II. “Marketing saw it and said, ‘We’d like to have that in a month.’” Madigan initially wanted to do a miniature of the famous building but the deadline did not allow enough time. “I flew to D.C. and was going around with a location scout. It’s interesting. With all of the pictures you see of the White House that’s not by accident. They’re all from the same place because they want to funnel you into certain areas where they can keep an eye on you. To get an angle on the roof is hard to do and we were starting on the roof. I noticed this restaurant right next to it that was on a rooftop and I asked, ‘Can we go up there?’ The location scout was like, ‘I know the people up there.’ She was in touch with the secret service and I had a big camera with a 400mm lens which is a paparazzi long lens. When we went up there they said no pictures of the White House.” The viewpoint was exactly what was needed to provide ILM with the right geometry and textures. “The location scout talked to someone and they said, ‘Okay, you sound like you’re approved to take pictures.’ I picked up my camera and started taking long lens pictures of the whole roof when suddenly the manager came running in and said, ‘What are you doing? The secret service just called us and they want to know who you are and why are taking pictures?’ We were like, ‘Oh, God!’ They had to make a call. We were held there. We explained it was for a movie and they were fine with it. It was funny because there are two guys on the roof of the White House with sniper rifles and when I looked through the [zoomed in] pictures it is full framed of these guys. The first guy there is looking through his binoculars and the next one his binoculars are pointed straight at me. Within seconds they knew someone was on a roof taking pictures of them! We immediately went back. It was helpful to have it just to see the texture of the roof. How things are laid out. Where there are chimneys. We made sure that we didn’t put anything else that we saw there on the roof. I circled around it and took a lot of high resolution stills of the White House. Unfortunately, it is pretty featureless. That is the trick when trying to make CG things look real. You put little bits of weathering on it and ageing; that building is in really good shape. It is pure white.”
“The White House pull-back was created by our digital matte department,” remarks Bill George. “We had a plate that Jim had shot of the Cobra soldiers raising the flag at the head of the shot. This element was combined into our CG model of the White House. Jim had risked being thrown in jail to get us really good texture reference of the real White House with his still camera in Washington D.C. In fact if you look to the sides of the soldiers in the pull-back you will see telescopes on the roof. Security men at these telescopes ‘busted’ Jim while he was taking photos for us so we thought it was only fair to include them in our shot. The dropping cobra banners were CG cloth simulations.” The risk taken by James Madigan was worthwhile. “We always start with ‘reality’ as a baseline but as I like to say, ‘Reality is boring!’ You always want to push the limits of making a shot in a film like this as dynamic as you can without going over the line. I find that it mostly comes down to speed. When designing a previs sequence to make it exciting things tend to be very quick. Translating those 1 to 1 to shots would make them feel very false and video-gamey. We had a great advantage on this show in that the stunt performers really hung from ropes and swung through frame. This showed exactly the limits of what makes this action “real”. Right from the start though there were changes made to make things more exciting like shooting slightly undercranked to speed up the action. It was a great benefit though to know exactly what a real swing looks like.” George adds, “Although I am really happy with the London destruction scene, I am proudest of what we created for the Himalayan sequence. We used every creative trick we had to give the shots depth and contrast. Knowing what the sequence looked like when it was turned over to us, you really see the vast scale of our contribution. Both Jon and Jim seemed to have a good time directing and refining our shots and all of their feedback made the shots better. It was the type of working relationship that, as a supervisor, you really hope for.”
“Digital Domain was amazing,” says James Madigan. “DD was doing work straight out of the box version one that looked great; they’re one of the best and was able to come in really reasonable. There’s a whole sequence with HISS Tanks which are these Cobra machines that are like these giant tanks that articulate and can do all kinds of things and a lot of it had to be CG. There was a big battle that we figured out. There is this mini tank which is a real thing. You see it in the trailer attacking them and that needed a lot of work. There were a lot of shots which got made late in the day. They had already been building the satellite and then the satellite needed to do a different thing. DD did it in a manner of weeks, especially, at the end there was a set of shots, then we reordered things, and suddenly with weeks to go we had major shots they were just starting and those guys hit it so hard. Anything in space is DD. Anything with the HISS Tanks is DD. There is a bit where we have a character’s face change into another character which is a fully CG face that DD did. Those were the big tickets items. They did the Archangel. There is this special Joe plane. It’s a real thing but basically the engines turn up so it can hover like helicopter. There’s a real one in the military but it has propellers whereas this has jets. They did all of that. It was a lot of DD in London [Reliance MediaWorks] that did a good bit of it. They did really good work.”
“James was a wonderful, articulate person who had no problem telling us what he wanted,” states Digital Domain Visual Effects Supervisor Thad Beier. “He was astonishingly well-prepared; he does more planning for the visual effects work than I have seen in a lot of people. For the tank battle James had gone out there and surveyed the whole environment himself; he basically published a book which had every shot in it, where the tanks should be and what the shots should be.” Not everything went accordingly to plan as is normally the case once filming commences. “In the HISS Tank Sequence there were these giant tanks that they actually built and moved around. They were difficult to move and had limited action capabilities.” The mechanical props gave a “tremendous performance” but kept on breaking down and had to be continually rebuilt. “Because of previs we were able to take care of those situations as they came up. As far as the camera work goes the second unit director did vary quite a bit from the previs as they often do. He had the previs as a side point for the camera work but that is something the effects unit director often wants to change, and the second unit DP as well. They want to put their own look on the thing. Previs helps to figure out what the actions should be; as far as the camera moves they were more often up to the day.”
Digital HISS Tanks were utilized for the close-up shots. “In a lot of cases the CG tank is a lot more detailed than the physical one,” states Thad Beier. “The CG tanks could go much faster and move a lot better so in those cases where the tanks had to move quickly they’re all CG. The big challenges were when we were blowing up tanks. They only had one of these physical tanks and didn’t want to blow that up. They built a box of a tank out of the exact same scale out of metal and they would blow that up. We would have to put the CG tank in the middle and fire. Trying to simulate the effects of the lighting of the fire with the tank inside of it as all those surfaces would be lit was a lot of fun. You usually think about lighting things more from sources which are faraway like the sun or sky but here you’re getting the tank in the light source. It was an interesting lighting challenge to try to make it look like it would be affected by light and not lighting itself.”
The transition between the two men had to appear physical rather than morphing. “The idea was as the president potentially injures himself, the nanomites,” explains Thad Beier, “the micro machines that make up his mask rush to heal him and when they did that they would reveal the character underneath. Once he was healed within a few seconds they would rush back to recreate the mask.” No previs was produced to orchestrate the shot. “That was a shot we did on the day as we shot it. We had several hundred visuals of that shot which were composited for only a few seconds of the movie. It was by far our most complex shot. It was a lot of fun.” The final result was accomplished without the aid of facial capture. “It came down to the last two or three weeks where I was sitting in a chair with the animator everyday going over it,” remarks James Madigan. “Faces are all about the subtleties. What you think your face would do to convey a certain thing it is not really what it does.” A lot of effort was made to avoid the ‘uncanny valley’ effect which results in lifeless eyes. “It was a lot of work of me and the animator days on end looking in a mirror filming ourselves trying to figure out exactly what we needed to do.”
The concept art for the Cobra Commander Helicopter was supplied by the Art Department for the movie. “It had some features which were like a Comanche Helicopter which was never put into production,” states Thad Beier. “We had our team in London build a [digital] model helicopter based on those drawings.” A physical replica was built and used for one shot. “The real thing defined what the look had to be. You had to match that thing exactly.” Lots of photography was taken along with LiDAR scans of the aircraft to be used as reference material. “There is one point where the helicopter crashes in the movie and there were spectacular helicopter crash footage we were looking at but in the end they changed the script so that the Cobra commander escapes and we didn’t end up using that. For a lot of the shots there was a real helicopter which was about the same size and scale as the Cobra Commander Helicopter.” The aircraft supplied essential environmental subtleties such as those caused by it hovering over the water and grass. “The Archangles were a late edition for us and we had to scramble to get those done,” recalls Thad Beier. “They were giant vertical take off and landing airplanes. We only see them at a dark night. It is one of things where what you don’t see is more important. You have to image what the shape is and let your imagination fill in everything you can’t see. We had a single drawing to work from to build it and we built it. As you move around the object the flaws in the original drawing became apparent so we had to remodel it so it could look good from all points of view. The biggest challenge with that by far was the lighting. Trying to make it look like a dark night but to be able to see everything you wanted in the Archangel. They shot the environment flying over day for night but on a cloudy day so there was no direct moonlight even and to design a balanced lighting of the Archangles.” An edge light was devised to help give definition to the black aircraft which allowed the viewer to be teased as it quickly flies by.
“The big shot where Firefly jumps the motorbike, and it breaks apart into pieces and those pieces turn into rockets which fire and explode at the prison; that was a complicated sequence indeed,” states Thad Beier. “Madigan shot the real motorcycle which was a spectacular reference to work from but as it is a slow-motion shot you got to examine it in detail.” The sequence which featured practical explosions was heavily art directed with almost every frame being separately lit. “The interaction of the pieces and explosions was a real challenge. We tried severely to get the motion of the Firefly motorcycle to work with a camera move so it looked like a natural move that could happen.” Animation Supervisor Erik Gamache was given the task of portraying the imaginary high tech vehicle as if it was something that existed. “The Firefly character our main villain for the piece had to be animated as well and had to have natural motions. Erik is a motorcycle rider and is a fanatic about motorcycles; he was engaged with the thought and decided to make it as realistic as possible. As Firefly is jumping he has to hit various controls and pull a ripcord. Erik was concerned that a motorcycle rider would see all of the motions he was doing as a reasonable thing to do. He would never let go of the handlebars at that point of the jump. He would do it here. Erik was the right guy for the job. It’s a shot that has been in every trailer for the movie and we are pleased with the way it turned out.”
Production provided the concept art for the Zeus Satellite. “Its profile in the movie grew as the movie went on. Erik and I looked at and said, ‘That looks cool. We can use that here.’ We ended up with a lot more close-up shots and shots of the satellite with the Earth behind it,” remembers Thad Beier. “Later on in the production they decided to blow up the satellite. Explosions in space are a real challenge. What do explosions in space look like? We wanted to come up with something physically plausible yet still interesting looking. The idea is the explosion in space doesn’t have an atmosphere so it keeps on growing and would never stop. It would never billow in the way explosions do on Earth. We had an idea that you would see the fuel burning and continuing to expand, and as it expands the force of it would split the satellite apart. A lot of the simulations of the satellite blowing up we had expected the need to do a lot of keyframe animation of the satellite parts breaking apart.” The simulation turned out better than expected. “We were surprised by the time the simulation was finished all hundred thousand parts were breaking apart and ripping and tumbling. It looked fabulous. We looked at it when all of the hero animation was done except for one shot of the Cobra logo coming at the camera, the explosion itself worked fabulous. Jeremy Hampton was our main simulation effects guy and did a spectacular job.” There were only a few weeks left in the project which meant there would not be many opportunities to run the giant simulations to see how they looked. It was a nice destruction of the Cobra satellite system in the end.”
Digital Domain was also responsible for the destruction of the ICBM (Intercontinental Ballistic Missile) fleet. “I was surprised when we were doing the ICBM shot that none of my younger animators were familiar with the term ICBM,” notes Thad Beier. “It surprised me how the missile fleets of the US or the Soviet Union had faded from memory to the extent that kids don’t even know what ICBM means anymore. We liked the look of the MX Peacekeeper Missile, the last ICBM ever built.” The weapon requires solid rather liquid fuel which causes to blow up in way that resembles giant fireworks. “Tens of hundreds of thousand of little particles are burning and flying away from the explosion. Again since this is in space these wouldn’t explode down the atmosphere, they would explode out and continue to move outward as the shot progressed. Jeremy Hampton our effects supervisor had a lot to simulate there. The volume that would contain this explosion would have to be large in the end but still have a lot of resolution to capture all of the details of the explosion. At the end the explosion simulation volume had 10 billion voxels in it. By far the biggest one we had ever attempted. It was a relatively realistic explosion of a solid fuel rocket and it was choreographed well.” Each explosion was positioned on the screen so to assist the storytelling as well to maximize its visual impact. “The explosions we had done low resolution simulations so we only had one chance to run the final explosion before we had to deliver.” Good fortune prevailed. “The first time we saw it everyone loved it. ‘That’s exactly right. That’s the explosion we want.’ I was pleased that it worked out.”
“Some of the work on the Archangel sequence came from a different facility and we took that over,” states Thad Beier. “Digital Domain spread the work out between three facilities. We have our group in Venice which did a lot of work. We had a group in London that did most of the tank battle sequence and in India.” In regards to maintaining a unified look, Beier remarks, “That’s absolutely the supervisor’s job to make that happen. We’re viewing things from all the facilities everyday and making sure that they’re on the same page. The producer and I flew out to London a few times to meet him on the ground and talk to him in a way you can’t do over the phone or video link or cyber.” The decision for G.I. Joe: Retaliation to be screened in 3D resulted in a minor complication as a strict back to front layer policy was not maintained. “The visual effects shots can be better if they’re broken into parts into layers and the conversion company can use the layers separately. It makes their job much easier than if they’re presented with the final composite.” The fantastical elements were handled realistically. “This movie does a lot better job of that than the first movie did. It is much more plausible. It is much more about the characters and how people react to plausible things. The viewer is engaged with the story better because nothing magical happens. These guys have to work for it given the tools that they have. They have real things to work against. The HISS Tanks are wonderfully physical objects and the Joe tank is a beautiful piece of machinery. Those are real physical things that the viewers get in engaged with and understand on a visceral level which will be a great part of this movie. They will get into it in a way than a more fantastical movie.” Jon Chu made the effort worthwhile with his positive attitude. “I’d like to say that I thought that the director provided an inspiration and an enthusiasm throughout the process that affected everybody.”
“Method did a lot of the stuff in the bunker,” states James Madigan who had the visual effects company handle the sequences involving the explosive fireflies which are part of an arsenal belonging to the villain Firefly played by Ray Stevenson (Jayne Mansfield’s Car). “He releases these little mechanical fireflies which are like small little bombs that can fly at someone who thinks they’re looking at an insect so they can get close and blow them up. It had to sell at the beginning before we first see them operate that it was like a real firefly.” The Vancouver facility for Method Studios was responsible for the inciting incident which propels the G.I. Joes into action when their base camp is attacked. “They did the whole desert battle which is all CG Apaches in the night scene. We had some special effects on the ground blowing things up. Jon wanted all the bullets being fired to be tracers and tracers are one of those things that they always wind up looking like Star Wars [1977]. Depending on your exposure, time of night, how much light, sometimes they’re yellow or red. It’s a tricky thing. They did all of that plus missiles firing with the jet trails behind them. They made that sequence hum and cranked out a lot of that stuff early on for the trailers. We need to believe that the Joes are not spooked by the approaching helicopters. ‘Oh, here comes the evac team.’ Then all of the sudden it goes horribly wrong. There was a careful way Method had the helicopters marching at us from shot to shot until their behaviour immediately changes. We had to go through the continuity of each shot. How the first wave happens. How the second wave happens. Where they are coming from? How distant they should feel when they’re firing missiles. There was a lot of technical geometry we had to work out in order for those shots to feel right. They did an amazing job with that stuff.”
“We had worked with Jim Madigan a couple of times in the past,” states Method Studios Visual Effects Supervisor Ollie Rankin. “Prior to G.I. Joe we had done work for him on Red and also he got involved with the tail end of Man on a Ledge [2012]. We had established a relationship with him through those previous shows and when this show came up it certainly was an exciting body of work which we were quite keen to get involved in.” Rankin observes, “Jim is a hands on supervisor and believes very much in previs; he goes through and does quite a rigorous previs of all the action sequences. That is the first line of communication to us. We know in advance the way the story plays out across the shots and when we would do a turnover with him he would talk to us in-depth about any particular requirements the director had for a shot.” Previs was an indispensable tool. “The turnaround in changes to animation and editorial that previs allows are enormous. The time to take to do it is minuscule compared to how long it takes to put those changes through a fully rendered and composited shot. They can try out different things a lot more easily in previs and reduce the number of iterations that we would need to go through in post-production.” The film editors combined footage from other films and the Internet to be used as a creative point of reference. “It was cobbling together an edit that gave the level of energy and the timing of the sequence. One by one we would go through and replace those shots with either fully CG or integrated plates with CG versions of the shot as they were related to the G.I. Joe assets.” The digital versions were not completely the same. “You don’t want your shot to be exactly like they were in some other movie so there is some creative license in there to push the shots in a different direction. For instance, the Helicopter Attack Sequence that we worked on in G.I. Joe was a good example of that where before we even got started on it we were able to have mapped out for us the crescendo of violence, chaos and pyrotechnics over the course of the sequence. It starts out quiet and tame by comparison to how it ends up. It is full hellfire and brimstone at the end.”
“It was almost a reboot of the franchise because a lot of people were disappointed with the first G.I. Joe film,” observes Ollie Rankin. “This director is himself a bit of a fanboy so he came into it with a lot of history and familiarity with the characters and the universe of G.I. Joe; he wanted to give it a grittier more realistic feel than the first one had.” A movie made by British filmmaker Ridley Scott which depicted an ill-fated US military extraction mission in Somalia was a major source of inspiration. “One of the keystones we had been given for the Helicopter Attack Sequence was Black Hawk Down. There was a desire to try and evoke some of the realism of military combat that had been conveyed succinctly in that film. Beyond that one of the only upsides of their being active wars in the Middle East is that YouTube is full of footage of attack helicopters and explosions; we certainly spent quite a lot of time watching footage of these actual machines in action and of the various weaponry systems that they have. There was a certain point where in order to tell a story you need to diverge a bit from reality. Some of the weapons systems were slightly modified from how they would be in reality in order to stage the combat scene as it was intended to play out.” Rankin remarks, “One of the biggest challenges was that Apache helicopters are black and stealth aircraft designed to be invisible but because they play such a pivotal role we needed to show them up. It was finding that balance between making them look like invisible stealth aircraft and yet to still have them feature visually.” It all came down to lighting. “That was a lot of tricky lighting involved in making that happen. Some of the most dramatic shots in that sequence are where the helicopters are almost invisible against the night sky and then is lit up by the flame of a rocket being launched and the explosions would highlight them as well.”
During the Helicopter Attack Sequence, Roadblock (The Rock) drops down into a deep well and bullets are fired into the water after him. “Method did this too,” states James Madigan. “They’re tricky. That was one of those ones that went back and forth. Anytime you see that kind of thing for real in a movie that’s a bullet going 20 times slower than a real one would. It’s usually special effects firing something into the water. We wanted that to feel real, and it’s all digital, dynamics and bubbles. You’ve got to vary the size of the bubbles, and have a big meaty bubble trail that goes through the middle and then more peripheral ones that are small bubbles. Different size bubbles rise at different rates. It looks like a quick little shot we went round and round on that one. There is also the focus. Half the bubbles would be in focus and the other half which are closer to the camera are not. There was a lot of finessing which had to happen there to make you buy that.”
“One of the characters is named Firefly and he carries with him a canister containing a swarm of robotic fireflies,” states Ollie Rankin. “In one sequence he opens that canister up and deploys an entire swarm of several hundred fireflies with each one having its own intelligence.” The exploding robots had to behave realistically enough to believably fool a distant observer. “We looked at a lot of reference of actual fireflies and the insectoid design of the robots was certainly inspired by that. Around the time we were researching this there were also a number of videos on the Internet of little hovering drones being programmed at various educational institutions around the world so we used that as well as a guide that they might move.” Method Studios was in charge of all the sequences involving the lethal technology. “The fireflies feature throughout the movie so there is one firefly in the Apache Sequence and then two or three other sequences that were all about the fireflies being used as weapon to achieve certain ends. At one point the character Firefly is going to break two of the other Cobra agents out of an underground prison and uses the swarm of fireflies to kill all of the guards outside the prison and make his entry into the underground part of it. There is another sequence later on in the movie where the fireflies are being used as a weapon to take out the character Roadblock, and there was a final sequence that is a showdown between Roadblock and Firefly.”
The Prison Break Scene which happens at night is the first time the fireflies have a prominent role to play. “The sequence was much about fooling people into believing that these could be perceived as real fireflies,” states Ollie Rankin. “They had this fluid illuminated canister that provided a lot of the illumination for them. We tuned their appearance to take advantage of that. A couple of the other sequences were daylight sequences; they required different lighting characteristics in order for them to look visually appealing.” The change in lighting conditions led to other creative opportunities. “On the one hand night is definitely more forgiving but daylight on metallic and glass based objects give you nice opportunities for reflecting an environment dome in the surfaces. It gives a lot more subtlety to the metal without that much more complexity in the approach.” Rankin remarks, “They just used one at the beginning of the Helicopter Attack Sequence to tip the audience off to the fact that Cobra was about to get involved. The swarming of the fireflies was quite an interesting challenge. One of our animation TDs [Technical Director] wrote a custom swarming solution in Maya to provide for that. It was quite fun. These fireflies started off mounted on this contraption and were packed in tightly together; you couldn’t tell what they were. It had the look almost of a metallic ear of corn. There is quite a nice transition as individual fireflies come to life and deploy their wings and then take off from the mounting.” To make the fireflies appear to be realistic the tiny robots could not all be doing the same thing. “Part of that was done by randomly offsetting and varying the speed they were moving their wings, and also their legs which are hanging below them would move in an organic fashion. This was not manually keyframed; it was done algorithmically rather than having a large live intelligence swarming approach. The approach taken was to pre-calculate all of the motion paths of the fireflies. What that meant was that we got this great initial first pass at the dispersal, and the fireflies consolidating into this swarm and heading down the road in the direction that they’re all guided to go. Because the motion paths were live in Maya we were able to manipulate individuals which were doing the wrong thing and were able to affect the choreography on a minute level if we needed to.”
“Later on in the Prison Escape Sequence, once the Cobra cohorts have been broken out of the glass cylinders they have been imprisoned in, they need to escape from the prison complex,” says Ollie Rankin. “The warden chooses to rather than trying to shoot one of the escapees to shoot a part of the cooling system which triggers a chain reaction and sets off a bunch of explosions as the cooling system overheats. This culminates in a huge sequence of pyro and destruction as bits of masonry gets blown up, pipes explode and doors get flung around. One of the fleeing characters gets engulfed completely in flame as he is running away from the explosions.” When it is suggested that the judgement of the prison warden was rather questionable, the native of New Zealand laughs, “It seemed like he wasn’t thinking very far into the future.” Practical explosions were incorporated into the scene. “Jim Madigan is a big fan of real pyro so it was quite a hybrid of real pyro and CG destruction going on in those shots. Over four shots that are in the movie there is this one character who flees through the explosions. They filmed the character running through the environment for each of these shots and then they blew up that environment with gaseous pyro explosions for multiple takes from multiple camera angles. But of course none of the structure exploded in that case. We would take those plates of the explosions and extract the character running from the plates that had been shot of him, and track him into those exploding pyro plates. Because there was no motion-control camera used that was quite often a tricky task getting his feet to stick to the ground as he runs through. Once we had the shot laid out with the explosions that were in there we would time the CG destruction to coincide with gaseous explosions that were there, and would tear up bits of concrete and piping as they would be affected. We also added additional 2D explosion elements, lots of smoke and small debris, tiling being ripped out of the floor, and a fully CG ventilation fan as a 2D explosion placed behind it which sends the fan spiralling off into the room. It was fun to pull these shots together because a lot of people get into visual effects for an opportunity to blow shit up. When you can put four or five of them together on a shot and make each of them responsible for a different kind of destruction [it is a dream come true].”
“There is a scene where The Rock’s [Dwayne Johnson] character Roadblock is being pursued by a smaller swarm of fireflies but this time probably less than 50,” recalls Ollie Rankin. “They chase him into and through a warehouse each trying in turn to get closer enough to him to blown themselves up and kill him but he manages to escape unharmed. In that scene the set special effects guys had done a good first round of setting off explosions in the practical photography. Roadblock was running through this warehouse pushing over shelving and knocking boxes out of the way as little explosions were going off around him. The director seemed to think that the explosions were big enough and certainly with the other sequences with the fireflies the explosions were much bigger. We found ourselves match-moving the cameras and all of the environment geometry, and choreographed the paths of the fireflies through the warehouse and timed them to coincide with the explosions that were going off. We then added additional explosions where there weren’t enough already in the plate or where the explosions in the plate were not dramatic enough.”
A resurrected scene turned out to be the biggest challenge which was completed Method New York. “There was a sequence which was originally planned to be in the film and as we were partway through developing it, it was dropped,” explains Ollie Rankin. “But then during the 3D conversion and re-edit of the film they decided to reintroduce that particular sequence. It was a bit of technology that the screenwriters had come up with where Roadblock was wearing a pair of fingerless gloves that were superheated. They had these heating coils on them so hot that he was able to use them to melt his way through a wire mesh fence. That posed a particularly interesting technical challenge for us because the way it was shot was not practically. He was wearing these gloves and they had cut a big hole in the fence. What we needed to do was replace the hole in the fence and then simulate the softening, bending, melting and dropping of these bits of metal fence which was a challenge we tackle in Houdini. We created a system where we match-moved the gloves he was wearing, simulated the heat emission from the element on those gloves, and distributed that heat into the fence and used it to drive the fluidity of the metal and viscosity of it as it liquefied. We were able to burn it from deforming metal into liquid, and generate sparks and smoke as appropriate.”
“That came about as the body of work grew at a time when our facility here in Vancouver was busy with other projects as well and we were fortunate enough that our facility in New York had some additional capacity at the time,” states Ollie Rankin. “We were able to package up a couple of sequences of shots and a couple of groups of shots within other sequences that we internally outsourced to our sister studio. That was a successful model as we were concerned. For instance, once we had established the look of the trace fire here in Vancouver, we were able to outsource a lot of those shots to them to complete. We tried to keep all of the things here in Vancouver which required complex CG to complete but anything that was primarily 2D we were able to package up and get them to do it on our behalf. From our client’s point of view that was transparent and seamless because we remained their primary point of contact.” The three hour time zone discrepancy did not complicate matters. “That’s one instance where the time difference is actually advantageous. They would have a chance to submit work to us in morning before we arrived. We would come in and give them feedback quickly which allowed them to carry on working for the day.”
“There didn’t end up being as much background replacement as we had originally planned for,” reveals Ollie Rankin. “They did quite a good job of designing the set in a way that they were fully encompassing. There were a couple of cases where we needed to do matte paintings to extend the background beyond either the set or the location that they used. There was one sequence where the surviving G.I. Joes are walking through the desert hear and the sound of aircraft. They duck behind a sand dune to avoid detection and combat crawl back up to the top of the sand dune only to find that there’s civil cargo airport beyond it; that was a combination of matte paintings, CG projections and a CG airplane.” The 3D conversion of the movie was not problematic. “It wasn’t so much of an issue. We provided our Nuke scripts in many cases to the conversion facility so they would have the flexibility to separate out the elements that they needed. Some of our shots they were able to convert without any help from us. We certainly didn’t have any major challenges in helping them to complete that.”
Two California based visual effects facilities were contracted. “Luma and a company called SaintsLA which is like my in-house team,” states James Madigan. “Zachary Kinney, who I have had with me since Red, enabled us to use ILM. They do the smaller shots in-between, things we could get done cheaply like paint outs or muzzle flashes. Those are Luma who I love. They did a lot of the stuff like throwing stars which you see right before the Himalayas start where there is fight in the hallway between Snake Eyes [Ray Park] and Storm Shadow [Byung-hun Lee] and we go into a Bullet Time mode; this big fight happening with Storm Shadow throwing all kinds of throwing stars and Snake Eyes shooting them out of the air. They did everything there. They did all of the stuff with Bruce [Willis] and Adrianne Palicki [Red Dawn] in the car when Bruce kicks down the tailgate and starts firing the machine gun. All that stuff was done on green screen which is in the trailer a lot. It’s the whole, ‘Are you okay? My cholesterol is a little high.’ There are a bunch of scenes because we had a limited time with Bruce they were locations we had already left or hadn’t gotten to yet that we shot a lot of him against green screen that they put together. They were great and also right here in L.A. so it was nice that we could pop by; Luma was an easy facility to deal with.”
“We've had a long-standing relationship with Paramount and were introduced to VFX Supervisor, Jim Madigan, and VFX Producer, Pablo Molles, with whom we hit it off right away,” states Luma Pictures Visual Effects Producer Steven Swanson. “We started with a handful of trailer shots, which, after quickly turning around finals, ballooned to over 220 shots.” Visual Effects Supervisor Vince Cirelli, who works for the VFX company which has facilities in Santa Monica, California and Melbourne, Australia, remarks, “James is very hands on, and was excellent at giving us very clear examples of what he wanted a specific effect to look like. Sometimes he would send us reference, or draw over the top of clips to illustrate a given motion or timing.” There was room for creative freedom. “Jim was very open to ideas, and was focused on making each shot as great as it could be, no matter where the ideas came from.” A prominent fight sequence needed to be produced. “Our main responsibility was the battle between Storm Shadow and Snake Eyes, where ninja stars were being shot out of the air,” says Luma Pictures Digital Production Manager Michael Perdew. “We also were required to spruce up the missile near the beginning of the film, augmented Storm Shadow’s escape from prison, and added plenty of chaos and mayhem to the end Gun-Fu Sequence.”
“For one of the shots, we realized that there was a very specific glass shatter that wouldn’t be easily simulated because of how it needed to interact with the environment,” explains Vince Cirelli. “So rather than simulating it, we bought sugar glass and a crossbow, yes I said crossbow, and captured the fracturing using our RED. It turned out to be the perfect element, and the shot finaled within the first or second submission.” Luma Pictures Executive Visual Effects Supervisor Payam Shohadai remarks, “We’re happy with the slow-mo CG throwing stars effect. The hallway is a CG recreation, and has throwing stars travelling down it, while they are being shot at. It was a fun shot to execute, and ended up being used in most of the promotional trailers.” A crucial element was the ability to incorporate a speed ramp. “If we had done that in real time you wouldn’t see anything,” observes James Madigan. “At this moment we’re illustrating one what a great shot Storm Shadow is and his martial arts abilities and two what a great shot Snake Eyes is. We knew from the beginning exactly how we were going to do that shot so I’m shooting the pieces accordingly at the right frame rates. I have to build it in a way that everything stays true to it. For example, on that shot I set it up where I made a template where I animated CG throwing stars going through the air at the speed they would really go. So if you threw them and they went 40 feet and would go SHABAAM! They would hit in five or ten frames. When the throwing stars go out of real time that’s not fake; they’re travelling at the right speed. And from there I figure out how much I would need to crank them down to have them behave in the right way. To have it go at the slow speed I would want I know how to shoot the photography. I know when I try to put the shot together how much I’m ramping. It’s almost like you do it in real time to make sure that you’re staying true to all the physics of it so when the slowed down happens it is not made up. A lot of the times those specials we were planning on them from the beginning and we’re building it accordingly. It just becomes part of the shot.” Cirelli enjoyed the time spent on G.I. Joe: Retaliation. “The project was great to work on. The film is going to be a lot of fun to watch.” Steven Swanson concurs with his Luma Pictures colleague. “It was a real pleasure working with professionals who are passionate about their work and sensitive to the needs of VFX facilities.”
“We’ve been in the trenches with Bernardo Jauregui since the old Digital Domain days,” states Saints LA Visual Effects Supervisor Mark Larranaga. “He brought us on to Red to help out and that’s where we met James Madigan. They are a great team to work with so this was just a continuation of that relationship.” There was no trouble understanding the needs of the production. “An eye for perfection and the ability to articulate the correct direction in a shot is a skill that James possesses. And working closely with Jon Chu on this job; he was always on the same page. The best visual effects blend in seamlessly into a scene without drawing attention to them. In a show like this you’re always going to have shots that are part of the original scope of work and shots that are created after primary photography.” Visual Effects Supervisor Zachary Kinnery who also works for Saints LA explains, “Our work started with James in the postvis process, so I had a desk at their offices at the new Technicolor building on the Paramount lot for a good portion of the project. There was obviously a pretty direct line of communication for that. Once we started in on doing finals, I moved back to our studio in downtown LA so I could work more closely with Mark and our artists. Communication basically consisted of Cinesync sessions, and the occasional review screenings back at the Technicolor theatres. James is a very direct person, and he usually knows exactly what he wants which is a godsend. Whether we were on the phone or there in person, there was never really a question of what he was looking for.”
“G.I. Joe in general, from my standpoint, has always been about big and bold, so that’s what we were aiming for on our shots,” states Zachary Kinnery. “A difference between this film and the last one is this one has a bit more of that gritty realism. We never felt like we necessarily had to refer to it for too much. Even in the cartoons, this type of G.I. Joe had not really been done before, as far as the gritty aspect of it, so we never felt too tied down to anything. It was just, ‘Let’s make this cool!’ Mark Larranaga believes, “With a palette like Retaliation, especially a sequel, creative freedom is a necessity. SaintsLA thrives on design and shots that need creative input.” The California facility tackled more of the 2D side of things. “While PLF [Pixel Liberation Front] which was also working out of our offices at Paramount handled more of the 3D shots,” says Kinnery. “There was a lot of collaboration. We post vis’d, I think, every FX shot in the movie. It was a lot of fun, and we helped design some pretty crazy sequences. Back in our office at SaintsLA, Mark and I supervised a small crew to finish off about 145 shots for final.” Some background replacements and CG augmentation was required. “We did a little bit of both. We did a beautiful 2.5D mountain vista matte painting which we tracked into a flashback scene, some minor set extensions, and we added a small fleet of CG helicopters to another.” Managing the workload within the required time frame was the biggest accomplishment. “We are more proud of the sheer amount of shots we did as such a small company and on schedule and budget.”
“They [Paramount] were looking at a lot of other films coming through and people enjoying them in 3D,” explains James Madigan. “We were looking at the film and this thing is screaming for it. It was one of those things where it needed it but there was no way it could fit into the schedule. The only solution was to move the June 29th release date and it is one of those movies you want to have it.” Madigan adds, “To be honest it was always coming up and we talked about it in the beginning and then it came up again. Jon had done Step Up 3D [2010]. Any kind of new technology Jon is a real master of. It was something he had in mind. I’m not saying that he set up all of his shots with an eye towards conversion but he thinks that way. When we were reviewing and creating things he was like, ‘This would be great in 3D.’ Sort of lamenting it because we weren’t doing it. No one wants to push the release date but when the producers offered to put the money on the table to do it, it was like, ‘Great!’ It was more Paramount seeing what they had and saying, ‘This is worth it.’” Madigan was pleased with the decision. “It looks great in 3D. It does feel like a movie that was always intended for it.”
“I lived in London for five years and blowing that up was fun. My old apartment is in one of those shots,” states James Madigan when reflecting on his favourite moments in the $180 million production of G.I. Joe: Retaliation. “And having previs and coming up with the Firefly bike scene breaking up. It was always in the script but it was how it does and how the shot goes is great. It was great to see those come to life. The Himalayas were a blast because we did go up there and we did shoot it for real. It was an idea of me, Jon and Paul Borne sitting down months before having so much fun creating this thing and then shooting it. Paul Borne was the stunt coordinator on that sequence. Having a real world guy in the room and being up on the side of an 11,000 foot high mountain shooting the actual thing and being in the helicopter shooting all of that stuff and having it as real a feel as it could have gotten. That was exhilarating.”
Production stills © 2011 Paramount Pictures. All Rights Reserved.
VFX shots © 2011 Paramount Pictures. All Rights Reserved. Images supplied by Digital Domain and Saints LA Visual Effects.
Many thanks to James Madigan, Bill George, Thad Beier, Ollie Rankin, Vince Cirelli, Mark Larranaga, Zachary Kinnery, Payam Shohadai, Steven Swanson, and Michael Perdew for taking the time to be interviewed.
Make sure to visit the official websites for G.I. Joe: Retaliation, ILM, Digital Domain, Method Studios, Luma Pictures, and Saints LA Visual Effects.
Trevor Hogg is a freelance video editor and writer who currently resides in Canada.