Trevor Hogg chats with Hugh Macdonald about the visual effects found in Frankenweenie...
“Nvizible had previously worked with VFX Supervisor Tim Ledbury on Fantastic Mr. Fox [2009], Wes Anderson’s stop-motion film, and he approached us to work with him again on Frankenweenie [2012],” explains Hugh Macdonald who is a VFX Supervisor and the Head of Technology for the British visual effects facility. “On Fantastic Mr. Fox, we got involved right towards the end of the post production period, helping out with some of the left-over shots that needed completing. Conversely, on Frankenweenie, we were involved from a far earlier stage; this gave us the ability to take a longer view of the work and to visit the set so to have a better understanding of how it was to be shot.” Ledbury was able to visually articulate what he was looking for to Macdonald and his team. “As someone who has a background as a matte painter, Tim was able to quickly put together rough digital paintings illustrating the look and palette of a shot. We were also working on a number of sequences and looks alongside his in-house team, which gave us ideal reference to match to. There were also a number of sequences where we came up with a number of look options ourselves, and submitted these both Tim Ledbury and Tim Burton [Alice in Wonderland] to decide on the final look.”
“There are a number of things that make visual effects on stop-motion more complex that traditional live-action – rigs and motion continuity,” says Macdonald. “When you consider how a stop-motion film is made, any character or object that is not touching the ground, or has only minimal contact, a rig is needed to ensure that it stays in the same position in the shot. These rigs all need painting out in post. An extreme example of this in the work that we did is the shots with the sea monkeys climbing all over the telephone booth. Each sea monkey had its own rig, and while great care was made to ensure that the rigs blocked as little of the action as possible, this wasn’t always feasible; we did have to re-build sea monkey legs or tails on a number of frames. The other complexity is motion continuity. In a live-action film, if a ball is thrown the laws of physics dictate the path that it will follow, and any rotoscoping can make use of interpolation features. However, in a stop-motion film, this isn’t necessarily the case. The ball would have to be positioned every frame by hand; it is not always in the same place that physics would define it to be. Rather than rotoscoping keyframes and letting interpolation do a lot of the in-between work, we would often end up having to tweak the roto on every single frame. On the other side of this, when adding CG elements to a shot, for example the water splashes or the popcorn explosion, we would add a very slight randomisation to the position of each particle on each frame, giving it less of a smooth look. One area where stop-motion does make some elements of visual effects easier is the lack of motion blur; this makes edges cleaner, and much more straightforward to key and rotoscope.”
“If anything the fact that the final result was to be black and white made our lives easier,” observes Hugh Macdonald. “It was shot in colour, and we had the original colour footage to work on, which was essential for any shots involving green screens. We also had the LUT that gave us what the final result was going to look like. When working on the shots, artists would have the ability to view the result in both colour and black and white. Their primary aim would be to ensure that the black and white version of the shot looked seamless, but having access to the original colour was a good indicator when elements of the shots weren’t quite sitting together properly.” In regards to the visual research which had to be conducted, Macdonald remarks, “The style of the film had to come first and foremost. We had a lot of reference for the look of certain practical elements, and a lot of our work, for example the goop from the exploding sea monkey, was based on this. For the water that comes out of Sparky’s back when he is drinking from his water bowl, they had shot some tests which they liked the look of, but didn’t feel that it would be practical to get the motion of the water with Sparky jumping around. This reference formed the basis of our CG water.”
“Our biggest challenge on the film was the popcorn sequence,” states Hugh Macdonald. “This involves an over-filled popcorn machine that overflows and fills up a tent, populated with animated sea monkeys. The tent fills, expands and eventually bursts, showering popcorn and the ‘goop’ that the sea monkeys turn into all over the place. The main challenge here was how to get the stop-motion popcorn that was inside the popcorn machine to transition to CG popcorn that would then fall around the sea monkeys, filling up the tent whilst still interacting with their movement. To achieve this, we built proxies of the sea monkeys and animated them to match the real sea monkeys. We then had a particle system consisting of a million particles, with various different types of popcorn instanced. These interacted with the sea monkeys, and came to rest on a floor that was animated up to simulate the popcorn piling up. The goop, for both the tent explosion shot and the close-up of a sea monkey exploding, was another particle system that was meshed. Most of the particles were general ‘goop’, but a small proportion of them were made larger and stickier, to represent the pieces of popcorn inside the goop. For the look of the goop, we had a practical reference shot of Bob having the goop run down his face. We used this to match the look and feel of our CG goop, including the translucency and viscosity.”
“The drinking sequence was the first one of these that we tackled,” reveals Hugh Macdonald. “For this, we used a similar solution to the one we used for the goop – a meshed particle system, with a water shader. They had shot some practical water reference, which we were able to use as a look reference, and then created our dynamic system that would be controlled by Sparky’s movement, and splash onto the floor, leaving puddles and damp patches. To give the final water effect, we ended up with four separate particle systems, each with a certain amount of interaction between them. The master system was the main stream of water in the air; to enhance where it came out of Sparky’s back there was also a spray pass which was a finer mist at the base. As the stream hit the ground it generated two new systems – the small splash, which only appeared for a few frames for each drop, and then the pool which appeared on the ground. The pools also interacted with Sparky’s feet whenever he stepped in them. The swimming pool was done using a completely different method. We investigated a few solutions for it; including full-on fluid simulation, and generating 2D height maps and passing these through to 3D for the surface itself. The solution we eventually settled on was a 2D one, based on noise patterns and simulating reflections and refractions. This generic motion was then enhanced with ripples generated by any objects interacting with the surface, including the pole and the sea monkeys that Bob initially pours into the pool. For the shot in which the lightning hits, we used the same system that we had built for the water bowl sequence to generate the water splash.”
“The period of the film where the other children’s pets are all being brought back to life was one of the major parts of the film for us,” says Macdonald. “Each pet’s resurrection had a different challenge for us, including the Sea Monkeys coming out of the swimming pool, which I’ve previously mentioned. The turtle accidentally brought back with Miracle Gro which causes it to grow to a giant size, involved combining different scales of model especially for the shot where he crushes the shed. When Weird Girl’s cat is zapped by the lightning, creating the cat/bat hybrid, our main task was to integrate the lightning effect for the sequence, plus the aftermath of the smoking bed. We had one artist who specialized in creating the lightning effect for all of the sequences that we worked on; he was able to quickly and easily add or tweak any lightning effect, giving us the flexibility to be able to quickly ramp up or down the effect to give the required result. The last of the resurrection sequences that we were involved with was when Nassor brings his hamster back from its crypt. In the shots where we reveal the newly-resurrected hamster, we added dry ice elements rolling out of the entrance, and ensured that movement of the huge shadow that the hamster cast was accurately tied in with the far smaller hamster that appeared in the entrance.”
“My goal with these sequences was to make the environments subtle, and to ensure that nothing screamed ‘visual effect,’” states Hugh Macdonald. “These sequences were all about the characters and the emotion of the scene, so our job was to ensure that the setting was seamless. In the final scene of the film, this involved dropping in a dark background and adding atmosphere to the beams from the car headlights. Nothing we did could look anything other than as if they had been able to shoot the whole scene for real, and this was forefront in my mind when we were working on it.” Having to deal with stereo images made designing the visual effects more challenging. “When a film is post-converted to 3D, it is filmed in 2D and then made 3D in post production, any visual effects shots need to go through a process that is known as ‘bag and tag’. This is when the layers that make up the shot are split out separately for the 3D conversion team; in a way that would make them easy to re-combine to form the final shot, but also allows the conversion team to add depth to each layer separately, and then recombine them to create the second eye. This process adds a certain amount of complexity at least when compared to traditional visual effects shots; if a shot is approached with this in mind, it can be composited in a way that allows for the bag and tag elements to be extracted.” Reflecting on the $39 million production, Macdonald remarks, “It was a pleasure working with Tim Burton, Allison Abbate and Tim Ledbury. Working on a film like Frankenweenie is ultimately satisfying and rewarding because of the unique style of the director. To have been involved with this film, which has been a project in Tim Burton’s imagination for 30 years, was an honour. Thanks also to our amazing team at Nvizible, including VFX Producers Gil James and Kim Phelan, CG Supervisor Martin Chamney, and all of the artists who worked on the film with us.”
Many thanks to Hugh Macdonald for taking the time for this interview.
Trevor Hogg is a freelance video editor and writer who currently resides in Canada.
“Nvizible had previously worked with VFX Supervisor Tim Ledbury on Fantastic Mr. Fox [2009], Wes Anderson’s stop-motion film, and he approached us to work with him again on Frankenweenie [2012],” explains Hugh Macdonald who is a VFX Supervisor and the Head of Technology for the British visual effects facility. “On Fantastic Mr. Fox, we got involved right towards the end of the post production period, helping out with some of the left-over shots that needed completing. Conversely, on Frankenweenie, we were involved from a far earlier stage; this gave us the ability to take a longer view of the work and to visit the set so to have a better understanding of how it was to be shot.” Ledbury was able to visually articulate what he was looking for to Macdonald and his team. “As someone who has a background as a matte painter, Tim was able to quickly put together rough digital paintings illustrating the look and palette of a shot. We were also working on a number of sequences and looks alongside his in-house team, which gave us ideal reference to match to. There were also a number of sequences where we came up with a number of look options ourselves, and submitted these both Tim Ledbury and Tim Burton [Alice in Wonderland] to decide on the final look.”
Hugh Macdonald |
“If anything the fact that the final result was to be black and white made our lives easier,” observes Hugh Macdonald. “It was shot in colour, and we had the original colour footage to work on, which was essential for any shots involving green screens. We also had the LUT that gave us what the final result was going to look like. When working on the shots, artists would have the ability to view the result in both colour and black and white. Their primary aim would be to ensure that the black and white version of the shot looked seamless, but having access to the original colour was a good indicator when elements of the shots weren’t quite sitting together properly.” In regards to the visual research which had to be conducted, Macdonald remarks, “The style of the film had to come first and foremost. We had a lot of reference for the look of certain practical elements, and a lot of our work, for example the goop from the exploding sea monkey, was based on this. For the water that comes out of Sparky’s back when he is drinking from his water bowl, they had shot some tests which they liked the look of, but didn’t feel that it would be practical to get the motion of the water with Sparky jumping around. This reference formed the basis of our CG water.”
“Our biggest challenge on the film was the popcorn sequence,” states Hugh Macdonald. “This involves an over-filled popcorn machine that overflows and fills up a tent, populated with animated sea monkeys. The tent fills, expands and eventually bursts, showering popcorn and the ‘goop’ that the sea monkeys turn into all over the place. The main challenge here was how to get the stop-motion popcorn that was inside the popcorn machine to transition to CG popcorn that would then fall around the sea monkeys, filling up the tent whilst still interacting with their movement. To achieve this, we built proxies of the sea monkeys and animated them to match the real sea monkeys. We then had a particle system consisting of a million particles, with various different types of popcorn instanced. These interacted with the sea monkeys, and came to rest on a floor that was animated up to simulate the popcorn piling up. The goop, for both the tent explosion shot and the close-up of a sea monkey exploding, was another particle system that was meshed. Most of the particles were general ‘goop’, but a small proportion of them were made larger and stickier, to represent the pieces of popcorn inside the goop. For the look of the goop, we had a practical reference shot of Bob having the goop run down his face. We used this to match the look and feel of our CG goop, including the translucency and viscosity.”
“The period of the film where the other children’s pets are all being brought back to life was one of the major parts of the film for us,” says Macdonald. “Each pet’s resurrection had a different challenge for us, including the Sea Monkeys coming out of the swimming pool, which I’ve previously mentioned. The turtle accidentally brought back with Miracle Gro which causes it to grow to a giant size, involved combining different scales of model especially for the shot where he crushes the shed. When Weird Girl’s cat is zapped by the lightning, creating the cat/bat hybrid, our main task was to integrate the lightning effect for the sequence, plus the aftermath of the smoking bed. We had one artist who specialized in creating the lightning effect for all of the sequences that we worked on; he was able to quickly and easily add or tweak any lightning effect, giving us the flexibility to be able to quickly ramp up or down the effect to give the required result. The last of the resurrection sequences that we were involved with was when Nassor brings his hamster back from its crypt. In the shots where we reveal the newly-resurrected hamster, we added dry ice elements rolling out of the entrance, and ensured that movement of the huge shadow that the hamster cast was accurately tied in with the far smaller hamster that appeared in the entrance.”
“My goal with these sequences was to make the environments subtle, and to ensure that nothing screamed ‘visual effect,’” states Hugh Macdonald. “These sequences were all about the characters and the emotion of the scene, so our job was to ensure that the setting was seamless. In the final scene of the film, this involved dropping in a dark background and adding atmosphere to the beams from the car headlights. Nothing we did could look anything other than as if they had been able to shoot the whole scene for real, and this was forefront in my mind when we were working on it.” Having to deal with stereo images made designing the visual effects more challenging. “When a film is post-converted to 3D, it is filmed in 2D and then made 3D in post production, any visual effects shots need to go through a process that is known as ‘bag and tag’. This is when the layers that make up the shot are split out separately for the 3D conversion team; in a way that would make them easy to re-combine to form the final shot, but also allows the conversion team to add depth to each layer separately, and then recombine them to create the second eye. This process adds a certain amount of complexity at least when compared to traditional visual effects shots; if a shot is approached with this in mind, it can be composited in a way that allows for the bag and tag elements to be extracted.” Reflecting on the $39 million production, Macdonald remarks, “It was a pleasure working with Tim Burton, Allison Abbate and Tim Ledbury. Working on a film like Frankenweenie is ultimately satisfying and rewarding because of the unique style of the director. To have been involved with this film, which has been a project in Tim Burton’s imagination for 30 years, was an honour. Thanks also to our amazing team at Nvizible, including VFX Producers Gil James and Kim Phelan, CG Supervisor Martin Chamney, and all of the artists who worked on the film with us.”
Production stills © Disney. All rights reserved. Images courtesy of Disney and Nvizible.
Visit the official websites for Frankenweenie and Nvizible. To learn more make sure to read Trevor's Tim Burton filmmaker profile, Freakishly Clever.
Many thanks to Hugh Macdonald for taking the time for this interview.
Trevor Hogg is a freelance video editor and writer who currently resides in Canada.