Due to a sadistic groundhog and six more weeks of winter, Royal Pains was forced to pull off rarely accomplished “Winter for Spring” shots this past March. While shooting several episodes that kicked off their third season, the tree-lined streets of Montauk, Long Island were unfortunately a landscape of barren branches. Luckily the VFX artists at The Molecule, experts in “invisible” effects, were there to the rescue adding the thousands of necessary leaves to complete the shots.
“When they asked us, ‘Can you put leaves on the whole environment?’ we were hesitant because of the potentially impossible amount of work,” said Luis de Leon, Visual Effects Supervisor – The Molecule. “We decided that as long as we were able to block and frame the photography we could minimize a massive job.”
On set, Luis worked closely with the camera department and the director making sure actors didn’t step into the path of any soon to be digitally budding leaves and hair didn’t intermingle with branches. Luis oversaw the framing and actor’s blocking and made alterations when needed.
Back at the studio, VFX artists used Nuke to complete the more demanding tracking shots. Nuke’s 3D tracker system creates a 3D point cloud in which each point is connected to an object (or part of one) in the scene. In this case the leaf footage was connected to the points that correlated to the branches and trees. This allowed the leaves to stay in proportion as the camera moves on a 3D plane. Additionally, Nuke’s advanced technology allowed VFX artists to alter the leaves while keeping them proportionally sized in relation to their distance from the camera.
First, leaves consistent with the region are sourced, laid in and connected with points in point clouds. Then, after all the leaves are added the footage is reviewed, and shots where foreground action blocks the trees are rotoscoped. Finally the leaves are color corrected to make them look as if they are on the same tree and shot at the same time of day.
“The 3D tracks were tricky because of the layers of branches. When you have branches behind branches the 3D tracking system can’t always tell which branch is in the foreground and Nuke can incorrectly resize the leaves,” Luis added. “However, without Nuke’s 3D tracking ability, certain shots, like one that consisted of a 200 degree camera turn, would be difficult to pull off.”
We challenge interested viewers to keep on the lookout for these “invisible effects” as they are a large part of The Molecule’s daily VFX work but be warned: At The Molecule, the hand is quicker than the eye.