When it comes to weird projects, I have to say this is probably the one that takes the cake. It’s not every day that you get asked to find a way to put a bearded mouth on a hand and have it talk and interact with the person whose hand it is.
Our first approach was to do an object track on the hand footage and add a fully cg mouth and then add a cg beard on top of the geometry. While our first test was satisfactory, it was by no means efficient. Here’s a quick render of one of the bits from our R&D stage.
Knowing beforehand that doing it all in 3D wasn’t ideal, but having it as a fail-safe gave us the confidence to proceed in a completely 2D approach. When it came down to the beard not looking flat, the obvious choice was to have the beard already in the hand and then use some compositing tricks to place the mouth under it. Same with placing an ear on the top of the hand – much easier and more realistic to have a practical prosthetic ear than to have to add it in post.
Once we had a detailed shotlist, it was also apparent that tracking and compositing the mouth into the longshots would yield an unrealistic approach – instead better to have our prosthetics artist make a static mouth for those long shots and shoot that separately.
The plan of attack for compositing the mouth under the beard was to key out some green that was painted on the skin underneath the prosthetic beard (although a simple garbage mask would’ve worked instead of keying). That green area had tracking markers which we then used to attach not only the mouth but also an added layer of CG-generated hair.
The key to selling the composite was to have the movement of the skin around the mouth be affecting the surrounding beard. To achieve this we put in quite a few tracking markers around the mouth plates which would feed X and Y data to Mettle’s FreeForm Pro.
Once we got the look and feel we liked, we started applying that technique on all the shots. With the workflow being established and having the rest of the team feel comfortable with the tracking, compositing, and hair simulations we were able to get into the 3D area of the project.
One of the shots in the video called for a crab claw to come out of the hand. We had a real (and dead) crab claw on set to be used for the long shot, so we used the reference pictures taken on set to get a general feel of the shape but more importantly used the multiple angles to create a texture map for it.
I’m unsure of when I’ll be able to post the video on here, but you can find it on youtube.