SuperHeroes Lab: Face Tracking & Head Replacement

Last week us here at SuperHeroes were busy doing some RnD using matchmoving and face tracking for an upcoming project.

There might be some slightly nsfw content in there. Don’t say I didn’t warn you.


The results are far from final, but good enough to show the client our approach. Something not featured in the video is how we used a Microsoft Kinect to do a scan of Chris’s face to help us out when we incorporated Object/Geometry Tracking.

Besides that we took the reference photos, applied them to a base mesh that somewhat resembled Chris, but lacked the details of the Kinect scan. Both proved to have their pros and cons, but in the end the Kinect scan was the clear winner for more precise results.

Besides that we also made our own prototypes for tracking rigs, all purchased from the market around the corner (handy!). Our first attempt was a headband with 3 sticks attached to it, with tracking markers at the end of said sticks. While that gave us lots of parallax and tracking data, it wasn’t sturdy enough and the wiggling around was giving us bad data.

Our solution was to acquire an Orgasmatron (weird name, I know), and attach a single (but larger) tracker on top. The latter tracking rig yielded the best results, so we plan on developing that one further for the actual project.


And here’s some screengrabs of what you don’t get to see in the video.



  1. Pingback: Stichting Lezen: 2.5D Worlds with the help of Kinect scans | Motion Sketches

    • Hey Tilan, we didn’t develop apps as much as we used existing software. For the 3D geometry tracking we used PFTrack, for 3D scanning I believe we used Skanect, and everything brought together in Cinema4D.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: