Stichting Lezen: 2.5D Worlds with the help of Kinect scans

I’ve not been posting lately, much of that has to do that I’ve been super busy with work lately.

One of the projects that’s consumed a lot of time (but has been a treat to work on) are these two spots for Dutch non-profit called Stichting Lezen (The Reading Foundation).

In a nutshell: these spots aim to promote reading out loud to boost creativity and imagination. One spot is about Dutch Sesame Street presenter Gerda Havertong reading a book about sea adventures to kids, while the other spot is about Dutch journalist Merel Westrik reading about how the time is ripe to travel abroad. I give this context to all the non Dutch speakers out there.

This was one of those multi-faceted projects. It involved quite a bit of PrePro and RnD (see the earlier post on when we first tried 3D face tracking), as well as a nice amount of 2D animation before we even began shooting. Motion intern Fredrik Dejert (two Freds in the Motion Dpmt, madness I tell you!) helped out with the 2D animation as well as got to get his toes wet with some deeper compositing.

My role was that of VFX Supervisor on set, 3D tracking, and then Motion and Compositing Lead for the rest of the spots.

Here’s a Making Of video for the process involved:

I brought a Kinect on set to do a scan of whoever was going to get the VFX treatment of the illustrations popping out of the head. Usually I use Manctl Skanect for the scans, but seeing as to how I was on a Mac platform on set I had to find an alternative: KinectToSTL. The main reason why I like Skanect is that you can get a fairly decent 3D 360 turnaround relatively painlessly, whereas KinectToStl gives you a scan that looks like Han Solo frozen in carbonite. Basically that means having to stitch up a front scan and two profile scans together in C4D later.

The reason for the scans? To assist in 3D geometry tracking – I find that the quirks of an individuals face helps the tracker substantially and gives a cleaner and smoother track quicker. Quirks such as bridge of the nose, or interesting forehead inclination, nose angle, etc…

2D animation was done in After Effects, and then 3D animation and reveals were done in Cinema4D using mostly MoGraph and the occasional dash of the popular polygon unfolding script by Rown.

When it came to compositing, it was fairly straight forward with rotoing and masks for the background layers/planes. However for the foreground we had some difficulties: the subtleties of overlapping hair. The solution for this was to simulate sections of hair where each foreground layer was, and then use that rendered hair as a luminance matte. While not perfect, it was a makeshift approach that

The end result is a mix of nice 2D animation layered in a 2.5D space.

Director: Tobias Pekelharing
Creative Director: René Klinkhamer
Producer: Severien Jansen
Ilustration: Eva Galesloot (via Shop Around)
Motion, VFX and Animation: Fred Huergo & Fredrik Dejert
Sound design: Sander Houtman

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: