As a Creative Technologist at Wildbytes, I took part in the development of this pioneering real-time face projection mapping performance for the launch of Kat Von D Beauty by Sephora.
I was responsible in collaboration with Nacho Cossio for building the software pipeline in Unity, synchronizing real-time face tracking data with a 3D head model, and enabling adaptive videomapping with sub-frame precision. Additionally, I collaborated as a hardware advisor, contributing to projector calibration, camera rig alignment, and optical distortion correction.
The custom software included:
- Real-time warping and projection mapping
- Face texture alignment and lens correction
- Video playback control inside Unity
-
Dynamic shader effects simulating digital makeup
System Overview
- 5-camera head tracking system capturing facial movement at 240 fps
- Unity-based visual engine rendering live on a 3D facial mesh
- Real-time projection with a digital makeup layer using custom shaders
This performance transformed Kat Von D’s face live on stage, merging makeup artistry with cutting-edge computer vision and immersive technology.
▶ Watch the Project
Kat Von D’s Live Face Projection Mapping on Vimeo.