Mary and The Monster
Lead Technical Artist
NYU | 2019 | Magic Leap One Headset
Mary and the Monster: a shared AR experience
Mary and the Monster is an immersive AR/VR theater experience inspired by Frankenstein. I led the graphics pipeline, covering animation, 3D assets, texturing, and lighting, and helped bring the experience to Oculus Quest and Magic Leap. The project was showcased at SIGGRAPH 2019.
Capturing the Soul of Performance
For Mary and the Monster, we used motion capture and facial tracking to bring the cast’s performances to life. I helped set up capture sessions, clean the data, and retarget performances to digital characters, ensuring expressive and real-time–ready animation.
Facial Expressions conversion to Data
Facial animation was captured with Faceware and retargeted onto character rigs to preserve expression and emotion. I refined the data by smoothing transitions and adjusting lip-sync to ensure accurate, expressive performances in real time.
The Faces Behind the Performances
Each character was sculpted at high resolution and paired with an automatic Faceware-compatible facial rig, allowing captured performances to drive expressions directly and reducing the need for manual keyframing.
Refining Performance Through Motion Data
After capture, motion data was cleaned and retargeted in Maya, with subtle adjustments added for weight, timing, and clarity before being exported to Unity for use in the AR/VR experience.
Trailer : Mary and the Monster
This trailer showcases Mary and the Monster, an immersive AR/VR performance blending theater, motion capture, and real-time storytelling. Live performances were translated into digital characters and environments, creating a bridge between stage and virtual experience.
