A university research team recently developed a new method for real-time facial re-enactment that works with a regular web camera, overcoming several challenges in computer vision technology. It’s also creepy as hell.
In a paper, the researchers lay out some key challenges facing computer modelling: reconstructing a scene from a video is difficult for computers to do. The approach that they have taken allows them to analyse a film frame-by-frame, and alter the video as it plays. The resulting motion capture allows the team to record a subject’s facial expressions with an off-the-shelf camera and renders those movements as the video as it plays out, in real time. Again, the effect is both impressive and uncanny.
What’s especially interesting about this new method is how is how seamless it is. Here’s the video from the team:
The result here could be new software that easily allows for digital manipulation of videos, without a lot of specialised hardware. Or you could just spoof your friends into thinking Donald Trump said something that made sense.