As much as video-calling technologies like Facetime and Zoom have helped the world stay connected over the past year, they’ve also demonstrated their severe limitations, as it’s hard to feel emotionally connected to someone through a low-quality video feed. Today, Google revealed a technology that’s not quite ready for primetime but promises to make the other person on a video call seem like they’re sitting right in front of you.
It’s true that 3D TVs and movies died a much-deserved death several years ago, but it wasn’t because people didn’t want 3D — that’s how humans see the world around them, after all — it was because the 3D effect wasn’t all that great, and often relied on awkward accessories like battery-powered shutter glasses that had to be worn the entire time. With Project Starline, Google is still leveraging several different technologies, but they all do their thing in the background, so the experience for the user is still just staring at a screen, but what they’re seeing is markedly different.
In addition to using high-quality video cameras to record each person on a video call, Project Starline also uses a depth camera to create a real-time 3D depth map of everyone on the call. All of that data — the video and the depth information — is then compressed, sent across the internet, and reconstructed on the other side to create a real-time 3D representation of each person, but one that looks photorealistic, and not like something out of the unsettling uncanny valley: a problem that often plagues people recreated in CG.
Project Starline also relies on a technology known as light field displays which are able to produce different visual information depending on the angle they’re viewed. What that means to the viewer is that the person they’re chatting with on a video call suddenly has depth, and their movements exhibit an effect known as parallax (where objects in the distance appear to move slower than those in the foreground) which is part of how humans see the world. So even just slightly leaning from side to side while sitting on a call will make the other person appear to properly shift in 3D space, which goes a long way to making the human brain think it’s looking at a real person, and not just video on a 2D screen.
There’s no timeline on when Project Starline will become a real product, if ever. The technology used, including the need for a very fast and reliable internet connection, doesn’t come cheap so it’s probably not going to arrive in smartphones anytime soon. But in a blog post, Google does mention that “We look forward to sharing more later this year” so maybe Project Starline will be more than just a neat tech demo destined to forever remain in the company’s R&D labs.