You may have missed it in between the colourful new iMacs and AirTag announcements, but the iPad Pro models that Apple introduced last week come with an interesting video-calling feature called Centre Stage, which is designed to keep you in the centre of the frame at all times. Here’s how it’s supposed to work, and the other video-calling tech it’s competing with.
The latest iPad Pro models obviously aren’t out yet, so we haven’t gotten any hands-on time with the new feature, but we do know quite a few details about it. It’s enabled by the new 12-megapixel ultra-wide front-facing camera on both the 11-inch and 12.9-inch iPads Pros announced last week (both will be available in May). That wider camera angle enables a digital cropping technique — the camera isn’t actually moving around, it’s just selecting different parts of its field of view to show.
The TrueDepth tech in the camera, which is able to detect objects and how far away they are from the device, also offers an assist, as does the machine learning capabilities built into the M1 chip that powers these top-of-the-line iPads. We’re assuming the chip deploys some AI magic to know when it’s looking at faces, making adjustments on the fly as those faces move around (and new faces appear).
“As users move around, Centre Stage automatically pans to keep them in the shot,” Apple says. “When others join in, the camera detects them too, and smoothly zooms out to fit everyone into the view and make sure they are part of the conversation. So whether whiteboarding with coworkers or attending a virtual family gathering, the experience of connecting is now more engaging than ever.”
The wide field of view of the camera and the power of the M1 chip is something Apple’s other devices will struggle to match — cheaper iPads won’t get the processing performance and MacBooks won’t get the camera. This could be an iPad Pro exclusive for a while, and indeed the tablet is probably already a device that users are comfortable propping up and using hands-free.
We’ve learned a few more tidbits about Centre Stage since it was announced: It will work with other apps besides FaceTime, thanks to an API for developers, and it won’t get human faces mixed up with the faces of your pets. Centre Stage will be a setting you can toggle on an app-by-app basis for apps like Zoom, Microsoft Teams, and more.
During its Spring Loaded media event, Apple showed off how Centre Stage works in the kitchen, which is definitely one scenario where it could come in handy, but really any situation where you don’t want to be holding your device and want to talk as you move around would work. Centre Stage will keep you in view at all times, unless you really try hard to get out of the frame.
This isn’t exactly new, though. A similar feature is one of the key selling points of the Facebook Portal devices, which are available both as digital frames and in the form of a camera that you can stick on top of your TV (which then doubles as the Portal display). These gadgets can also keep you in view as you move around, and again ultra-wide-angle cameras are used to pull of the trick. Facebook calls it a “smart camera” system that “automatically adjusts to stay with the action, whether you’re moving around the kitchen or chasing the kids through the living room.” As with the Apple implementation, the view will widen if more people join the shot.
The camera’s field of view ranges from 114 degrees to 140 degrees depending on the Portal product you go for, so you’re going to be able to get a lot of a room in the shot. You would almost (but not quite) have to be behind the Portal for it not to see you. All this happens seamlessly, whether you’re using WhatsApp, Facebook Messenger, or Zoom to make your video calls.
You can actually turn the Smart Camera feature off on your Portal, at which point the camera zooms out to the widest position and locks in place. The use cases are very similar to the ones for Centre Stage, with family get-togethers in the living room the one that Facebook seems to be promoting the most.
Then there’s the new Amazon Echo Show 10, which has a camera and a display that follows you around the room in a somewhat creepy fashion. If you can get over the sinister surveillance vibe, it’s perhaps an even better solution than what Apple and Facebook are doing to keep your face in view during video calls.
In the case of the Echo Show 10, it’s not some clever digital cropping and panning that’s going on. The camera is actually watching you and moving as you move. As an added bonus, you can use it as a security camera and pan around the room while you’re away from home. The base speaker that the display and camera is stuck on has 350 degrees of motion, so we’re talking a lot of rotation here.
Amazon says it uses a combination of audio and visual cues to figure out where you are and follow you around. Again, machine learning and AI algorithms are implemented to figure out where in physical space the human beings are, and again it supports multiple people, zooming out as needed to keep more faces in view. It can be disabled too, via Motion and Motion Preferences in Settings on the device.
There are no doubt more versions of Centre Stage to come, too. Having a video feed that auto-adjusts as you shift position or as more people join the call is undoubtedly a useful feature to have, and one that artificial intelligence techniques are well suited for. And maybe these features will actually spark joy when we’re not forced to video conference multiple times per day post-pandemic.