Facebook kicked off its annual F8 Developer Conference today in dramatic fashion when CEO Mark Zuckerberg announced an ambitious new augmented reality (AR) platform that will let people overlay digital images onto photos and videos in Facebook.
All images: Facebook
"We're going to make the camera the first mainstream AR platform," said Zuckerberg. He explained that the platform itself will open in closed beta today, and it will help thousands of developers from around the world build unique filters for the camera. These filters will eventually become available in all of the different Facebook-owned properties, including Instagram, Messenger and WhatsApp, and will be available to download for free.
"You're going to be able to swipe to your camera and swipe through the effects: Face masks, art frames, style transfers," he said. "But instead of a few options to choose from, you'll have thousands to choose from." Behind him, an image of an expanded filter option menu showed in the lower half of a mobile phone camera screen. It looked similar to an App Store, but instead of apps, there were thousands of different camera filters.
The update could be a seismic shift for Facebook, which has until now largely relied on text and photos for original content. The launch of these new augmented reality capabilities could encourage more people to share at a time when people are starting to share less on Facebook. So, with all of this in mind, here are five things you should know about Facebook's huge new augmented reality push.
1) Facebook knows exactly where you're standing
Facebook's new camera effects are doing much more than analysing information picked up from your camera. The new AR platform will also let developers use precision location data to determine how digital images are displayed.
One example of how this might work is at a restaurant. Your friend will be able to leave an augmented reality sticky note on the menu, letting you know which menu item is the best or which one's the worst when you hold your camera up to it.
Another example is if you're at a celebration, like New Year's Eve or a birthday party. Facebook could use an augmented reality filter to fill the scene with confetti or morph the bar into an aquarium or any other setting corresponding with the team's mascot. The basic examples are similar to Snapchat's geo-filters — but with more sophisticated uses because it will actually let you leave digital objects behind for your friends to discover. Very cool!
2) Facebook knows exactly what you're looking at
This is where things start to get a little crazy, and potentially creepy for sensitive people. Facebook already owns and uses some of the most advanced object recognition software in the world, and now it's opening up some of those capabilities to developers.
This means that if you're taking a photo of a coffee mug, Facebook will be able to detect that specific object and serve up different digital images or effects based on this fact. It could, for example, show a burst of confetti to encourage you to start your day or create a digital duplicate of the object. The options are basically limitless.
The reason why it's creepy is because this is just another avenue for Facebook to gobble up information about your personal life. It already knows about many of the places you hang out, news you read, and things you like. Now, through the use of the camera app, Facebook is going to be able to see in the room you're standing in and detect things like a Coke can or desk ornament. In the end, this information will be used to send more targeted ads directly to you.
3) Developers can add notes onto basically anything
One of the primary uses of Facebook's new AR platform will allow developers to layer information cards digitally onto the real world. Zuckerberg used the example of visiting the Coliseum and holding up a phone to learn more about the history and structure of the building.
This type of information card could actually prove to be really helpful, and it's hard to even imagine all of the different use cases for something like this. Basically, imagine being able to add a sticky note to almost any real world object so that your friends (or others) can see it later. The new AR tools that Facebook is giving to developers will enable that.
4) Developers can add digital objects into any setting
One of the more crazy elements of the platform will let developers add digital objects onto the real world. Zuckerberg used Pokemon GO as an example while speaking on stage, but mentioned that it will soon become way more sophisticated.
Rather than being able to see a little Pikachu down the street from your house, Facebook is going to let developers add almost anything to the real world. This means you'll be able to discover all sorts of crazy stuff while checking out the new camera filters as they're developed. We imagine this might work something like Snapchat's World Lenses but will offer a much great variety of objects that can be placed in your real world setting.
5) Making AR mainstream is an uphill battle
Even Zuckerberg admits that this could take a while to get off the ground. "It's going to take a while to develop. Your experience won't change dramatically overnight," he said during his keynote speech. "Over time, I think this is going to be a really important technology that really changes how we use our phones and eventually all technology."
But Facebook is not the first company to recognise this inevitable trend and try to capitalise on it. Google's early effort to make Google Glass popular is one of the most notorious product flops in the last decade, and Facebook will likely face the same type of challenges as it tries to popularise camera effects. It's also worth mentioning that Snapchat played an important role in popularising this trend and Facebook has been widely ridiculed for copying Snapchat.
Zuckerberg also admits that he was slow to catch onto the idea, and stopped just short of giving Snapchat credit for identifying the trend. "Even if we were a little slow to add cameras to all of our apps," Zuckerberg said, "I'm now confident that we're going to put this technology forward."