It’s no secret that Facebook has an interest in virtual and augmented reality, but today, the company published a deep dive into the audio component of this emerging tech. Put simply, Facebook’s trying real hard to make “smart audio” a thing.
At the heart of Facebook’s research is a concept called “social presence.” The problem with VR and AR right now is even if you slap on a headset and are visually transported to a different surrounding, if the sounds you hear don’t match up, it can take you out of the experience. A division of the company, Facebook Reality Labs Research, has purportedly been working on figuring out a way to create virtual sounds that are “indistinguishable from reality” and, perhaps somewhat ominously, “redefine human hearing.” The former is relatively self-explanatory, but FRL Research defines the latter as the ability to amplify specific sounds while dampening irrelevant ambient noise. It’s sort of akin to how superheroes and spies in movies can zero in on a conversation despite being in a noisy setting.
[referenced id=”1233120″ url=”https://gizmodo.com.au/2020/07/apple-glasses-rumours-have-me-stressed-about-ars-biggest-unsolved-problems/” thumb=”https://gizmodo.com.au/wp-content/uploads/2020/07/23/xxohibfyfy92s0thpk5d-300×169.jpg” title=”Apple Glasses Rumours Have Me Stressed About AR’s Biggest Unsolved Problems” excerpt=”The hype behind Apple’s rumoured augmented reality headset and glasses just won’t die. Every few weeks, almost like clockwork, some sort of Apple AR rumour or news makes its way through the tech blogosphere, even as other smart glasses and AR companies are capsizing left and right. This has always…”]
So how does that relate to smart glasses? The FRL Research blog goes super in-depth into the audio science behind it, but the gist is that AR glasses could pick up audio via microphones and then, by analysing your head and eye movements, identify what you’re most interested in hearing. At that point, the glasses could enhance certain sounds while dampening others. So if you’re in a loud bar and you look at a TV, maybe the sounds on that TV become a bit clearer while the drunkard blathering in the corner gets a little quieter. This could also be used in conjunction with Facebook’s other AR-based tech, like LiveMaps, a “virtual map with shared and private components.” The example Facebook gives in this instance is if a person walks into a loud restaurant, the glasses could identify different noises — the air conditioner, clanking silverware, conversations — and then, using AI, remove distracting sounds.
Another application might be if you’re on a call, the glasses (or maybe a VR headset) could project audio to specific parts of the room. So, if you’re on a virtual conference call and your boss is “sitting” in the virtual chair to your left, it could adjust the audio so it really sounds like your boss is right next to you in the same room.
All this makes sense from an AR and VR perspective, but there’s also a neat accessibility angle. What FRL Research is describing is similar to tech currently used in hearing aids. So theoretically, it’s possible that this research could be used to help those with impaired hearing or folks experiencing hearing loss.
“Hearing sciences is an area that we’re starting to explore separately from our work on AR glasses,” Facebook said in its blog.
As far as the smart glasses landscape goes, Facebook is not the only company that’s dabbled in audio-focused augmented reality. Bose also released a pair of audio-only AR glasses, the Bose Frames. The audio AR on the Frames was pretty cool in practice, but nowhere near as advanced as what FRL Research describes. While you could get real-time directions or privately listen to your music in public, it wasn’t capable of things like noise cancellation. And, sadly, Bose shut down its AR division a few months ago.
That said, it’ll be a long time before we see any of this tech. Last September, CNBC reported that Facebook was thinking of launching its consumer AR glasses sometime between 2023 and 2025, meaning we’re still at least three years away from any potential launch. Still, this new research could very well be the fruit of a move last year in which Facebook allocated more resources to its smart glasses project. Now we just have to hope that Facebook’s rumoured partnership with Ray-Ban signals its AR specs won’t look completely dorky.