A New York Times report out Friday identifies more than 250 mobile games in Android's Play Store that use always-on audio recognition software to track users' TV- and ad-viewing behaviour. Experts cited in the report also say the software's developer, a startup called Alphonso, does not clearly describe how the games, some of which explicitly target children, use players' microphones.
Image: New York Times
As the Times describes, the apps take advantage of mics in players' tablets and smartphones to listen for "audio signals," which can detect if someone is watching a specific ad, TV show, or movie. This ambient surveillance is ongoing, even if the games are running in the background, and even if the game itself doesn't necessitate a microphone to play. Users are, of course, prompted for permission to access their microphones, but one expert quoted in the piece argues the wording is still misleading.
"When you see 'permission for microphone access for ads,' it may not be clear to a user that, Oh, this means it's going to be listening to what I do all the time to see if I'm watching 'Monday Night Football,'" Justin Brookman, director of consumer privacy at Consumers Union, told the Times. "They need to go above and beyond and be careful to make sure consumers know what's going on."
Founded in 2013, Alphonso's speciality is "TV retargeting." Because people tend to turn to tablets and smartphones during TV commercial breaks, Alphonso collects viewing data to send targeted ads to mobile devices based on what consumers are watching. Alphonso reportedly has a deal with Shazam, the well known music identification app that Apple recently acquired.
According to Alphonso's chief executive, Alphonso's technology picks up snippets of audio information, then the company has that information recognised and categorized by Shazam, and then Shazam sells that information back to Alphonso.
Some of the games utilising Alphonso's technology are also available in Apple's App Store, the Times reports.
On its site, Alphonso markets its ability to use targeted advertising to "reach audiences of TV shows that [brands] are not currently buying." A makeup brand, for example, may want to target viewers of Keeping Up with the Kardashians. This way, they can buy ad time from the network and send ads to viewers' phones because it knows what you're watching.
"A lot of the folks will go and turn off their phone, but a small portion of people don't and put it in their pocket," Ashish Chordia, Alphonso's CEO, told the Times. "In those cases, we are able to pick up in a small sample who is watching the show or the movie."
Always-on listening devices like Amazon's Echoes are becoming increasingly common. And even if these devices are designed to capture minute snippets of information, people are wary of ways their conversations may end up, at best, in a targeted advertising database. Vizio lost a lawsuit in February because of how its TVs collected data from consumers, while Mattel cancelled the Aristotle device, a smart speaker designed for children.
But Alphonso's case is arguably much creepier because users may simply not understand what they're agreeing to. As ubiquitous audio recognition tech becomes commonplace, it might be time for us to consider that someone, or something, is always listening.