There are times when you can’t remember the name of something but still need to find it — and a new AI wants to solve that problem when it comes to movies.
To do that, it chews through the raw footage of movies to spot actors, action, locations and other details about scenes that it can use to provide answers to more human search terms. So you can search for something like “Sean Connery with torpedoes” and you will indeed find The Hunt For Red October. Valossa explains what it actually does:
Descriptive movie search is based on our research on what is called “Deep Content”. Deep Content is everything we can see and hear in a video, but cannot mechanically analyse — until now. Deep Content includes transcripts, audio, visual patterns and basically any form of data feed that describes the video content itself. After analysing the deeper levels of the video, we automatically convert it into advanced metadata. This metadata is then processed by the beating heart of our engine: a cognitive machine learning system that understands natural language queries and matches it with our metadata.
As Engadget points out, this is unlike other systems that search through films, because it actually uses the raw footage: Others rely on scripts, subtitles and the like. So far, the new AI has churned through 40,000 movies.
It’s not perfect, of course: Valossa points out that the “demonstrations on this site have been developed for research purposes and Proof of Concept for the industry. For instance, searching for “that movie where Robin Williams pretends to be a woman” doesn’t turn up Mrs Doubtfire.
But why not try it out anyway?