When we watch movies, we pay attention to specific parts of the scene and focus on different actors and look at certain things on the screen. Most of what we see is influenced by what the directors want us to see, our attentions are easy to grab after all. So what if you showed the same movie to computers? What would they see? How would the movies look to them?
Benjamin Grosser created a computational system to reveal what computers see in movies. He wrote software using vision algorithms and artificial intelligence routines to give “the system some degree of agency, allowing it to decide what it watches and what it does not”. The vision of the computer is reflected in temporal sketches, the sketching is synced with the movie’s audio. Grosser asks:
Viewers are provoked to ask how computer vision differs from their own human vision, and what that difference reveals about our culturally-developed ways of looking. Why do we watch what we watch when we watch it? Will a system without our sense of narrative or historical patterns of vision watch the same things?
The movies the computers ‘watched’ are 2001: A Space Odyssey, American Beauty, Inception, Taxi Driver, The Matrix, and Annie Hall. Grosser’s project, which was found by Laughing Squid, shows the difference between how tightly wound the focus is in certain scenes of 2001 and how messy action sequences of Inception can get.
I’d love to see these computer watching habits superimposed on the movie scenes and how they compare to human watching habits too. Though it is interesting to watch the movie play out with simple lines. You can see the individual scenes here.
2001
Inception
The Matrix
American Beauty