The phrase cultural analytics generally refers to analyzing vast amounts of image, text, or other media through computational methods. Think of it as data science aimed towards arts and culture. But unlike data science, cultural analytics isn’t necessarily asking political-social-economic questions. Rather, cultural analytics seeks to help us see the world in a new way, generating more questions than answers.
In this lab we’ll attempt a special kind of cultural analytics. Instead of looking at a vast number of texts (say, the way Ben Schmidt analyzes State of the Union addresses, or how Lev Manovich analyzes Instagram selfies), we’ll break apart a single text—a film—into a vast number of discrete parts, and analyze those parts in the aggregate. Some researchers call this technique “image summation.”
Elements of this procedure have been adapted from Dr. Brian Croxall’s similar exercise at Brigham Young University. Thanks, Brian! We’ll also be using an online image analysis tool developed by Dr. Zach Whalen at the University of Mary Washington. Thanks, Zach!
First, you’ll need to extract still images from the film that you’ve ripped or otherwise acquired.
You can do this with the open source media player VLC. After you’ve installed VLC, you need to change its settings in order to extract frames. Make these changes before you open your video file in VLC.
- Go to preferences.
- Click “show all”
- Click on “video”
- Click on “filters”
- Find “scene video filter” and check that box
- Scroll under “filters” to find “scene video filter.” Select it to edit its preferences.
- Paste in a directory path for where you want the screenshots to be collected.
- Set the “recording ratio” to be how often you want a still to be grabbed. For our purposes, you can set this to “60.” This grabs one frame every two seconds. (Because the video runs at a 30fps rate.)
- Click save.
- Open the video file in VLC and let it play.
- Your still images should start appearing in the directory you named in #7. If no extracted frames appear in that directory, make sure you have the correct directory name, and load up the video again through File > Open.
This method extracts frames in real time, which means it will take several hours (as long as the film) to extract all the images. Obviously, we don’t have enough time in class to complete this process. You’ll work on your own film outside of class. For the purposes of class, I’ve extracted frames from three different works: “The Entire History of You” from Black Mirror, The Fast and the Furious, and the first episode of Game of Thrones. You can select one of these three videos to use during class.
For analysis of our images, we’re going to use Imj, a web-based image analysis tool. As the tool’s creator, Zach Whalen says, this technique isn’t all that powerful compared to other desktop-based tools, but it does “enable some low-level visualizations that might help researchers or students determine whether an investigation with more robust tools is warranted.”
In particular, Imj supports three types of visualizations: barcodes, montages, and scatterplots. Basically, you upload your folder of extracted frames (up to 9999 frames), and let Imj do the work.
For the purposes of writing your lab report, you’ll use Imj on a film of your own. Follow the instructions above for using VLC to extract frames. Then subject your video to all three visualization types. Download the results (the barcode, montage, and plot) in order to include these images in your lab report.
In a 300-500 word lab report, reflect on some of the following questions:
- What does each resulting image type tell us about the film?
- What elements of the video stand out through these visualizations? What elements disappear?
- If you compare the resulting image summations with each other, which one is most useful? Define what you mean by useful.
- What did you learn from these visualizations that you couldn’t have learned by watching the film alone?
- Many times the power of these image summations comes not from the analysis of an individual film, but from a more longitudinal of multiple videos. For instance, Dr. Kevin Ferguson has analyzed every Disney animated film with these techniques. Or imagine comparing every episode of a television series to see if the series’ visual signature changes over time. Or comparing barcodes of 30 years of horror movies. What kind of comparative analysis would you like to do if you had the time and resources? What would you hope to learn through such a comparative analysis?
Share the report with email@example.com as a Google Doc by end of the day, Monday, November 20. (Remember there is no class on Monday, November 20).