Matthew J. Westerby opens his article “Annotating Upstream: Digital Scholars, Art History, and the Interoperable Image” with the story of how downstream uses of the British Library’s IIIF images and manifests lost their own access when the Library was the subject of a cyberattack. This domino effect showed a “change in the relationship between institutions and users of online collections in which separate layers exist to serve images and also to present the digital surrogate of a real-world object like a book.”1 Similar complex layers of delivery and downstream/upstream usage have been on the minds of video game scholars for a long time. There is always a layering between the game’s software and the hardware or the display, as well as between the game as code and the physical object that carries that code, such as a cartridge. Especially with online-all-the-time consoles, a game can be pulled or updated or changed at any time without user consent or warning. These delivery layers are perhaps more apparent to the average gamer than the average user of an IIIF image art surrogate because they have demonstrably affected how games are released and experienced. It is now common (though not acceptable) practice for a game to release in a broken state with day-one patches to download. Patching post-launch was a near impossibility before internet-connected gaming, so the uneven quality of games at release has been felt by all video gamers.
This is not the only place where Westerby’s suggestions and insights for art history run parallel to emerging practice in game studies. Annotating video games is one way game studies is looking to share knowledge weither via annotating games themselves, annotating screen captures of video games, or through examining existing annotation practices like wikis and let’s plays that are a part of the larger life of “networked hyperlinked videogames”.2 For the practice part of this post, I used Panopto to experiment with annotating a brief screen capture of me playing The Blood of Dawnwalker.3
Panopto was the assigned tool for class but I didn’t find it particially useful for the annotations I would like to make where it would be helpful to have them show up on specific parts of the screen during viewing.
One of the reasons wikis are such a robust source of information for video games is that they are such complex artworks that one person could never annotate a full game by themselves. It also adds to the robustness of scholarship when sharing is part of the process, not just the outcome. The Zelda object-focused digital humanities project4 that I mentioned in my post about Collection Builder was one project that used this style of sharing. My portion was about multiplicity and repetition of objects in the Zelda series, focusing on small keys, but I was doing it alongside Sydney Bauer whose object was a single American painting, Hannah Iovanna who was examining the still life paintings of a single artist, and Addie Magyar who worked with me on the Zelda portion and also acted as coordinator (Magyar was actually who noticed that small keys are the only constant in all the games). Being able to do those separate case studies with the same methodology while in communication during the process made the projects much richer and more enjoyable. Our discussions and brainstorming sessions brought up the question that I swear one day I will write about: why is there always a wheel of cheese with one little slice out in every video game? I sometimes wish I had picked the history of the wheel of cheese with one little slice out as my art history thesis: a study of why it is the most filling and delightful of foods across time and medium.
Annotating interoperable images is a mode of scholarly communication that is fundamentally different from the traditional sharing of only finished writing in journals and academic press books but no less a creative or scholarly act.5 When there’s a wider variety of acceptable ways to communicate information, writing a book or a journal article becomes a choice, not an obligation. The quality of all scholarly communication increases as forms increase because the output mode is not just one default option regardless of the needs of the argument or the data. This also allows for more modes of scholarship that cannot be studied or communicated as well via paper alone, like GIS mapping projects from my previous post. I was unfamiliar with the idea of thick data or data pollution6 as terms when I wrote the mapping post, but they are great terms for the concept I was exploring at the end of that post: messy data that balances the line between being so messy it’s no longer a useful tool but isn’t so clean that it carries unquestioned authority with data pollution also covering the source of data like AI or how data is used.
Westerby’s choice to end the article with how you can’t control other people’s usage of your images or metadata if you share them, and how the fear of misuse should be a known risk but not a reason to abandon openness was nuanced and powerful.7 There will always be people who take good things and warp them. Making decisions based around absolute prevention of misuse is an impossibility and an unnecessary burden. It is better to work from being risk-aware, with practices like the usage tags that Westerby suggests.8 It reminded me specifically of the Wretched & Alone SRD,9 a system written by Matt Sanders for making solo journaling games in the style of Chris Bisette’s game The Wretched.10 The Limitations section of the SRD includes “Finally, nazis, homophobes, transphobes, and other bigots are expressly forbidden from making new content using the Wretched and Alone system. If you have to ask whether this includes you, it does.”11 Are parasites going to ignore the usage terms? Likely. Still, by making clear the usage and specifically taking a stance on who is forbidden to use the material, bigots cannot hide behind not knowing or accidental infringement. It makes the deliberate nature of the theft apparent.
Matthew J. Westerby, “Annotating Upstream: Digital Scholars, Art History, and the Interoperable Image,”Open Library of Humanities 10, no. 2 (2024), https://doi.org/10.16995/olh.17217.↩︎
Austin Anderson, “Dark Souls as Networked Hyperlinked Videogame,”Electronic Book Review, ahead of print, September 28, 2025, https://doi.org/10.64773/1f7a-df6c.↩︎
“The Blood of Dawnwalker – A Dark Fantasy Narrative Sandbox RPG,” The Blood of Dawnwalker – A Dark Fantasy Narrative Sandbox RPG, accessed October 5, 2026, https://dawnwalkergame.com/us/en/home.↩︎
This week I explored the digital art history tool imgs.ai and got some interesting results. imgs.ai provides “a Web-based interface, and machine-learning-based backend that allows the end user to both descriptively and visually search arbitrary image dataset.”1 More simply put, it allows a human to guide an application using computer vision to find images in selected museum databases that are the most similar to an uploaded image, text prompt, or both. The idea being that the user will then continue to engage with more images by selecting from the suggested images that are “positive” or “negative,” furthering the search with this new information.2
The ability to mark negative results is where I hit my first issue: it wasn’t an option for me. This is likely because I was using the site without a login (getting one takes days for approval). While I understand the necessity of gatekeeping databases in the age of AI scraping and how intentionally using AI is fundamentally different from random bots shutting down your site, the irony of the website being protected being built on AI scraping images was palpable. How many of the models imgs.ai uses were built using the same tactics the required login was keeping out? Even if copyright was the driving force instead of defense against bots, how many artworks outside of these museum collections had their copyright violated to train the models to “see”? I am not anti-AI, but I think they are questions worth asking. A team that calls out “Transparency” as one of five “essential criteria”3 should say more about the black box components at work beyond naming the databases or models used.4 Minimally, it should give an explanation of why options are locked behind sign-up instead of forcing the user to guess.
I also wouldn’t be as harsh in my critique if imgs.ai as a tool had provided me an experience that felt more like what the article outlined. To show where imgs.ai gave me interesting results and where it fell short, let me walk you through my experimental searches.
I started with the image from Vermis I: Lost Dungeons and Forbidden Woods I digitized in my last post.5
[Scanned image from Vermis I]. plastiboo, Vermis I: Lost Dungeons and Forbidden Woods, 11th printing, 2026. Used here for educational and commentary purposes; no copyright infringement intended. All rights remain with the original owner.
I knew this wouldn’t necessarily be an easy image with its dithered graphics, but I thought it was fair to work with an image I am actively using in my art history study. While imgs.ai can only show results from the databases it has access to, unlike some other digital art history projects, it does not limit itself to a specific time period or type of art. So I didn’t expect it to return other video game related results necessarily, but I did expect it to be able to get an accurate read on the image’s general contents since testing it with an AI alt text tool had produced flawed but not totally off-base results.
imgs.ai has four feature extractors to choose from for each search, each with a different focus: “stylistic similarity,” pose, “raw color data,” and CLIP which allows combining an image with a text prompt.6
Results were mostly okay if not super useful until I switched to the pose extractor.
Pair of Armchairs. Late 17th–early 18th century. Carved and gilded walnut, with crimson and gold silk cannelé fabric (not original), Italian. The Metropolitan Museum of Art. Public domain.Manufacture Royale, Aubusson (tapestry weaver). Armchair (part of a set of nine). Mid-18th century. Carved and painted walnut; Aubusson tapestry upholstery, French. The Metropolitan Museum of Art. Public domain.
Well that’s a choice. There weren’t only but a variety of objects that had no relation to the pose regardless of the database used. The chairs would take on an added strangeness with my next search though.
I decided to try another image. One I took directly from the Smithsonian collection.
The results were better but still not great. I only got results that were clearly connected to the search image but still novel enough to be useful when I added “videogame” to my CLIP search.
Screenshot of imgs.ai search results for the positive query “videogame” (image: Atari 2600 Video Chess box/screen) within the MoMA collection subset, showing 50 of 72,365 results. Author’s screenshot, 9 Sept. 2026.
All was well until I tried the pose feature extractor again.
Pair of Armchairs. Late 17th–early 18th century. Carved and gilded walnut, with crimson and gold silk cannelé fabric (not original), Italian. The Metropolitan Museum of Art. Public domain.Manufacture Royale, Aubusson (tapestry weaver). Armchair (part of a set of nine). Mid-18th century. Carved and painted walnut; Aubusson tapestry upholstery, French. The Metropolitan Museum of Art. Public domain.
Oh no.
Louis Delanois. Armchair (one of four) (part of a set). ca. 1770. Carved and gilded mahogany, modern silk damask, French, Paris. The Metropolitan Museum of Art. Public domain.Reading Chair. ca. 1720–30. Turned and joined walnut, leather, brass, tin, England. Cooper Hewitt, Smithsonian Design Museum. CC0, public domain.Side Chair. 1815–35. Mahogany, poplar, probably made in Philadelphia, Pennsylvania, American. The Metropolitan Museum of Art. Public domain.
Ohhhh nooooooooo.
I wracked my brain for the cause of this seating-based poltergeist haunting my results. Was it a similarity of a curve in the chair to the curve in the image? Then why were some of the EXACT SAME CHAIRS in the results from both starting images? Other more obvious connection images were not shared between results.
Away from the chairs, most results made sense as to why they were selected. They just didn’t provide anything useful beyond confirmation that yes, it has that color or a frame or someone in armor. Some images I found really intriguing, and they made me think of my image in a different way. It was most successful again when CLIP added texted to the search, in this case, “Skeleton.”
That was when I realized something important. While the point of imgs.ai is not to have an autonomous AI match images, but rather to include the human user in the research loop actively,7 I was doing the heavy lifting for the AI instead of with it. I was taking an image that seemed random to me and taking on the task of finding the connections and assigning meaning. In some ways, this is great. The human should be the one doing those tasks. But I was also struck that I could have done the same cognitive work and gotten the same benefit if the AI had served me completely random images from the collection and told me they were related to my image.
Two more images gave me additional insight into a major issue. Olivia Butler’s typewriter which appeared for both images…
The computer wasn’t “seeing” the artworks as objects but as the images themselves. So not only was the background included but it was given as much consideration as the art. The tool appears to search hierarchically level parts, not the whole.
Perhaps if I had access to the GRAD-CAM to show me where the AI was seeing similarity,8 I would have had a richer experience, but either I could not find that feature or it was behind the login gate. Ultimately, the need driving the creation of the tool seems valid. But until OpenAI understands (not just consumes) Emma Stanford’s “A Field Guide to Digital Surrogates”,10 imgs.ai will be like building a chair with a staple gun: some chance of surprise success, maybe worth the experience, but you can’t make an Eames with it.
Fabian Offert and Peter Bell, “IMGS.AI.: A Multimodal Search Engine for Digital Art History,”International Journal for Digital Art History, no. 9 (2023): 5.30, https://doi.org/10.11588/dahj.2023.9.91295.↩︎
Plastiboo, Vermis I : Lost Dungeons and Forbidden Woods : Official Guide, ed. Michele Nitri, with Marco Cirillo Pedri, Vermis 1 (2023; 11th printing, Hollow Press, 2026), Paperback (Regular Edition).↩︎
Emma Stanford, “A Field Guide to Digital Surrogates,” in The Routledge Companion to Digital Humanities and Art History, 1st ed., ed. Kathryn Brown (Routledge, 2020), https://doi.org/10.4324/9780429505188-19.↩︎
[Scanned image from Vermis I]. plastiboo, Vermis I: Lost Dungeons and Forbidden Woods, 11th printing, 2026. Used here for educational and commentary purposes; no copyright infringement intended. All rights remain with the original owner.
Scanning this image from Vermis I: Lost Dungeons and Forbidden Woods by plastiboo1 made me consider the layers of processing necessary to make a reproduction similar to the original. To get this image to come close to the original image in Vermis, I had to not only use a high-resolution scanner with the matching software, but to distort the original itself. I had to hold the book at an awkward angle, pressing it against the glass of the scanner enough to grab the image clearly without causing the spine to crack or the pages to be damaged (While this was with my personal copy of Vermis and I do not mind a cracked spine and other signs of wear, I wanted to practice as if it were not my own book). Using a paperback copy likely helped this process but I still would have had to take additional time if I had wanted the complete page, as parts of the full page were cut off or too shadowed. Even with the portion clearly scanned, I had to add a sepia photo filter to achieve the closest possible reproduction of the original in the time allotted as the scanner had dramatically brightened the image. I also had to add a Gaussian blur in Photoshop to smooth the honeycomb texture the scanner adds to books.
Emma Stanford in “A Field Guide to Digital Surrogates” explains the long lineage of reproductions, photographs, and other surrogates used in the study of art history.2 Perhaps more importantly, Stanford argues that these surrogates often go unnoticed. They are such a necessary part of our study and pedagogical process that we no longer think of them as imperfect substitutions. If we do notice, we often do not think of just how distant our surrogates remain from the original artwork.
How far removed is my scan from the original image? Removing it from the book takes it out of context. It is also removed from the surrounding page graphics and faux signs of use that were cropped out. While that doesn’t make it useless for study, its use is not the same as the original image.
I can zoom in on the scanned image.
Close up of the skeleton’s skull. Notably the honeycomb artifacting from the scanner is still visible in extreme close up. [Close up of scanned image from Vermis I]. plastiboo, Vermis I: Lost Dungeons and Forbidden Woods, 11th printing, 2026. Used here for educational and commentary purposes; no copyright infringement intended. All rights remain with the original owner.
I can view it directly beside a comparison image.
[Scanned image from Vermis I]. plastiboo, Vermis I: Lost Dungeons and Forbidden Woods, 11th printing, 2026. Used here for educational and commentary purposes; no copyright infringement intended. All rights remain with the original owner.
Screenshot of Wizardry: Proving Grounds of the Mad Overlord for the IBM PC (MS-DOS), level 1 of the maze. Orion Blastar, Wikimedia Commons, 2013. Licensed under CC BY-SA 3.0.
But what happens when the reproduction is all that remains of a work or the only accessible version?
I have mixed feelings on video game emulators. I acknowledge that they are invaluable for making video games accessible and have amazing archival potential. Despite legal challenges, emulators may be the only cost-effective, practical way to preserve some games. Still, what is lost when we consider a video game as software alone? Hardware is an important part of video-game art, whether the game is played on a console or a computer. Compatible hardware may no longer exist for the original physical game, making emulation a practical means of preserving access. Yet an emulator is still a surrogate, not a complete replacement for the original. Treating the code or software as the game itself can overlook the role of hardware in shaping the experience.
When an exhibit uses the same generic controller and display for all the games presented, what changes? Are the awkward shape and biting corners of an NES controller part of an NES game? I don’t believe there is a blanket answer to every exhibit, game, or situation. Without asking the question, we don’t explore other options.
How deep should the layers of surrogacy go? If original hardware could not be sourced, STL files for 3D models of consoles and controllers could probably be found. Is a printed hardware replica helpful surrogate or deception? Nintendo attracted considerable criticism after visitors discovered that its official museum relied on PC emulation.3 Nintendo has a long history of holding their copyright with an iron fist, pursuing legal action against mods and emulators. But when faced with presenting its own history, Nintendo also had to rely on the same surrogates used by game researchers to access their back catalog. Instead of facing this paradox, they attempted to hide it, using system controllers fed into a wall with the screen embedded above.
Even when the layers of surrogacy are not deliberately hidden, thinking of emulation in the same space as a slide projector in an art class calls into question what is included and what is excluded from the experience. The Internet Archive hosts playable MS-DOS games in it’s Software Library (Sorry not sorry for ruining any plans you had this evening if you didn’t know about this resource already).4 When you click play on a game, after a brief modern download screen for the emulator, a super-speed version of the DOS process of opening a DOS game flashes by. Are there circumstances where you should still have to type in the path in the blinking white text of the black DOS box while you frantically try to find the scrap of paper you wrote your save extension on? Should this process be part of the emulation? Should original load times or run times be emulated? Would doing so bring something valuable to the experience of video games as art? Would this surrogacy of nuance of experience be valuable or deceptive since modern technology does not require these processes?
Plastiboo, Vermis I : Lost Dungeons and Forbidden Woods : Official Guide, ed. Michele Nitri, with Marco Cirillo Pedri, Vermis 1 (2023; 11th printing, Hollow Press, 2026), Paperback (Regular Edition).↩︎
Emma Stanford, “A Field Guide to Digital Surrogates,” in The Routledge Companion to Digital Humanities and Art History, 1st ed., ed. Kathryn Brown (Routledge, 2020), https://doi.org/10.4324/9780429505188-19.↩︎
Paul B. Jaskot’s “Digital Art History as the Social History of Art: Towards the Disciplinary Relevance of Digital Methods” questions not only what research questions are best suited for examination using computational tools but what art history itself is. By placing digital art history in the heritage of social art history, Jaskot presents digital art history as capable of changing the art history field like the interventions of feminist and Marxist critiques that came before it. Digital art history is capable of this change by allowing the examination of understudied and noncanonical areas of the field.
The situating of digital art history within social history is not surprising given Jaskot’s background. Jaskot serves as the Co-Director of the Digital Art History & Visual Culture Research Lab at Duke University. His research centers on architecture and Nazi Germany. His projects primarily use GIS to question assumptions about construction and modernism in the era.
One of the more surprising aspects of Jaskot’s article for me was his expansive definition of social art history. Jaskot acknowledges the historical importance of Marxism and feminism and other -isms in reinvigorating art history towards previously unexamined artworks and ideas, but ultimately presents social art history as any approach that treats art and society as deeply interconnected. That relationship is the backbone and focus of social art history. As someone who does not generally consider themselves a social art historian since I rarely use specific identity or political lens, this definition made me reconsider my assessment of my work. My research and projects are often specifically focused on the relational aspect of art and society, mainly understudied areas of video game culture. Even outside that niche, I have an investigative bend in research (likely from growing up watching so much Law & Order as a kid that I played assistant district attorney the way other kids played house). Investigation is about what lives in the cracks of assumptions and following threads of connection to find new truths.
Jaskot also breaks down four of the most common methods used in digital art history: digital storytelling, text analysis, network analysis, and spatial analysis. Jaskot calls out network and spatial analysis as the most used and most fitted to art historical research (as of 2019 when the article was written). This assessment is because Jaskot focuses on large scale projects as the most fertile ground for any type of digital art history. Jaskot is correct that some illuminating projects have used digital art history tools to see areas that would be very difficult to examine without the distance viewing digital art history methods can provide. Network analysis and geographic mapping certainly can work well to preserve large scale complexity alongside a point of view and thesis.
I would be interested to know how Jaskot feels about more recent digital art history projects that use the same tools to look closer than traditional close looking. These projects present a microscopic version of close looking when compared to the space telescope version of distance viewing Jaskot focuses on in the article. One that came to my mind immediately is the presentation of historical research in video games like Pentiment.
Spoiler note: Pentiment – Examples come from interactions that can happen in the first hour or so of the game. I do not touch on the central plot.
Pentiment by Obsidian Entertainment released in 2022, but it’s status in academic research has only continued to grow in the years between then and now (2026). A point-and-click mystery set in Upper Bavaria in the 16th century, Pentiment is extraordinarily well researched. While series like Assassin’s Creed use extensive research in their creation, they often present this research primarily as world building or in objective sounding codex entries that the player can access in the menu. Pentiment embeds the research across the game at all levels. When picking where the player character, a journeyman manuscript illuminator named Andreas, previously studied, you are given the options of Italy, Basel, or Flanders. Each option provides unique dialogue and skills grounded in the history of those places, but also shows how people of different social classes in a small Alpine village conceptualized those places through specific interactions with Andreas.
The art style of Pentiment is also vital to how the game shares insights into perceptions of status, education, and technological progress. Different characters “speak” (via text bubble) in different lettering styles based on Andreas perception of them. The town printer’s speech appears in upside down blocks before reversing into readable script all at once with a clunk. Monks in the scriptorium speak in a flourished ink style that fills in religiously significant words like “Christ” in red after the rest of the text is written. This was a practice I had no idea existed until I search online to see why the script behaved that way.
Pentiment takes this digital narrative presentation beyond research and into a ludic thesis. There are two notable moments early in the game that showed how a ludic thesis could work that stood out to me.
The first moment is when Andreas discovers that a local shepherd is much more well-read then he thought. The shepherd’s speech rewrites itself from a messy illiterate script used for all the peasants to the clear measured text of the academics and educated. The lettering change subtly communicates how status and education worked in the background of all interactions at the time. The second moment is when retrieving books for the Sister who runs the abbey library. As she discusses the books with Andreas, they enter into the books themselves. Here, she has the freedom to explain the lack of choice for women in their lives. She focuses on how this lack of choice is visible in the treatment of the female characters in the books. Women are presented as inherently wicked or only present to suffer for a male character’s benefit regardless of the books being bawdy popular tales, classical texts, or religious allegory.
As the player experiences these moments as Andreas, there is a clear imperative to move beyond enjoyment of the historical setting to reflection on the perspectives about status, equality, progress, and justice that are raised. Despite not explicitly presenting as digital art history, this expansion of purpose makes Pentiment digital social art history. By using digital storytelling, the creators of Pentiment relinquish some control and clarity of their arguments in favor of allowing the player a much closer look than would be possible normally. The player is able to actually control an artist in the time period and choose how to confront the issues raised in game. Despite how the player decides to play Andreas, the reactions of the other characters and the art choices show the clear point of view and arguments being made. It will be interesting to see how new games approach digital storytelling as more works inspired by Pentiment are released. I also wonder if the game studies practice of presenting research in both traditional modes of scholarship and non-tradition modes like game design will expand to other fields).
