Matthew J. Westerby opens his article “Annotating Upstream: Digital Scholars, Art History, and the Interoperable Image” with the story of how downstream uses of the British Library’s IIIF images and manifests lost their own access when the Library was the subject of a cyberattack. This domino effect showed a “change in the relationship between institutions and users of online collections in which separate layers exist to serve images and also to present the digital surrogate of a real-world object like a book.”1 Similar complex layers of delivery and downstream/upstream usage have been on the minds of video game scholars for a long time. There is always a layering between the game’s software and the hardware or the display, as well as between the game as code and the physical object that carries that code, such as a cartridge. Especially with online-all-the-time consoles, a game can be pulled or updated or changed at any time without user consent or warning. These delivery layers are perhaps more apparent to the average gamer than the average user of an IIIF image art surrogate because they have demonstrably affected how games are released and experienced. It is now common (though not acceptable) practice for a game to release in a broken state with day-one patches to download. Patching post-launch was a near impossibility before internet-connected gaming, so the uneven quality of games at release has been felt by all video gamers.
This is not the only place where Westerby’s suggestions and insights for art history run parallel to emerging practice in game studies. Annotating video games is one way game studies is looking to share knowledge weither via annotating games themselves, annotating screen captures of video games, or through examining existing annotation practices like wikis and let’s plays that are a part of the larger life of “networked hyperlinked videogames”.2 For the practice part of this post, I used Panopto to experiment with annotating a brief screen capture of me playing The Blood of Dawnwalker.3
Panopto was the assigned tool for class but I didn’t find it particially useful for the annotations I would like to make where it would be helpful to have them show up on specific parts of the screen during viewing.
One of the reasons wikis are such a robust source of information for video games is that they are such complex artworks that one person could never annotate a full game by themselves. It also adds to the robustness of scholarship when sharing is part of the process, not just the outcome. The Zelda object-focused digital humanities project4 that I mentioned in my post about Collection Builder was one project that used this style of sharing. My portion was about multiplicity and repetition of objects in the Zelda series, focusing on small keys, but I was doing it alongside Sydney Bauer whose object was a single American painting, Hannah Iovanna who was examining the still life paintings of a single artist, and Addie Magyar who worked with me on the Zelda portion and also acted as coordinator (Magyar was actually who noticed that small keys are the only constant in all the games). Being able to do those separate case studies with the same methodology while in communication during the process made the projects much richer and more enjoyable. Our discussions and brainstorming sessions brought up the question that I swear one day I will write about: why is there always a wheel of cheese with one little slice out in every video game? I sometimes wish I had picked the history of the wheel of cheese with one little slice out as my art history thesis: a study of why it is the most filling and delightful of foods across time and medium.
Annotating interoperable images is a mode of scholarly communication that is fundamentally different from the traditional sharing of only finished writing in journals and academic press books but no less a creative or scholarly act.5 When there’s a wider variety of acceptable ways to communicate information, writing a book or a journal article becomes a choice, not an obligation. The quality of all scholarly communication increases as forms increase because the output mode is not just one default option regardless of the needs of the argument or the data. This also allows for more modes of scholarship that cannot be studied or communicated as well via paper alone, like GIS mapping projects from my previous post. I was unfamiliar with the idea of thick data or data pollution6 as terms when I wrote the mapping post, but they are great terms for the concept I was exploring at the end of that post: messy data that balances the line between being so messy it’s no longer a useful tool but isn’t so clean that it carries unquestioned authority with data pollution also covering the source of data like AI or how data is used.
Westerby’s choice to end the article with how you can’t control other people’s usage of your images or metadata if you share them, and how the fear of misuse should be a known risk but not a reason to abandon openness was nuanced and powerful.7 There will always be people who take good things and warp them. Making decisions based around absolute prevention of misuse is an impossibility and an unnecessary burden. It is better to work from being risk-aware, with practices like the usage tags that Westerby suggests.8 It reminded me specifically of the Wretched & Alone SRD,9 a system written by Matt Sanders for making solo journaling games in the style of Chris Bisette’s game The Wretched.10 The Limitations section of the SRD includes “Finally, nazis, homophobes, transphobes, and other bigots are expressly forbidden from making new content using the Wretched and Alone system. If you have to ask whether this includes you, it does.”11 Are parasites going to ignore the usage terms? Likely. Still, by making clear the usage and specifically taking a stance on who is forbidden to use the material, bigots cannot hide behind not knowing or accidental infringement. It makes the deliberate nature of the theft apparent.
Matthew J. Westerby, “Annotating Upstream: Digital Scholars, Art History, and the Interoperable Image,”Open Library of Humanities 10, no. 2 (2024), https://doi.org/10.16995/olh.17217.↩︎
Austin Anderson, “Dark Souls as Networked Hyperlinked Videogame,”Electronic Book Review, ahead of print, September 28, 2025, https://doi.org/10.64773/1f7a-df6c.↩︎
“The Blood of Dawnwalker – A Dark Fantasy Narrative Sandbox RPG,” The Blood of Dawnwalker – A Dark Fantasy Narrative Sandbox RPG, accessed October 5, 2026, https://dawnwalkergame.com/us/en/home.↩︎
Sanders, “Wretched & Alone System Reference Document V1.1.”↩︎
1 Comment
Caroline Fenstermacher5 Oct 2026
Your perspective as an art historian of video games brings an especially interesting dimension to an analysis of Westerby’s article. I was not able to see your video yet, but I understand why digital video and image annotations are a significant method of scholarship-sharing in your field of interest! I especially appreciate your observation that with the rise of digital methods for sharing scholarship, it liberates the scholar to choose from any number or combination of scholarship sharing like methods that we have covered so far in class (spatial analysis, digitization, creating digital repositories, etc.). This week, Westerby introduced the idea that metadata itself can be a form of public knowledge-sharing.
I also noted Westerby’s point about the benefits of decentralized metadata to mitigate data loss in the event of a change in technology or cyber attack. Westerby’s concept of annotating “upstream” also allows for several different perspectives and a more encompassing context to be gained by viewers of this additional metadata. I understand, however, why some scholars may be hesitant to add this additional data.
Once information is made available to the public to be used for any number of projects, it is inevitable to avoid unintended manipulation of data. Your example with The Wretched & Alone SRD system is helpful to illustrate this concern, and I think that the creator at least partially solved this issue through a flat denunciation of use by bigots. Besides use for less than noble or scholarly purposes, I think the concept of intellectual property is also relevant. Because image annotations like this are typically more of a byproduct of research rather than an intentional result, it may subject the creators to intellectual theft more than a polished digital humanities project created for public use.
Practically, the best-suited scholars to annotate certain images may lack the necessary skills to add their own information to IIIF. This is a valid obstacle, but I think it highlights a positive characteristic about digital art history: the necessity of collaboration between art historians and across disciplines. Creating Westerby’s ideal world of accessible “thick data” would require collaboration between the field of art history, library and data sciences, and computer engineering.
Annotating “upstream” and allowing the scholar to participate in the metadata-creation side of digital images and videos allows the scholar to share their own knowledge from a digital object’s foundation. This ability involves the scholar in the very categorization and public understanding of a digital object.
The inspiration for my Cryptid Art map was growing up hearing tales of the Mothman and the Braxton County Monster (also called the Flatwoods Monster or Braxxie to her friends). The Mothman has risen in fame in the last decade, and yet most people I talk to who know the Mothman is from Point Pleasant, West Virginia, only know that as a general factoid or from the video game Fallout 76. Most have no idea the Mothman has any connection to the 1967 Silver Bridge collapse, which is a shame.
It may seem strange to connect a massive infrastructural disaster that ended in the deaths of 46 people and the creation of the National Bridge Inspection Standards to the celebration of a cryptid. But the lore of both is deeply intertwined. The Mothman was seen perched on the Silver Bridge the day before it collapsed, and so has taken on the status of an omen or, more charitably, the idea that he was “trying to warn us.” The tragedy was also caused by systemic infrastructure failure, a reason not to trust what the government tells you, a common theme in West Virginia stories. The danger of trusting the faceless people in power is echoed in the phrase that I heard most during the Mothman Festival hayrides and have a pin repeating: “Don’t fear the Mothman. Fear the Men in Black.” The Mothman behaved like an animal, and so could hurt someone, but never with malice. The Men in Black who came after from the government were the ones who threatened people. Admittedly, it’s a paranoid viewpoint. But given West Virginia’s history, from the Mine Wars to vote buying, and most recently the opioid crisis, I don’t think it’s a misplaced view.
I used Google My Maps1 for the Cryptid Art map, but I also explored Knight Lab’s StoryMapJS.2 I chose My Maps because it focuses it’s display on place, while StoryMapJS prioritizes content. Many of the artworks and related sites have very little information available, so a small description and image work well. For pins with more information available, it is nice to be able to focus on the most unusual or helpful source, like using the Braxxie lantern documentary for the Exxon station where they are sold. StoryMapJS is better for focusing on the object first and the location second, particularly in how it pulls up a large secondary window for the embedded material for each pin. When I was thinking of projects that might work with it, I was thinking of comparatively looking at falconry gloves within different archives and museum collections: where they were found, where they were made, where they were used, and where the material was sourced. This is also reflected in how StoryMapJS uses fixed icons that are tied to the type of material embedded and also the ability to control the order that pins are navigated through.
I think there is some baked-in skepticism of the accuracy and completeness of maps that maybe isn’t there for museum exhibitions, and that skepticism is valuable to engage with. Because we’ve probably all had a GPS app try to take us down a closed road or otherwise lead us astray, it’s all the more illuminating to notice how much maps can be used to both enforce and question authority. Béatrice Joyeux-Prunel in her chapter for The Routledge Companion to Digital Humanities and Art History, uses the example of how nationalists (including the Nazis) use “artistic geovisualization” to claim spaces, heritages, and artworks as their own and to justify their crimes.3 Joyeux-Prunel continues by explaining how art history is returning to cartography via digital mapping because it allows new questions to emerge “that had been buried for a long time, either because they could be embarrassing (such as the place of origin of a style, too often understood as a ‘center of influence’), or the geopolitics of art, or because they could not be given satisfactory answers[.]”4
An excellent example of case studies that ask all three of those types of questions and investigate them with “Historical Geographic Information Systems (GIS) methods” is Paul B. Jaskot and Anne Kelly Knowles’ “An Approach to Institutional History and the Built Environment in Nazi Germany.”5 In my opinion, the case studies have the added benefit of dejustifying Nazi crimes by picking them apart and not allowing them to hide in historical assumption. Mapping the SS camp system and the architecture of the SS camp at Auschwitz-Birkenau, Knowles and Jaskot used different sources of information and visualization approaches. What they had in common was how GIS was able to shed new insight even in a field as highly studied as the SS camps and raise new questions because it was able to connect information to a spatial context.6
Building my own map, I was surprised by how difficult many of the decisions I needed to make were and how they immediately made me realize I was creating from a position of authority whether I wanted that or not. One of my earliest decisions was whether to pin the Silver Bridge Wall mural. The Mothman and Silver Bridge stories are connected, but they are not the same. Still, the Mothman stripped of the Silver Bridge is stripped of an important part of its history in Point Pleasant. So I included the mural, with a simple description focused on the purpose of the map: “The Mothman was seen on top of the Silver Bridge the day before it collapsed, leading some to believe he is an omen of doom or was trying to warn the town of the approaching tragedy.”
I plan to continue to work on my Cryptid Art map with the mission that there should be art of cryptids in their hometowns (as long as those communities want it). So in the future I plan to add cryptid origin points even where there isn’t art yet. I may also add local artists who make cryptid art. Lately, instead of worrying whether something fits a set of rules or parameters, I’ve been asking whether it’s kinder to include it than to leave it off. Being messier but kinder in action is perhaps how we keep tools with an inherent authority, like cartography, from having so much power to warp or control. The future of transparency in digital tools may be to strip back the veneer of clean data and UIs without losing the purpose of the tool, embracing messiness and a degree of friction so that questions naturally arise about where the information comes from and who decided its presentation.
Béatrice Joyeux-Prunel, “Digital Humanities for a Spatial, Global, and Social History of Art,” in The Routledge Companion to Digital Humanities and Art History, ed. Kathryn Brown, Routledge Art History and Visual Studies Companions (Taylor & Francis Ltd, 2022), 88–89.↩︎
Joyeux-Prunel, “Digital Humanities for a Spatial, Global, and Social History of Art,” 89.↩︎
Working on the Artist Book Critical Index (ABCI), I’ve already gotten to work with data I really enjoy: linked data, the semantic web, distributed knowledge graphs. I enjoy ontology and vocabulary creation and being part of building semantic frameworks that other organizations can then pick up and use.
Angela Dressen’s “Cultural Heritage Data for Research – An Introduction” covers semantic web work in the art history space with clear appreciation and understanding, but in the Linked Art section, it also presents the idea that with proper tools, you then don’t need data scientists in the museum space.1 The idea that if the structure’s in place and it’s easy enough to use, museums don’t need data professionals in permanent positions is part of why digital humanities projects fall short or become obsolete. When data projects are approached as short-term, quick-burst projects, they die out quickly. Not because the data isn’t good anymore or the tools are bad, but because there is no one cranking the engine to improve the data and move the project forward. If there’s one thing I’ve learned working on the ABCI, it’s that there has to be human energy for all aspects of the project, including the data handling and project maintenance.
The ABCI started as a Zotero library made by the Sloane Art Library’s now-retired manager, Josh Hockensmith, who is a book artist. He was making an index of reference works connected to artists’ books: where they were cited and on what page an image could be found. Graduate students were then assigned to the project and started expanding beyond UNC’s own collection by adding WorldCat links. It is a really unique, robust data set that could only come from human beings.
One of the key points Broman and Woo make in “Data Organization in Spreadsheets” is that the raw data should live in a different place than where it is analyzed and manipulated.2 This was one of the untidiest aspects of the original Zotero ABCI library. The workflow for photographing artist books was living in the same Zotero, and it was difficult to untangle verified entries from unverified ones, even when they were marked with workflow tags.
We also had to deal with missing data, the equivalent of what Broman and Woo talk about with leaving cells empty.3 The early ABCI entries had full metadata. But as the point became simply tracking citations between books and linking to the catalog entry at UNC or WorldCat.org, entries started having just author and title, with no standardization in capitalization, author name order, or which creators should be listed. And because the citation information had to live in Zotero’s notes field, which is a free-for-all of text, we had to figure out some constraints, along with using Claude Code, to pull the citation data out and attach it to the items it belonged to.
One of my favorite parts of the process has been creating ontologies and vocabularies. It’s modeling how the data should look in triple format, in our case Turtle, with an example, because often when you start doing an example is when you start saying, “Oh no, that data would be great to have, but we don’t have that,” or, “What do we call a construction method?” Luckily, I was able to piggyback off the work of the Book Arts Research Database4 and Opening Artists’ Books5 for reference.
Sometimes a tool becomes unexpectedly good at another job, in this case, metadata checking. One unexpected side effect of the ABCI has been pointing out where our catalog metadata was misentered or left out. More than once, the Sloane Art Librarian, Alice Whiteside, has asked me to use the tool to pull books with a particular construction method, only to find that a book is missing or mislabeled. If you are working with incorrect data to start with, you’re going to end up with incorrect data.
As for data I would like to work with in the near future, I’ve been fascinated lately by maps of crafters and craft spaces. Especially having read more of Brendan Keogh’s recent work about viewing video game developers as craftspeople, I’d like to make a network map of video game creators and the companies or specific games they’ve connected to. Art history already acknowledges that who an artist is taught by and where they have worked has a large influence on their oeuvre. This type of lineage study is not something I’ve encountered yet in game studies.
My first step for a network map of game artists/craftspeople would be to model one simple connection. I generally do not start in a spreadsheet but think with pen and paper about how the graph might flow and then start throwing it into Turtle triples. I would start with a relatively well-established indie or smaller-team game from a company where English is a primary language. From there, I would pick one creator in an art leadership role, like an art director or character designer, and work through their CV. Once I know what information I’m looking for, I’d break out the spreadsheet to gather everything about one person, translate it over, see how it works, and troubleshoot. Then I’d look to other sources of linked data I could link out to, like Wikidata. From there, whether or how to proceed to a larger scale would really depend on what the data revealed in the small model.
Dressen, A., (2025) “Cultural Heritage Data for Research – An Introduction”, Open Library of Humanities 11(1). doi: https://doi.org/10.16995/olh.18710↩︎
Broman, Karl W., and Kara H. Woo. 2018. “Data Organization in Spreadsheets.” The American Statistician 72 (1): 2–10. doi:10.1080/00031305.2017.1375989.↩︎
Broman, Karl W., and Kara H. Woo. 2018. “Data Organization in Spreadsheets.” The American Statistician 72 (1): 2–10. doi:10.1080/00031305.2017.1375989.↩︎
For a recent class assignment to make a Collection Builder exhibit, I created an exhibition of the first 10 boss fights from the mainline Legend of Zelda video games, specifically the final blow in each boss fight (https://beeyop.github.io/zelda-death-blow/). What weapon has to be used? Who’s involved in the fight? What part of the timeline, via the Hyrule Historia, does it come from? This was an expansion of a previous project where I looked at repeat objects in the Legend of Zelda series (https://sydbauer.wixsite.com/scaling-stories/zelda ).
I immediately ran into some issues. I couldn’t get the YouTube links to show properly if I wanted to start them at a specific time. I also struggled for a while trying to figure out why a thumbnail wasn’t showing, only to discover that it was an errant capitalization. Frustrations aside, Collection Builder did a really nice job of creating a professional-looking digital exhibition space. For that, I think it fulfills a valuable niche.
I only had one major criticism of the Collection Builder exhibits we examined in class: all the websites felt the same. By the second collection I looked at, that sameness made me gloss over and lose interest in the content. The visual layout was pretty much identical, even with different themes and some custom CSS. I think this is indicative of a larger issue. I believe many cultural institutions view tasks like metadata curation as mechanical, logical chores. In reality, it is a fusion of artistic skills and data skills. If the visuals and experience don’t feel curated to the data, then it is a cute toy, not scholarship or a tool. Some of the exhibits we examined did have a cohesive feel and told a clear story, but I don’t know that they did more than a well-annotated photo gallery.
I think Collection Builder is a gift for smaller museums, organizations, and student projects as a tool for displaying work when you’ve got one, maybe two people working and there isn’t room to expand. In some environments, finding someone with the education and experience in both curatorial work and digital tools would be very hard. Hiring online is an option, but then you run into control issues and management duties. In a single-person or very small team space, hiring out isn’t saving any time or workload the way I think Collection Builder does. I think the well-written and well-organized guide does most of the work of making it a better alternative for that use case. Anytime I wanted to know how to do something with my collection, the guide had an easy-to-understand, easy-to-find, simple process. I have some background in using GitHub and VS Code, but I heard from other people in the class that they also appreciated how well thought out the guide is.
What concerns me is that every time I see a tool like Collection Builder that makes what seems to be a mechanical task, like cataloging, display, or access, easier, I see organizations pushing people to do more with less instead of hiring someone to do these things or paying people appropriately for an increased scope of their job. Budget issues often become the scapegoat, but that is no excuse. Most individuals’ budgets are equally, if not more, stressed than those of any organization they work for. That’s not Collection Builder’s fault by any means, but it is something to note, especially as we decide when to use what tools and what projects to pursue.
In Paige Morgan’s blog post “How to Get Your Digital Humanities Project Off the Ground,”1 Morgan heavily cautions against making a digital humanities project part of your dissertation. The current status of digital humanities projects in academia is reflected in Morgan’s own experience creating projects while working on a separate dissertation. Morgan notes there were times when the dissertation had to take precedence and the projects suffered. That shouldn’t have to be the case, but from what I have experienced and witnessed, not much has changed between when Morgan wrote the blog post in 2014 and today.
While the cautioning is absolutely valid, and a realistic way of looking at the field and the way academia treats digital humanities projects, I think this advice exacerbates the problem. It’s going to take some people who are willing to push back and say, “No, this is a valid form of scholarship,” to really upend the idea that the only valid way to present mastery over information or share scholarship is the often recursive ouroboros of academic paper and book publishing.
With the rapid pace at which people are expected to graduate, to write, to do, we’ve lost a lot of the time to reflect and a lot of the purpose of academic scholarship. Academia must slow the speed and return to a contemplative pace or start accommodating other modes of scholarship as equally valuable and equally valid (preferably both).
This week I explored the digital art history tool imgs.ai and got some interesting results. imgs.ai provides “a Web-based interface, and machine-learning-based backend that allows the end user to both descriptively and visually search arbitrary image dataset.”1 More simply put, it allows a human to guide an application using computer vision to find images in selected museum databases that are the most similar to an uploaded image, text prompt, or both. The idea being that the user will then continue to engage with more images by selecting from the suggested images that are “positive” or “negative,” furthering the search with this new information.2
The ability to mark negative results is where I hit my first issue: it wasn’t an option for me. This is likely because I was using the site without a login (getting one takes days for approval). While I understand the necessity of gatekeeping databases in the age of AI scraping and how intentionally using AI is fundamentally different from random bots shutting down your site, the irony of the website being protected being built on AI scraping images was palpable. How many of the models imgs.ai uses were built using the same tactics the required login was keeping out? Even if copyright was the driving force instead of defense against bots, how many artworks outside of these museum collections had their copyright violated to train the models to “see”? I am not anti-AI, but I think they are questions worth asking. A team that calls out “Transparency” as one of five “essential criteria”3 should say more about the black box components at work beyond naming the databases or models used.4 Minimally, it should give an explanation of why options are locked behind sign-up instead of forcing the user to guess.
I also wouldn’t be as harsh in my critique if imgs.ai as a tool had provided me an experience that felt more like what the article outlined. To show where imgs.ai gave me interesting results and where it fell short, let me walk you through my experimental searches.
I started with the image from Vermis I: Lost Dungeons and Forbidden Woods I digitized in my last post.5
[Scanned image from Vermis I]. plastiboo, Vermis I: Lost Dungeons and Forbidden Woods, 11th printing, 2026. Used here for educational and commentary purposes; no copyright infringement intended. All rights remain with the original owner.
I knew this wouldn’t necessarily be an easy image with its dithered graphics, but I thought it was fair to work with an image I am actively using in my art history study. While imgs.ai can only show results from the databases it has access to, unlike some other digital art history projects, it does not limit itself to a specific time period or type of art. So I didn’t expect it to return other video game related results necessarily, but I did expect it to be able to get an accurate read on the image’s general contents since testing it with an AI alt text tool had produced flawed but not totally off-base results.
imgs.ai has four feature extractors to choose from for each search, each with a different focus: “stylistic similarity,” pose, “raw color data,” and CLIP which allows combining an image with a text prompt.6
Results were mostly okay if not super useful until I switched to the pose extractor.
Pair of Armchairs. Late 17th–early 18th century. Carved and gilded walnut, with crimson and gold silk cannelé fabric (not original), Italian. The Metropolitan Museum of Art. Public domain.Manufacture Royale, Aubusson (tapestry weaver). Armchair (part of a set of nine). Mid-18th century. Carved and painted walnut; Aubusson tapestry upholstery, French. The Metropolitan Museum of Art. Public domain.
Well that’s a choice. There weren’t only but a variety of objects that had no relation to the pose regardless of the database used. The chairs would take on an added strangeness with my next search though.
I decided to try another image. One I took directly from the Smithsonian collection.
The results were better but still not great. I only got results that were clearly connected to the search image but still novel enough to be useful when I added “videogame” to my CLIP search.
Screenshot of imgs.ai search results for the positive query “videogame” (image: Atari 2600 Video Chess box/screen) within the MoMA collection subset, showing 50 of 72,365 results. Author’s screenshot, 9 Sept. 2026.
All was well until I tried the pose feature extractor again.
Pair of Armchairs. Late 17th–early 18th century. Carved and gilded walnut, with crimson and gold silk cannelé fabric (not original), Italian. The Metropolitan Museum of Art. Public domain.Manufacture Royale, Aubusson (tapestry weaver). Armchair (part of a set of nine). Mid-18th century. Carved and painted walnut; Aubusson tapestry upholstery, French. The Metropolitan Museum of Art. Public domain.
Oh no.
Louis Delanois. Armchair (one of four) (part of a set). ca. 1770. Carved and gilded mahogany, modern silk damask, French, Paris. The Metropolitan Museum of Art. Public domain.Reading Chair. ca. 1720–30. Turned and joined walnut, leather, brass, tin, England. Cooper Hewitt, Smithsonian Design Museum. CC0, public domain.Side Chair. 1815–35. Mahogany, poplar, probably made in Philadelphia, Pennsylvania, American. The Metropolitan Museum of Art. Public domain.
Ohhhh nooooooooo.
I wracked my brain for the cause of this seating-based poltergeist haunting my results. Was it a similarity of a curve in the chair to the curve in the image? Then why were some of the EXACT SAME CHAIRS in the results from both starting images? Other more obvious connection images were not shared between results.
Away from the chairs, most results made sense as to why they were selected. They just didn’t provide anything useful beyond confirmation that yes, it has that color or a frame or someone in armor. Some images I found really intriguing, and they made me think of my image in a different way. It was most successful again when CLIP added texted to the search, in this case, “Skeleton.”
That was when I realized something important. While the point of imgs.ai is not to have an autonomous AI match images, but rather to include the human user in the research loop actively,7 I was doing the heavy lifting for the AI instead of with it. I was taking an image that seemed random to me and taking on the task of finding the connections and assigning meaning. In some ways, this is great. The human should be the one doing those tasks. But I was also struck that I could have done the same cognitive work and gotten the same benefit if the AI had served me completely random images from the collection and told me they were related to my image.
Two more images gave me additional insight into a major issue. Olivia Butler’s typewriter which appeared for both images…
The computer wasn’t “seeing” the artworks as objects but as the images themselves. So not only was the background included but it was given as much consideration as the art. The tool appears to search hierarchically level parts, not the whole.
Perhaps if I had access to the GRAD-CAM to show me where the AI was seeing similarity,8 I would have had a richer experience, but either I could not find that feature or it was behind the login gate. Ultimately, the need driving the creation of the tool seems valid. But until OpenAI understands (not just consumes) Emma Stanford’s “A Field Guide to Digital Surrogates”,10 imgs.ai will be like building a chair with a staple gun: some chance of surprise success, maybe worth the experience, but you can’t make an Eames with it.
Fabian Offert and Peter Bell, “IMGS.AI.: A Multimodal Search Engine for Digital Art History,”International Journal for Digital Art History, no. 9 (2023): 5.30, https://doi.org/10.11588/dahj.2023.9.91295.↩︎
Plastiboo, Vermis I : Lost Dungeons and Forbidden Woods : Official Guide, ed. Michele Nitri, with Marco Cirillo Pedri, Vermis 1 (2023; 11th printing, Hollow Press, 2026), Paperback (Regular Edition).↩︎
Emma Stanford, “A Field Guide to Digital Surrogates,” in The Routledge Companion to Digital Humanities and Art History, 1st ed., ed. Kathryn Brown (Routledge, 2020), https://doi.org/10.4324/9780429505188-19.↩︎
2 Comments
admin15 Sep 2026
I can't remember who but there is a comedian that has a bit about how the purpose of life is to sit down. So maybe the AI wants to sit?
Miranda Hynes13 Sep 2026
Mert– this was hilarious to read. I'm not sure if you've just discovered a glitch in this AI model, or if there's something in the computer vision itself that is, for some reason, obsessed with chairs. The only thing that I can think of is when you use "pose" for anything more than one human figure, that somehow a chair is the only non-human object which can accommodate that pose. Honestly, these seems like more of a "composition" category, which I would expect a team of art historians to know. You make a good point about how ultimately, this asks humans to do a lot of the heavy lifting. I think that even if these models continue to advance, this will remain true: somewhere, a human will need to do labor in order for these machines to work.
Still, I love the idea of a computer that's just obssessed with chairs. so funny.
[Scanned image from Vermis I]. plastiboo, Vermis I: Lost Dungeons and Forbidden Woods, 11th printing, 2026. Used here for educational and commentary purposes; no copyright infringement intended. All rights remain with the original owner.
Scanning this image from Vermis I: Lost Dungeons and Forbidden Woods by plastiboo1 made me consider the layers of processing necessary to make a reproduction similar to the original. To get this image to come close to the original image in Vermis, I had to not only use a high-resolution scanner with the matching software, but to distort the original itself. I had to hold the book at an awkward angle, pressing it against the glass of the scanner enough to grab the image clearly without causing the spine to crack or the pages to be damaged (While this was with my personal copy of Vermis and I do not mind a cracked spine and other signs of wear, I wanted to practice as if it were not my own book). Using a paperback copy likely helped this process but I still would have had to take additional time if I had wanted the complete page, as parts of the full page were cut off or too shadowed. Even with the portion clearly scanned, I had to add a sepia photo filter to achieve the closest possible reproduction of the original in the time allotted as the scanner had dramatically brightened the image. I also had to add a Gaussian blur in Photoshop to smooth the honeycomb texture the scanner adds to books.
Emma Stanford in “A Field Guide to Digital Surrogates” explains the long lineage of reproductions, photographs, and other surrogates used in the study of art history.2 Perhaps more importantly, Stanford argues that these surrogates often go unnoticed. They are such a necessary part of our study and pedagogical process that we no longer think of them as imperfect substitutions. If we do notice, we often do not think of just how distant our surrogates remain from the original artwork.
How far removed is my scan from the original image? Removing it from the book takes it out of context. It is also removed from the surrounding page graphics and faux signs of use that were cropped out. While that doesn’t make it useless for study, its use is not the same as the original image.
I can zoom in on the scanned image.
Close up of the skeleton’s skull. Notably the honeycomb artifacting from the scanner is still visible in extreme close up. [Close up of scanned image from Vermis I]. plastiboo, Vermis I: Lost Dungeons and Forbidden Woods, 11th printing, 2026. Used here for educational and commentary purposes; no copyright infringement intended. All rights remain with the original owner.
I can view it directly beside a comparison image.
[Scanned image from Vermis I]. plastiboo, Vermis I: Lost Dungeons and Forbidden Woods, 11th printing, 2026. Used here for educational and commentary purposes; no copyright infringement intended. All rights remain with the original owner.
Screenshot of Wizardry: Proving Grounds of the Mad Overlord for the IBM PC (MS-DOS), level 1 of the maze. Orion Blastar, Wikimedia Commons, 2013. Licensed under CC BY-SA 3.0.
But what happens when the reproduction is all that remains of a work or the only accessible version?
I have mixed feelings on video game emulators. I acknowledge that they are invaluable for making video games accessible and have amazing archival potential. Despite legal challenges, emulators may be the only cost-effective, practical way to preserve some games. Still, what is lost when we consider a video game as software alone? Hardware is an important part of video-game art, whether the game is played on a console or a computer. Compatible hardware may no longer exist for the original physical game, making emulation a practical means of preserving access. Yet an emulator is still a surrogate, not a complete replacement for the original. Treating the code or software as the game itself can overlook the role of hardware in shaping the experience.
When an exhibit uses the same generic controller and display for all the games presented, what changes? Are the awkward shape and biting corners of an NES controller part of an NES game? I don’t believe there is a blanket answer to every exhibit, game, or situation. Without asking the question, we don’t explore other options.
How deep should the layers of surrogacy go? If original hardware could not be sourced, STL files for 3D models of consoles and controllers could probably be found. Is a printed hardware replica helpful surrogate or deception? Nintendo attracted considerable criticism after visitors discovered that its official museum relied on PC emulation.3 Nintendo has a long history of holding their copyright with an iron fist, pursuing legal action against mods and emulators. But when faced with presenting its own history, Nintendo also had to rely on the same surrogates used by game researchers to access their back catalog. Instead of facing this paradox, they attempted to hide it, using system controllers fed into a wall with the screen embedded above.
Even when the layers of surrogacy are not deliberately hidden, thinking of emulation in the same space as a slide projector in an art class calls into question what is included and what is excluded from the experience. The Internet Archive hosts playable MS-DOS games in it’s Software Library (Sorry not sorry for ruining any plans you had this evening if you didn’t know about this resource already).4 When you click play on a game, after a brief modern download screen for the emulator, a super-speed version of the DOS process of opening a DOS game flashes by. Are there circumstances where you should still have to type in the path in the blinking white text of the black DOS box while you frantically try to find the scrap of paper you wrote your save extension on? Should this process be part of the emulation? Should original load times or run times be emulated? Would doing so bring something valuable to the experience of video games as art? Would this surrogacy of nuance of experience be valuable or deceptive since modern technology does not require these processes?
Plastiboo, Vermis I : Lost Dungeons and Forbidden Woods : Official Guide, ed. Michele Nitri, with Marco Cirillo Pedri, Vermis 1 (2023; 11th printing, Hollow Press, 2026), Paperback (Regular Edition).↩︎
Emma Stanford, “A Field Guide to Digital Surrogates,” in The Routledge Companion to Digital Humanities and Art History, 1st ed., ed. Kathryn Brown (Routledge, 2020), https://doi.org/10.4324/9780429505188-19.↩︎
Paul B. Jaskot’s “Digital Art History as the Social History of Art: Towards the Disciplinary Relevance of Digital Methods” questions not only what research questions are best suited for examination using computational tools but what art history itself is. By placing digital art history in the heritage of social art history, Jaskot presents digital art history as capable of changing the art history field like the interventions of feminist and Marxist critiques that came before it. Digital art history is capable of this change by allowing the examination of understudied and noncanonical areas of the field.
The situating of digital art history within social history is not surprising given Jaskot’s background. Jaskot serves as the Co-Director of the Digital Art History & Visual Culture Research Lab at Duke University. His research centers on architecture and Nazi Germany. His projects primarily use GIS to question assumptions about construction and modernism in the era.
One of the more surprising aspects of Jaskot’s article for me was his expansive definition of social art history. Jaskot acknowledges the historical importance of Marxism and feminism and other -isms in reinvigorating art history towards previously unexamined artworks and ideas, but ultimately presents social art history as any approach that treats art and society as deeply interconnected. That relationship is the backbone and focus of social art history. As someone who does not generally consider themselves a social art historian since I rarely use specific identity or political lens, this definition made me reconsider my assessment of my work. My research and projects are often specifically focused on the relational aspect of art and society, mainly understudied areas of video game culture. Even outside that niche, I have an investigative bend in research (likely from growing up watching so much Law & Order as a kid that I played assistant district attorney the way other kids played house). Investigation is about what lives in the cracks of assumptions and following threads of connection to find new truths.
Jaskot also breaks down four of the most common methods used in digital art history: digital storytelling, text analysis, network analysis, and spatial analysis. Jaskot calls out network and spatial analysis as the most used and most fitted to art historical research (as of 2019 when the article was written). This assessment is because Jaskot focuses on large scale projects as the most fertile ground for any type of digital art history. Jaskot is correct that some illuminating projects have used digital art history tools to see areas that would be very difficult to examine without the distance viewing digital art history methods can provide. Network analysis and geographic mapping certainly can work well to preserve large scale complexity alongside a point of view and thesis.
I would be interested to know how Jaskot feels about more recent digital art history projects that use the same tools to look closer than traditional close looking. These projects present a microscopic version of close looking when compared to the space telescope version of distance viewing Jaskot focuses on in the article. One that came to my mind immediately is the presentation of historical research in video games like Pentiment.
Spoiler note: Pentiment – Examples come from interactions that can happen in the first hour or so of the game. I do not touch on the central plot.
Pentiment by Obsidian Entertainment released in 2022, but it’s status in academic research has only continued to grow in the years between then and now (2026). A point-and-click mystery set in Upper Bavaria in the 16th century, Pentiment is extraordinarily well researched. While series like Assassin’s Creed use extensive research in their creation, they often present this research primarily as world building or in objective sounding codex entries that the player can access in the menu. Pentiment embeds the research across the game at all levels. When picking where the player character, a journeyman manuscript illuminator named Andreas, previously studied, you are given the options of Italy, Basel, or Flanders. Each option provides unique dialogue and skills grounded in the history of those places, but also shows how people of different social classes in a small Alpine village conceptualized those places through specific interactions with Andreas.
The art style of Pentiment is also vital to how the game shares insights into perceptions of status, education, and technological progress. Different characters “speak” (via text bubble) in different lettering styles based on Andreas perception of them. The town printer’s speech appears in upside down blocks before reversing into readable script all at once with a clunk. Monks in the scriptorium speak in a flourished ink style that fills in religiously significant words like “Christ” in red after the rest of the text is written. This was a practice I had no idea existed until I search online to see why the script behaved that way.
Pentiment takes this digital narrative presentation beyond research and into a ludic thesis. There are two notable moments early in the game that showed how a ludic thesis could work that stood out to me.
The first moment is when Andreas discovers that a local shepherd is much more well-read then he thought. The shepherd’s speech rewrites itself from a messy illiterate script used for all the peasants to the clear measured text of the academics and educated. The lettering change subtly communicates how status and education worked in the background of all interactions at the time. The second moment is when retrieving books for the Sister who runs the abbey library. As she discusses the books with Andreas, they enter into the books themselves. Here, she has the freedom to explain the lack of choice for women in their lives. She focuses on how this lack of choice is visible in the treatment of the female characters in the books. Women are presented as inherently wicked or only present to suffer for a male character’s benefit regardless of the books being bawdy popular tales, classical texts, or religious allegory.
As the player experiences these moments as Andreas, there is a clear imperative to move beyond enjoyment of the historical setting to reflection on the perspectives about status, equality, progress, and justice that are raised. Despite not explicitly presenting as digital art history, this expansion of purpose makes Pentiment digital social art history. By using digital storytelling, the creators of Pentiment relinquish some control and clarity of their arguments in favor of allowing the player a much closer look than would be possible normally. The player is able to actually control an artist in the time period and choose how to confront the issues raised in game. Despite how the player decides to play Andreas, the reactions of the other characters and the art choices show the clear point of view and arguments being made. It will be interesting to see how new games approach digital storytelling as more works inspired by Pentiment are released. I also wonder if the game studies practice of presenting research in both traditional modes of scholarship and non-tradition modes like game design will expand to other fields).
1 Comment
Kathryn Spears25 Aug 2026
This is a really interesting analysis of Jaskot's article through the lens of "Pentiment". I, unsurprisingly, am unfamiliar with the game and was surprised by how thorough it seems to be in its art historical research. More importantly--as you noted--I am incredibly impressed by how extensive the world building in "Pentiment" seems to be based on your descriptions. I think it's a cool idea to carry that historical aspect over to the gameplay itself, fully immersing players in sixteenth-century European life. I used to watch one of my old roommates play "Assassin's Creed: Odyssey" just so I could see the world building through their character walking around, but I had zero interest in the actual plot of the game itself. Is it a stunning game that I could watch all day? Yes. Would expanding all of the research that went into its world building to the gameplay make me actually consider playing it? Also yes.
At the end of the day, I think Paul B. Jaskot would probably be open to games like "Pentiment" as tools of digital art history because it further expands the narrative beyond just the visual mechanics of the game. If the goal is to get people to interact with art history, games like "Pentiment" do it in an incredibly effective way. It's like learning math through the real CoolMathGames (not Papa's Pizzeria)--you're learning but you aren't necessarily aware of that fact.
