IIIF AI/ML Community Group

About

Though the IIIF API specifications were not designed for machine learning applications, they have nevertheless proven valuable in recent years as applications like computer vision, classifier training, and automated tagging have become increasingly popular in the cultural heritage space. IIIF manifests and collections lend themselves to the kind of image and canvas aggregation that make it easy to feed machine learning tools, while the flexibility of the IIIF image and annotation mechanisms (based on the Web Annotation data model) provide a compelling means of connecting images and regions therein with tags, attributes, and qualities like those often derived from machine learning processes.

Likewise, the IIIF Content State API, designed to “provide a way of describing a IIIF Presentation API resource, or a part of a resource, in a compact format that can be used to initialize the view of that resource in any client that implements this specification,” may prove to be a useful means of describing the outputs of some ML process, e.g., regions of interest for tagged elements.

The utility of IIIF in these contexts is clear, but the best practices in this domain are less so, and awareness of the full extent of IIIF capabilities is uneven. We believe there is value in bringing together practitioners in this domain to gather and highlight use cases, align output formats, and promote interoperability more generally.

Goals

  • Highlight and potentially publish best practices or guidelines about how IIIF can make AI/ML projects easier or more efficient
  • Provide feedback to IIIF community and spec editors on any use cases that are not yet suitably covered by existing specifications
  • Identify any gaps in tooling or knowledge-sharing that prevent existing AI/ML-enabled tools from using IIIF and/or prevent existing IIIF-based tools (e.g., annotation apps) from being integrated into AI/ML projects
  • Consider the possibilities and implications of representing descriptive metadata gathered through AI/ML in IIIF objects
  • Promote reusability and interoperability of IIIF-related datasets and outputs
  • Raise awareness of IIIF among the machine learning community, as both a source of data and a destination for storing and publicizing the outputs of ML analyses
  • Evaluate and recommend best practices for using IIIF and annotations as an interoperable means of conveying relevant data elements in the context of image-, audio- and moving-image collections, and showcase the benefits of IIIF in these cases

Chairs

  • Emma Stanford, Stanford
  • Peter Broadwell, Stanford
  • Peter Leonard, Stanford
  • Giles Bergel, University of Oxford

Communication Channels

Call Notes and Group Documents

IIIF AI/ML Community Group Folder

Call Connection Information