Friday, 11 January 2008

Image Annotation using common XML namespaces

This will outline methods for holding and describing the annotations of regions of images. I place emphasis on the work region as that is what I feel most people mean when they talk about adding metadata to an image; usually, they are more interested with the things in the photo, rather than the whole photo itself.

I'll give a few examples of what I mean:
A researcher wishes to indicate a particular architectural feature in some photographs of an archaeological site and indicate the same feature in a floorplan diagram of the same site.

A researcher cataloguing photos of insect life in the rainforest needs to be able to note the species of a particular insect in the image, who identified that particular insect and what plant/animal it is seen to be interacting with/eating/etc in this image - all of which pertain to an individual insect in the photo, although there may be one or more animals in the frame of the picture.

A foreign language book has been scanned in, and there needs to be a mechanism to relate each translated sentence and commentary to the relevant section of the image to which it applies.

A collection of ephemera from the 1900's (bus tickets, etc) have certain items which are compound items - a display case for a set of cigarette cards for example - and it is necessary to relate the location of the individual items to the original - locations on the display case continuing the example.
At first glance, it looks like we need something that can encode some metadata defining a 2-D shape and also encode the shape's relative positioning on the surface of an image. We will also need to be able to link arbitrary metadata to the individual shapes - arbitrary as we need to be flexible in terms of what information is recorded:

Region metadata --(links to)--> Region --(links to)--> Image

I knew I couldn't have been the first person to consider this, so I asked around. Benjamin Nowack (bengee) on the irc:// channel was very helpful and pointed me at the w3photo project which arrived at an ontology for defining regions of images:

Namespace for the ontology
Image Region Vocabulary: RDFS+OWL Documentation

[NB From what I've seen, most of the effort and work has gone into defining and producing metadata for the entire image, so I'll assume that a good image ontology/metadata format has been developed (I'm counting on the exif namespace myself, but a better one may appear.)]

So, to cut to the chase, this is a piece of RDF, hoping giving an example of what it is I am considering:

<rdf:RDF xmlns:foaf=""

<foaf:Image rdf:about=
<!-- insert misc image metadata here, concerning the entire image. -->
<imreg:coords>1002,920 1100,1035</imreg:coords>
<dc:description>Location of the 1st burial site.</dc:description>
<dc:creator>Ben O'Steen </dc:creator>
<dc:date>circa 200BC </dc:date>
<imreg:coords>1600,1200 1750,1300</imreg:coords>
<dc:description>Location of the 2nd burial site.</dc:description>
<dc:creator>Ben O'Steen </dc:creator>
<dc:date>circa 200BC </dc:date>

Combine that with a decent web interface for making these annotations, and a backend sophisticated to understand these user-made 'comments' on the image and you have yourself an exciting system, I feel. (I am using the Fedora RDF relationship of 'isAnnotationOf' to mark these connections between comment and original.)

(Oh, and for a good GPL'd DHTML method for drawing rectangles on an image, check out:
SpeedingRhino's Cropper tool, a working demo of which is here.

No comments: