topical media & game development

talk show tell print

print / present / tag(s)


content annotation

video annotation requires a logical approach to story telling

learning objectives

After reading this chapter you should be able to explain the difference between content and meta information, to mention relevant content parameters for audio, to characterize the requirements for video libraries, to define an annotation logic for video, and to discuss feature extraction in samples of musical material.

Current technology does not allow us to extract information automatically from arbitrary media objects. In these cases, at least for the time being, we need to assist search by annotating content with what is commonly referred to as meta-information.

In this chapter, we will look at two more media types, in particular audio and video. Studying audio, we will learn how we may combine feature extraction and meta-information to define a data model that allows for search. Studying video, on the other hand, will indicate the complexity of devising a knowledge representation scheme that captures the content of video fragments.

Concluding this chapter, we will discuss an architecture for feature extraction for arbitrary media objects.






6. content annotation



projects & further reading

As a project, think of implementing musical similarity matching, or developing an application retrieving video fragments using a simple annotation logic.

You may further explore the construction of media repositories, and finding a balance between automatic indexing, content search and meta information.

For further reading I advice you to google recent research on video analysis, and the online material on search engines.

the artwork

  1. works from  [Weishar (1998)]
  2. faces -- from, an interesting site with many surprising interactive toys in flash, javascript and html.
  3. mouth -- Annika Karlson Rixon, entitled A slight Acquaintance, taken from a theme article about the body in art and science, the Volkskrant, 24/03/05.
  4. story -- page from the comic book version of City of Glass,  [Auster (2004)], drawn in an almost tradional style.
  5. story -- frame from  [Auster (2004)].
  6. story -- frame from  [Auster (2004)].
  7. story -- frame from  [Auster (2004)].
  8. white on white -- typographical joke.
  9. modern art -- city of light (1968-69), Mario Merz, taken from  [Hummelen and Sill\'e (1999)].
  10. modern art -- Marocco (1972), Krijn Griezen, taken from  [Hummelen and Sill\'e (1999)].
  11. modern art -- Indestructable Object (1958), Man Ray, Blue, Green, Red I (1964-65), Ellsworth Kelly, Great American Nude (1960), T. Wesselman, taken from  [Hummelen and Sill\'e (1999)].
  12. signs -- sports,  [ van Rooijen (2003)], p. 272, 273.
Opening this chapter are examples of design of the 20th century, posters to announce a public event like a theatre play, a world fair, or a festival. In comparison to the art works of the previous chapter, these designs are more strongly expressive and more simple and clear in their message. Yet, they also show a wide variety of styles and rethorics to attract the attention of the audience. Both the faces and the mouth are examples of using body parts in contemporary art. The page of the comic book version of City of Glass, illustrates how the 'logic' of a story can be visualised. As an exercise, try to annoyaye the sequence of frames from the City of Glass can be described using the annotation logic you learned in this chapter. The modern art examples should interesting by themselves.

(C) Æliens 04/09/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.