learning objectives

After reading this chapter you should be able to describe scenarios for information retrieval, to explain how content analysis for images can be done, to characterize similarity metrics, to define the notions of recall and precision, and to give an example of frequence tables, as used in text search.

Searching for information on the web is cumbersome. Given our experiences today, we may not even want to think about searching for multimedia information on the (multimedia) web.

Nevertheless, in this chapter we will briefly sketch one of the possible scenarios indicating the need for multimedia search. In fact, once we have the ability to search for multimedia information, many scenarios could be thought of.

As a start, we will look at two media types, images and documents. We will study search for images, because it teaches us important lessons about content analysis of media objects and what we may consider as being similar. Perhaps surprisingly, we will study text documents because, due to our familiarity with this media type, text documents allow us to determine what we may understand by effective search.

...

1

...

10

questions

5. information retrieval

• (*) What is meant by the complementarity of authoring and retrieval? Sketch a possible scenario of (multimedia) information retrieval and indicate how this may be implemented. Discuss the issues that arise in accessing multimedia information and how content annotation may be deployed.

concepts

• How would you approach content-based description of images?
• What is the difference between a metric approach and the transformational approach to establishing similarity between images?
• What problems may occur when searching in text or document databases?

technology

• Give a definition of: shape descriptor and property descriptor. Give an example of each.
• How would you define edit distance?
• Characterize the notions precision and recall.
• Give an example (with explanation) of a frequency table.

As a project, you may implement simple image analysis algorithms that, for example, extract a color histogram, or detect the presence of a horizon-like edge.

You may further explore scenarios for information retrieval in the cultural heritage domain. and compare this with other applications of multimedia information retrieval, for example monitoring in hospitals.

For further reading I suggest to make yourself familiar with common techniques in information retrieval as described in  [Baeza-Yates and Ribeiro-Neto (1999)], and perhaps devote some time to studying image analisis,  [Gonzales and Wintz (1987)].

1. artworks -- ..., Miro, Dali, photographed from Kunstsammlung Nordrhein-Westfalen, see artwork 2.
2. left Miro from  [Kunst], right: Karel Appel
3. match of the day (1) -- Geert Mul
4. match of the day (2) -- Geert Mul
5. match of the day (3) -- Geert Mul
6. mario ware -- taken from gammo/veronica.
7. baten kaitos -- eternal ways and the lost ocean, taken from gammo/veronica.
8. idem.
9. PANORAMA -- screenshots from field test.
10. signs -- people,  [ van Rooijen (2003)], p. 252, 253.

The examples of match of the day by Geert Mul serve to illustrate the interplay between technology and art, and may also start you to think about what similarity is. Some illustrations from games are added to show the difference in styles.

---

[] readme course(s) preface I 1 2 II 3 4 III 5 6 7 IV 8 9 10 V 11 12 afterthought(s) appendix reference(s) example(s) resource(s) _

---

(C) Æliens 04/09/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.