topical media & game development

talk show tell print


multimedia casus

You can learn a great deal about technology, but there is no meaning to that unless the technology is applied to produce something worthwhile. In this final chapter, the outline of a multimedia casus will be presented, that is a course in which students face the challenge of creating a veritable (intelligent) multimedia information system.

In the studyguide, the course is described as follows.

multimedia casus

The assignment in the multimedia casus is to develop a virtual environment for some cultural or governemental institute or company.

The practicum takes the form a stage, in which external supervision plays an important role.

In the multimedia casus, techniques learned in previous courses (see the afterthoughts) will be applied to create the application.

At the start of the course the actual assignment will be determined.

Examples of possible assignments are: the development of a virtual exposition hall for the Dutch Royal Museum of the Arts, a virtual city square, which gives information about both the present and the past, a virtual shop, with online buying facilities, or an online broker, which offers facilities for inspecting houses.

In effect, the availability of a representative of a cultural institute, industry, or governmental department is crucial, otherwise the assignment might easily degrade to the type of toy assignments so common in academia. Now, what is the challenge in such an assignment?

augmented information

In the research directions of section 8.1 the notion of augmented virtuality was introduced to clarify the duality between information and presentation. More in particular, it was argued that the use of VR makes no sense unless there is some added value, that is by using the rich presentation and interaction facilities that come with this technology.

In an abstract fashion, we may rephrase the assignment as follows:

Given an information space, create a VR that resolves the duality between information and presentation, using intelligent multimedia technology. The VR must offer access to all relevant information entities, organized in a suitable spatial layout, and must allow for presentations from a variety of perspectives, making full use of graphical and rich media facilities.

Below, we will see how this may work out for a concrete assignment.


front page of the INCCA website


project assignment -- present a complex information space

Art is an interesting and complex phenomenon. No art, no culture! Hence, the preservation of collections of artworks is of crucial importance. The ICN (Netherlands Institute for Cultural Heritage) is a government-funded institute for the preservation of (dutch) cultural heritage. ICN gives advice, organises courses, does research, etcetera.

ICN is actively involved in the preservation of modern art, being project leader for INCCA (International Network for the Conservation of Contemporary Art), in the person of Tatja Scholte.


In 1999, a group of eleven international modern art museums and related institutions applied to the European Commission (Raphael Programme) under the umbrella International Network for the Conservation of Contemporary Art (INCCA). The INCCA project was accepted and work started in January 2000 led by the organiser, the ICN (Netherlands Institute for Cultural Heritage) and the co-organiser, Tate, London.

The objectives of INCCA are phrased as follows.


INCCA's most important set of objectives, which are closely interlinked, focuses on the building of a website with underlying databases that will facilitate the exchange of professional knowledge and information. Furthermore, INCCA partners are involved in a collective effort to gather information directly from artists.

The INCCA web site contains a wealth of information about contemporary artists, as well as links to virtual collections of the works of a variety of artists, as for example Mondriaan. The way the virtual Mondriaan collection is presented is interesting in itself. It is a running display with iconic representations of his paintings. The speed of the display varies with the user's mouse movement, and at any time the user may select a painting to obtain more information about it. This particular site suggests where our intelligent multimedia approach may fit in.

Returning to the INCCA project once more, as its mission statement we read:


INCCA's guiding mission is to collect, share and preserve knowledge needed for the conservation of modern and contemporary art.

By now, the outlines of our assignment should become clear. Our information space is information about modern and contemporary artists, in the form of digital representations of there work, photographs, audio recordings from interviews and written text. The project assignment is to organize (part of) this material in a virtual environment and to include interaction facilities that highlight particular aspects of this information.

At this stage it would be too ambitious to cover all the material in the INCCA database, so we should restrict ourselves to one or more smaller case studies. The challenge, obviously, is to create presentations with a solid narrative structure and to augment the presented material in a suitable manner, using intelligent multimedia technology. What is suitable, is part of the challenge!

project management -- roles

Can the challenge, stated above, be met? Well, there are many ways the project may loose its focus, or fail alltogether. Students should be aware of the fact that the challenge is real and that failure would bring about shame.

Since there are no golden rules for project management, the students themselves are responsible for keeping the project on track. In other words, project management is part of the experience. Here is a checklist.


  • roles -- create a team
  • project goal -- develop a vision
  • production -- construct the assets
  • quality assesment -- test and control
  • delivery -- present and archive
  • manage -- all along
  • document -- track project's history
The rule of the supervisor should be minimal, as a critical third party. The students work as a group, and they should take responsibility as a group, including the management of the project, assigning roles, and keeping track of progress. In such an approach intervision (students supervise one another) is a necessary mechanism in judging the final result of the project.


  • group -- (2) effort, 5 (product), 3 (documentation)
  • individual -- (4) responsibility, (3) productivity, (3) quality
On a scale of 0-10, both the group result and the individual efforts may be assigned a mark with proper weights, as indicated above. In addition, target deliverables should be defined to assure that the project meets its deadlines and to inspect the nature and quality of the students' work.


  • group -- project plan, design, project report, product
  • individual -- detailed weekly account of activities
Dependent on the time available a schedule should be defined indicating when the deliverables should be ... delivered.


  1. project organisation
  2. project definition
  3. planning and design
  4. construction and development
  5. integration and delivery
  6. presentation and archiving
Is this a realistic setup? It should be. Besides, it is not the supervisor's responsibility, is it? It is first of all the responsibility of the students themselves!

peer reviews -- to control group dynamics

Whether you are a student or responsible for supervising projects, this account of how our multimedia casus is organized should give you some indications of what you may encounter in a team project. But apart from the organizational issues, you should be aware of the group dynamics, that is the individual relations and clusters of persons that emerge during the process of development. In general it takes some time before the various roles are established, that is who takes leadership, who takes notes during meetings, and who does most of the technical work. Also, sometimes some of the more creative members of the tem are overshadowed by some of the more outspoken ones. Some people simply need to learn to assert themselves!

For a supervisor, it is often quite difficult to assess the contribution of each individual. I remedy this by having both informal peer reviews, in the group, as well as formal peer reviews, where marks must be given for responsibility, productivity and quality of the work. The order in which the students to be reviewed are presented to the reviewer is random, to avoid any bias due to presentation order.

I started using peer reviews about ten years ago, then informally, because I noticed that students could be much more direct in their criticisms that I dared to be. Recently, following suggestions from my collegue Johan Hoorn, who is an experimental psychologist, we formalized the procedure and introduced peer reviews for other courses as well, including the visual design course.


physical control cubehead-mounted display


example(s) -- tangible virtual museum

Don't touch that! Keep your hands off! This is what you often hear parents shouting at their children in a museum or gallery exhibit. More often, however, precious artifacts, ceramics, porcelain or bronzes, are stored away in glass showcases, precluding any kind of physical interaction, and many times a proper look as well.

To remedy this situation, researchers from the Academia Sinica and the National Cheng Kung University have developed a tangible photorealistic virtual museum, a system for real-time interaction with photorealistic museum artifacts, which allows for an immersive experience using tangible interfaces, in the form of a 3D control cube (image left),  [Tangible]. The display is a kiosk-like system showing a panoramic view of the exhibit, augmented with a collection of perspective photographs for each object. The user may examine any of the objects by using a handheld control cube (PCC) to control size and rotation of the object.

As indicated, the system is not 3D mesh-based but image-based, which allows for high resolutions on mid-range platforms, which would not be feasible according to the authors, when using 3D modelling techniques.

research directions -- metaphors and interaction style

Given a problem statement as the one above, to present information about contemporary artists, how would you proceed? You might start by asking potential users, or stakeholders, how they would like the system to be. The answer you will get this way is likely to be disappointing. They will probably tell you that it must be like something they already know. So it might be better to rely on your own intuition and find a creative solution by choosing a fitting metaphor.

Let me give an example. In creating the digital dossier, a notion that will be explained in the next section, for the artist Marinus Boezem, as presented in the research directions of section 9.2, we choose the artist's atelier as a metaphor, and we used the spatial layout of the atelier as an organizational principle for presenting the information. In this, indeed very naturalistic, approach, we used pedestals to present the artworks, a file cabinet to present the textual information and a video projector to present the video recorded interview with the artist. The extent to which the virtual atelier does represent the artist's atelier faithfully is not important, in this context. What is important is whether the spatial metaphor did function as a valid organizational principle for presenting the information.

Instead of arguing whether this is the case or not, or whether the graphics chosen were right, etcetera, I would rather like to refer you to the literature, so that you can investigate the issues involved yourself.

In  [Preece], it is observed that interface metaphors act as conceptual models to support particular tasks. For office tasks, for example, we have the wellknown desktop metaphor.  [Preece] lists a number of such metaphors, for a variety of application domains:

application areametaphorfamiliar knowledge
operating environmentdesktopoffice tasks
spreadsheetsledger sheetcolumnar table
object-oriented environmentphysical worldreal world
hypertextnotecardsorganization of text
learning environmenttraveltours, guides, movement
file storagepilescategorizing
multimedia environmentsroomsspatial structures
cooperative workmulti-agentstravel agents, servants
In the most right column it is indicated why the metaphors should work, assuming real world situations that we are familiar with.

In some cases it is necessary to speak of a composite metaphor. For example, scrollbars are not easily to be found on your natural desktop. Form a cognitive perspective then, we may speak of multiple mental models.

When we look at what interaction styles are supported from a more technical perspective, we have following  [Preece], the following options:

interaction styles

  • command entry
  • menus and navigation
  • forms fills and spreadsheets
  • natural language dialog
  • direct manipulation
However, each of these interaction styles may somehow be incorporated in the representation that we adopt for our metaphor.

2D vs 3D

Surprisingly, each year that I start with another multimedia casus group, there is a discussion whether the application should be in 2D, using traditional web technology or flash, or 3D, using VRML or any other suitable 3D technology. My answer to the students objections, which can partly be explained by the fact that they fear the complexity of 3D, is flatly that anything that can be done in 2D can be done in 3D. But looking at the list of interaction styles above, I am tempted to add that a 3D representation allows for a more rich repertoire of interaction styles, such as spatial navigation. It would be interesting to investigate to what extent the interaction styles used in game playing can be incorporated in 'more serious' applications.

(C) Æliens 04/09/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.