From a user perspective, virtual environments offer the most advanced interface to multimedia information systems. Virtual environments involve the use of (high resolution) 3D graphics, intuitive interaction facilities and possibly support for multiple users.

In this chapter, we will explore the use of (desktop) virtual environments as an interface to (multimedia) information systems. We will discuss a number of prototype implementations illustrating, respectively, how paintings can be related to their context, how navigation may be seen as a suitable answer to a query, and how we can define intelligent agents that can interact with the information space. Take good notice, the use of virtual environments as an interface to information systems represents a major challenge for future research!

...

Dam Square, Amsterdam

• it is a 'real world' environment
• it has 700 years of (recorded) history
• it has a fair amount of historical buildings
• buildings and street life change over time

how can we give access to the 'Dam square' information space

...

virtual context

• VR model of Dam square
• selection of related paintings fromRoyalMuseum
• viewpoint adjustment, to match painting
• (transparent) overlay of paintings over buildings

augmented virtual reality

• give user sense of geographic placement of buildings
• show how multiple objects in a museum relate to eachother
• show what paintings convey about their subject, and how

problems

• organised guided tours
• account for buildings that no longer exist
• change temporal context
• allow user queries as input

VRML

• declarative means for defining geometry and appearance
• prototype abstraction mechanism
• powerful event model
• relatively strong programmatic capabilities

...

virtual archeology

• variety of archeological sites
• various paths through individual site
• reconstruction of 'lost' elements
• 'discovery' of new material
• glossary -- general background knowledge

RIF

...

...

...

what are we searching for?

types of information

• viewpoints
• areas of interest
• objects
• persons
• text

availability

• static -- always
• shared -- users
• dynamic -- runtime
• temporal -- events
• hidden -- actions

scanning the scenegraph

• annotations
• node types
• textual content
• materials
• textures
• geometry

choose a metaphor

assumptions

• explicit annotation
• map for route planning
• keyword matching

requirements

• database -- annotations & map
• 3D (pseudo-immersive) interface

relaxing the assumptions

• annotation -- incremental and/or automatic
• (explicit) maps -- based on user navigation
• (keyword) matching -- text retrieval

...

future work

• shift in focus -- intelligent agents
• DLP + VRML -- distributed logic programming

Web Agent Support Program

www.cs.vu.nl/~eliens/research

3D GUI

...

WASP

DLP

RIF + WASP

• distributed logic programming -- uniform platform
• agent technology -- subsumes multi-user server

...

multi-user soccer game

• multiple (human) users -- may join during the game
• multiple agents -- to participate in the game (e.g. as goalkeeper)
• reactivity -- players (users and agents) have to react quickly
• cooperation/competition -- requires 'intelligent' communication
• dynamic behavior -- sufficiently complex 3D scenes, including the dynamic behavior of the ball

control points

• get/set -- position, rotation, viewpoint

agents in virtual environments

• virtual environments with embedded autonomous agents
• virtual environments supported by ACL communication

Living Worlds

• scene -- geometrically bounded, continuously navigable
• world -- collection of (linked) scenes

Shared Object

• pilot -- instance that will be replicated
• drone -- instance that replicates pilot

• pilot agents -- control state of a shared object
• drone agents -- replicate the state of a shared object

• object agents -- controls a single shared object (like the soccerball) pilot at server, drone at client
• controls users' avatar pilot at user side, drone at server or clients
• autonomous agents -- like football player, with own avatar pilot at server, drone at clients

programming platform

• VRML EAI support
• distributed communication capabilities (TCP/IP)
• multiple threads of control -- for multiple shared objects
• declarative language -- for agent support

...

taxonomy of agents

• 2D/3D -- to distinguish between text-based and avatar embodied agents
• client/server -- to indicate where agents reside
• single/multi -- as a measure of complexity

PAMELA

• autonomous and on-demand search capabilities
• (user and system) modifiablepreferences
• multimedia presentation facilities

H-Anim

• control points -- joints, limbs and facial features

presentation agent

persuasion games

• single avatar persuasive argumentation
• multiple avatar dialog games

PAMELA

...

A variety of applications may benefit from deploying embodied conversational agents, either in the form of animated humanoid avatars or, more simply, as a 'talking head'. An interesting example is provided by Signing Avatar, a system that allows for translating arbitrary text in both spoken language and sign language for the deaf, presented by animated humanoid avatars.

Here the use of animated avatars is essential to communicate with a particular group of users, using the sign language for the deaf.

STEP

• convenience -- for non-professional authors
• compositional semantics -- combining operations
• re-definability -- for high-level specification of actions
• parametrization -- for the adaptation of actions
• interaction -- with a (virtual) environment

DLP+X3D

evaluation criteria

• effective communication

conversational agents in VR

• presentational VR, instructional VR, educational VR

applications

• information agents, presentation agents

system perspective

• range of agent categories, open standards

user perspective

• naturalness -- contextual & emotional

initial target(s)

• build initial (throwaway) prototype
• explore content creation technology
• create tutorial(s) for content contribution
• analyse technological requirements

...

The history of Second Life is extensively descibed in the official Second Life guide,  [SecondLife]. Beginning 2004, almost out of the blue, Second Life appeared with a high adoption and low churn rate, now counting, March 2007, over 4 million inhabitants. Considering the cost of ownership of land, which easily amounts to 200 euro per month rent after an initial investment of 1500 euro for a single piece of land measuring 65,536 square meters, the adoption of Second Life by individuals as well as companies such as ABN-AMRO, Philips and institutions such as Harvard is surprising.

What is the secret of the success of Second Life?, we asked in  [VUSL], and we immediately confessed: We don't know! But in comparison to other platforms for immersive worlds, including MMORPGs such as World of Warcraft and Everquest, Second Life seems to offer an optimal combination of avatar modification options, gesture animations, in-game construction tools, and facilities for communication and social networking, such as chatting and instant messaging. Incorporating elements of community formation, commonly denoted as Web 2.0, and exemplified in MySpace, YouTube and Flickr, the immersive appearance, perhaps also the built-in physics and the inclusion of elementary economic principles, seem to be the prime distinguishing factors responsible for the success of Second Life. In addition, the possibility of recording collaborative enacted stories,  [Stories], using built-in machinima certainly also contributes to its appeal.

What has been characterized as a shift of culture, from a media consumer culture to a participatory culture,  [Participatory], where users also actively contribute content, (was) for our institution one of the decisive reasons to create a presence in Second Life, to build a virtual platform that may embody our so-called community of learners, where both staff and students cooperate in contributing content, content related to our sciences, that is.

The first idea that comes to mind, naturally, is to use Second Life to offer courses online. But, although we did have plans to give lectures (college) on law, probably including the enactment of a particular case, we did consider this approach as rather naive, and frankly I see no reason to include what may be considered an outdated paradigm of learning in our virtual campus, where there might be more appealing alternatives. Similarly, using the virtual laboratory for experiments might not be the best way to offer courses, although, again, we do intend to provide a model of a living cell, allowing students to study the structure, functionality and behavior of organic cells in virtual space.

active learning

• experience the world in new ways
• form new affiliations
• preparate for future learning

...

8. virtual environments

• (*) Discuss how virtual environments may be used for giving access to (multimedia) information. Give a brief characterization of virtual environments, and indicate how information (hyper) spaces may be projected in a virtual environment.

concepts

• What is meant by virtual context?
• Give an example of navigation by query, and indicate its possible advantages.
• Discuss the deployment of (intelligente) navigation agents.

technology

• Give a brief characterization of: VRML.
• What is a viewpoint transformation?
• What kinds of navigation can you think of?
• How may intelligent avatars be realized? Give an example.

As a project, I suggest the implementation of storytelling in virtual environments, with (possibly) an embodied agent as the narrator.

You may further explore or evaluate the role of agents in multimedia applications and virtual environments.

For further reading in (real) VR, I advice  [VR], and for gaining an understanding in story telling and applications you may try to get hold of the proceedins, of TIDSE 2003, and TIDSE 2004.

1. another series of dutch light.
2. virtual context -- Dam Square, Amsterdam, see 8.1.
3. VU Campus in VRML -- student project.
4. CWI 3th floor, floormap and model, see 8.2..
5. query -- on 3th floor of CWI.
6. navigation -- on 3th floor of CWI.
7. soccer game -- image from WASP project, see section 8.3.
8. digital beauties -- taken from  [Beauties].
9. digital beauties -- taken from  [Beauties].
10. VU @ Second Life -- screenshots.
11. signs -- sports,  [Signs], p. 276, 277.

9

post-modern design allows for sampling

digital convergence

  In my perception a portfolio is about the
  past and I feel much more related to the
  present especially in my work as a
  photographer. To me the relation with
  the present and the subject/assignment directs
  my creativity, the experiences over the 
  past draw my skills. My challenge is to be
  truly inspirred, to be present in the present.
  

  <form action="http://www.google.com/search?" method="GET">
  <input name="q" value="+site:www.cs.vu.nl/~eliens/media ">
  <input type="text" name="q" size=40>
  <input type="submit" value=" "> 
  </form>
  

  Information {
   informationNodes [
    GroupNode {
  	ID "MAIN"
  	shortName "Main"
  	longName "Main"
  	urlModel "models/conceptGraph/main/modelMain.wrl"
  	description [ "Central information node" ]
  	connectedNodesIDs [ "ARTWORKS", "KEYWORDS", 
  	                         "INTERVIEWS", "REPORT" ]
  	}
    GroupNode {
  	shortName "Artworks"
  	longName "Artworks"
  	description [ "Node that connects to all the artworks" ]
  	ID "ARTWORKS"
  	connectedNodesIDs [ "MAIN", "TRANSITORY", 
  	              "ULAY", "VIDEOINSTALLATION", "ABRAMOVIC" ]
  	urlModel "models/conceptGraph/artworks/artworksGroup.wrl"
  	}
  	## ...
   ]
  }
  

  Artwork {
    shortName "Terra degla Dea Madre"
    longName "Terra degla Dea Madre"
    description ["15:40 min, colour, sound."]
    ID "AV24"
    connectedNodesIDs ["VIDEOINSTALLATION", "DTV24", 
               "TTV24", "PV24", "CV24", "VV24", "G0"]
    urlPreviewImage "images/previewImages/AV24.jpg"
    widthPreviewImage 479
    heightPreviewImage 349
  }
  

  Video {
    ID "CV24"
    shortName "Interview clip Terra degla Dea Madre"
    longName "Interview clip showing Terra degla Dea Madre"
    url "interviewclips/interview_terra_degla.avi"
    width 320
    height 360
    urlPreviewImage "images/previewImages/interview_terra_degla.jpg"
    widthPreviewImage 320
    heightPreviewImage 240
    description [""]
    connectedNodesIDs ["CLIP", "AV24"]
  }
  

  TextItem {
    shortName "Instruction"
    longName "Green Dragon Lying instructions for the public."
    description ["Text explaining the way the public has to interact with the artwork."]
    ID "ITO05"
    connectedNodesIDs ["AO05", "INTERACTION"]
    url "text/AO05_instruction.txt"
  }