topical media & game development

talk show tell print

immersion is not illusion

We live in a media-rich culture, full of images and audio-visual stimuli. Somehow, we never seem to get enough of that, and after a day of work, sitting in front of a computer screen, we still explore for more exciting clips. The need for such audio-visual stimuli is not surprising, when you consider the high visual complexity of our daily (urban) life. The ability to process (visual) data quickly, seems to be a vital condition for our survival in the information society, where reality becomes an ephemeral phenomenon, as it is increasingly represented by images.

Images have a special status in our (western) culture. As Roland Barthes said, cited from  [Semiotics]:

analogon of reality

Certainly, the image is not the reality but at least it is its perfect analogon and it is exactly this analogical perfection which, to our common sense, photography. This can be seen as the special status of the photographic image, it is a message without a code.

In the light of our observation(s) on re-mediation, in section 2.3, it is no surprise that early computer graphics took photo-realism as a benchmark. In an extremely well readable discussion (with examples) on video game aesthetics, David Hayward observes that depending on your point of view, photo-realism is either a scourge or a grail. The main tendency seems to be towards ultra-realistic cinematographic visuals. However, fortunately, there are many examples where an aesthetically more viable approach is taken, either by introducing cartoon-style elements, or by abstractions suprpassing photorealism.

As member of the creative industry, that is multimedia or game developer, you will contribute to our visual culture, and hence it is worthwhile to reflect on what underlies the construction of images and (visual) narrative(s): such as perspective, modality and composition.

Perspective is fundamental in our understanding of images. Interestingly, it is a relatively new discovery,  [Alberti], dominating our visual culture from the days of the enlightment until now, despite artistic revolts of the 20th century. As we observed in section 9.4, the notion of perspective describes both


  • the organisation of the image, as well as
  • the (optimal) point of view of the viewer.
The intricate relation between viewer and image, dependent on perspective, implies that when looked at from the 'wrong point of view', there will be a distortion of the image. The 'normal' perspective, as we know it, is the 'central' perspective. Variants of perspective, such as the anamorphism depicted below, force the viewer into an abnormal point of view,  [Anamorphisms]. Better viewed online, watch it from the right, with you head close to the paper/screen.



In a multi-dimesnional space often a change of perspective, that is another point of view, brings to light many new aspects, not previously seen, or, in some cases, the correction of a reducing or distorting projection. Instruction in drawing perspective, by constructing a horizon with so-called dis-appearance points, used to be a standard element of art-school curricula, until recently.

Another interesting aspectof images is modality. The notion of modality may be implicitly characterized by using a quote from  [Semiotics], where they discuss the realism of documentary film:


... documentary modality of black and white realism ...

In a similar way, cartoons may be said to represent a particular modality, although not so much as a kind of realism, but rather as commenting on a situation, in a comic, or perhaps sarcastic or ironic way.



More in general, (visual) modality may considered to be part of a visual grammar, that, one way or another, defines the meaning of the image(s) of our culture: According to  [Semiotics]:

visual grammar

grammar goes beyond formal rules of correctness. It is a means of representing patterns of experience. It enables human beings to build a mental picture of reality, to make sense of their experience of what goes on around them and inside them.

In section 9.3, we have discussed features of PANORAMA that were inspired by compositional elements of the visual grammar proposed in  [Semiotics]. There we observed that, however helpful these notions were, some cultural relativism is necessary, because the meaning associated with visual elements and their composition does not adhere to universal truth(s), but is rather the result of social processes involved in the construction of meaning.

Interestingly, panorama displays, from which we took inspiration for our system supporting social awareness (PANORAMA), were wide-spread popular art-form of the 19th century. In a discussion of the origins of virtual reality,  [VirtualArt] observes

virtual reality

the idea of virtual reality only appears to be without a history: in fact, it rests firmly on historic art traditions, which belongs to a discontinuous movement of seeking illusionary image spaces.

Linear perspective, as well as (perspective-related) atmospheric painting techniques (think of fog in CG terms), contributed significantly to the success of the immersive image spaces of the panaorama displays of the 19th century. Often the panorama displays were not realistic in a social or political sense, though. For example, the Mesdag Panorama depicted a rustic image of the Netherlands, that in that time already had been destroyed by industrialization and urbanisation.

More in general, we may ask: were these panaroma displays art?  [VirtualArt] unambiguously states:


the concept of immersion when implemented as an artwork surrenders most of the essential properties of an artwork.

Although a discussion of what constitutes the essential properties of artworks is far beyond the scope of this manuscript, it is interesting to look at what properties an artwork may have, which according to  [VirtualArt] encompass:

properties of artwork(s)

  • form -- aesthetic whole
  • structure -- organisation of elements
  • function -- context of display
  • processuality -- (implicit) narrative structure
  • statement -- existential/political implication(s)
Both multimedia applications and games, one way or another, share these functions with art. Where art is probably most disctinct is in the existential implications, that is the way it makes a statement about our condition of being. Nevertheless, in a somewhat different way, als games, whether serious or not, do have implications with respect to our existence, and may be taken as a comment on our existince or an appeal to change our attitude towards it. From a more educational perspective, we may observe, following  [VirtualArt], that

collective memory

it is an apparent feature of the concept of immersion that it engages with the spatial and pictorial concentration of the awareness of one's own people, the formation of collective identity through powerful images that occupy the function of memory.

In other words, the media have a strong impact on what we consider to be part of our domain of living. However, evidently in our current visual culture, and as anyone famaliar with for example the VJ-culture can testify to, representation of meaning is not the primary target. As eloquently expressed in  [VirtualArt]:

ecstatic transport

using contemporary image techniques, immersive art very often visualizes elements that can best be described as Dionysian: ecstatic transport and exhilaration.



From a more epistemological perspective, reflecting on how we gain knowledge of the world around us, we may ask (ourselves) what role the media play in defining our reality. And as a consequence of the observations above, what is the meaning of this ecstatic transport, induced by the variety of audio-visual effects enabled by our new technologies?

With respect to the notion of realism  [Semiotics] observe:


a realism is produced by a particular group as an effect of the complex of practices which define and constitute that group.

Basically that means that realism is a neytral term, that is anything may be considered to be realistic, as long as it is supported by a group or community. This as a remakr, should make us careful in denouncing particular styles as naive, as for example has been done with african art. Extrapolating this relativistic notion of realism to the commonly used notion of naturalism,  [Semiotics] state:


each realism has its naturalism, that is a realism that is a definition of what counts as real, a set of criteria for the real, and it will find its expression in the right, the best, the most natural form of representing that kind of reality, be it a photograph or a diagram.

Indeed, for a scientist a diagram may be more natural than a colorful depiction of atmospheric pollution!



Let's return to the observation with which we started this section:

dominant paradigm(s)

the dominant standard by which we judge visual realism (and hence visual modality) remains for the moment, naturalism as conventionally understood, photorealism.

To make things slightly more complex, we may add that the visual modality is the dominant modality of our culture, which means that we pay more attention to what we see than what we hear. According to feminist criticism, this may be de-constructed as expressing male dominance, which puts higher value on the ability of sight, than of touch and the perception of sound. Whether this is tur, and if so why, I leave to your own research. Apparently, this might have a cause in our need(s) for survival, where hunting was (once) more important than anything else. Think about how our need to be partially aware of everything, as discussed in section 9.4, might be related to that.

To conclude this section, let's briefly consider what is involved in creating an immersive application such as PANORAMA, or slightly more general, what is involved in what Sidney Fels in a UIU04 keynote called designing intimate experience(s). A key observation here is that people form relationships with objects external to their own self.


  • response -- object disembodied from self
  • control -- self embodies object
  • reflection -- self disembodied from object
  • belonging -- object embodies self
These relationships, or notions of belonging, are essential in developing an aesthetics of interaction, a topic that will approach in the next chapter.



example(s) -- Monet's Nympheas

One of the recommendations I make to everyone who is planning to go to Paris is to visit Monet's Waterlilies which are exhibited in a small gallery near the Louvre. Monet, at that time at the at of his career, living in Giverney, was well aware of the paradoxical nature of his Herculean effort. As stated in  [VirtualArt]:

mass medium

thus, one year after Monet's death and fifty years after his Impression soleil levant, a late example of modern art reached the changed artistic landscape of the 1920's, transported in a derivative of the mass medium for images in the 19th century.

Being one of the innovators of painting in the early twentieth century, Monet nevertheless endeavored in painting panorama-wide paintings, with an aestehics of illusionistic immersiveness, however with a painterly touch so refined that this series of works easily trancends the ordinary illusionism of its predecessors. Monet himself might have seen it as an exercise to improve his painterly craft to the limits of perfection.

research directions -- information art

Given the rise of the creative industry it is not suprising that initiatives have been taken to provide academic curricula in creative technology. The Institute of Creative Technology (LA), which developed the Mission Rehearsal Exercise discussed in chapter 9, may be taken as an example of an institute combining both technological and creative expertise in its staff.

The new media or creative technology play an important role in our society, in that they contribute in envisaging our dreams and (through advertisement) selling our reality.  [InformationArts], which provides an in any sense exhaustive investigation in the relation between technology and art, observes:

new media

a consequence of the constitutive function of artistic-illusionary utopias for the inception of new media of illusion is that the media are both a part of the history of culture and of technology.

Although the research agenda's of scientists may easily conflict with the artistic agenda's of artists,  [InformationArts], describes numerous projects where projects where artists and scientists cooperate, with mutual benefit.  [InformationArts] is together with  [DeepTime],  [VirtualArt] and many others, part of series of books from MIT Press, dealing with the cross section between technology, art and culture We repeat the quote, already given in section 1.1, that accompanies this series of books:

cultural convergence

the cultural convergence of art, science, and technology provides ample opportunity for artists to challenge the very notion of how art is produced and to call into question its subject matter and its meaning in society.

Examples of projects with both relevance science and art, may for example be found in the area of tele-presence, to thematize, following  [VirtualArt]:


  • notions of artificial life
  • fusion with (infinite) virtual image worlds
  • transformation of self into digital data
Such themes, apart from all technical problems involved, deeply affect human aspirations, as expressed in our myths and movies:

human aspiration(s)

telepresence also combines the contents of three archetypal areas of human aspirations: automation, virtual illusion and metaphysical views of the self.

In particular, such notions may be used to analyse, or de-construct, our behavior(s) on the web and our adoption of, for example, Second Life:


what is being preached is the phantasm of union in a global net community, cybergnosis, salvation through technology, disembodied as a post-biological scattering of data that lives forever.

This, eveidently may lead to criticism(s), which after all is a function of art, to make us aware of our limits, and the intrinsic qualities of our existence:


what we observe are hyperzealots of a new technoreligion running wild, zapping, excerpting and floating in cyberspace.

However, although we may encounter similar criticisms in popular culture, that is cartoons and games, it is interesting to reflect on the difference between art and the of new media applications that are somehow related to art. Again following  [VirtualArt]:


since the eighteenth century, aesthetic theories have regarded distance as a constitutive element of reflection, self-discovery and the experience of art and nature.

Aesthetic distance, however, is a notion that is also subject to criticism from new developments within art itself, for example performance art, which aspire a more direct existantial impact, as for example the works of Marina Abramovic, discussed in chapter ,a href=1.html>1.

What is meant with creative technology or new media, and what will constitute the tools of the new culture is not entirely clear. For example,  [InformationArts] observes that


aesthetic distance is no longer tenable when artist are engaging the same systems used in general communications and research

Does that mean that we must adopt open source to make an artistic statement, or shy away from the powerful visual effects enabled by for example shader technologies. Of course not. But it does indicate the need to critically reflect on the need and functions of these tools, and not adopt a technology, style or for that matter realism simply because everybody does so. Art, through its history, teaches us how to fight against both visions of dominance, and dominant vision(s)!

(C) Æliens 04/09/2009

You may not copy or print any of this material without explicit permission of the author or the publisher. In case of other copyright issues, contact the author.