12

towards an aesthetics for interaction

experience is determined by meaning

towards an aesthetics for interaction

learning objectives

After reading this chapter you should have an understanding of the model underlying game playing, and the role of narratives in interaction. Furthermore, you might have an idea of how to define aesthetic meaning in a cultural context, and apply your understanding to the creative development of meaningful interactive systems.

As in music, the meaning of interactive applications is determined, not only by its sensory appearance, but to a high extent by the structure and functionality of the application. This observation may, also, explain, why narratives become more and more important in current video games, namely in providing a meaningful context for possible user actions.

In this chapter, we take an interactive game-model extended with narrative functionality as a starting point to explore the aesthetics of interactive applications. In section 12.1, we will introduce a model for interactive video games, and in section 12.2 we will present a variety of rules for the construction of narratives in a game context. Finally, in section 12.3, we will characterize the notion of meaning from a traditional semiotics perspective, which we will then apply in the context of games and interactive multimedia applications.

...



game theory


...



classic game (reference) model


rules vs fiction


game fiction is ambiguous, optional and imagined by the player in uncontrollable and unpredictable ways, but the emphasis on fictional worlds may be the strongest innovation of the video game.

...



theory of interaction


are games relevant for a theory of interaction?

effective service management game(s)


additional criteria


game play


... structure of interaction with game system and other player(s)

component framework


...



pattern(s)


...



intimate media object(s)


  1. glow tags -- a subtle way to trigger the person who has placed it or who sees it
  2. living scrap book -- to capture and collect information and media digitally
  3. picture ball -- as an object of decoration and a focus for storytelling
  4. lonely planet listener -- enabling people to listen to a real time connection to another place

intimate media experience(s)


...



experience as meaning


film as art


by still being read, the little treatise seems to prove that in spite of all the changes that have taken place in their form, content and function, films are still most genuinly effective when they rely on the basic properties of the visual medium.

...



illusion


... in film or theatre, so long as the essentials of any event are shown, the illusion takes place

patterns of light


... we can perceive objects and events as living and at the same time imaginary, as real objects and as simple patterns of light on the projection screen, and it is this fact that makes film art possible.

frames of reference


it is one of the most important formal qualities of film that every object that is reproduced appears simultaneously in two entirely different frames of reference, namely the two-dimensional and the three-dimensional, and that as one identical object it fulfills two different functions in the two contexts.

principles of montage


film technique


cinematographic motion


narrative implication(s)


composition


composition, ..., relates the representational and interactive meanings of the picture to eachother, through three interrelated systems.

representation(s)


information value


...



scientific context


societal context


technological context


creative context


learning/meaning


...



basic geometrical shapes


... basic geometrical shapes have always been a source of fascination, even of religious awe. And our scientific age is no exception.

nervous system


(basic geometrical shapes) have been thought to have the power to directly affect our nervous system, for instance by the constructivist artist Gabo: the emotional force of an absolute shape is unique and not replaceable by any other means ...

...



semiotics -- a theory of meaning


semiotic modes


... is the move from the verbal to the visual a loss, or a gain?

complexity


... it has to be handled visually, because the verbal is no longer adequate?

multimedia


the multi-modality of written texts has, by and large, been ignored, whether in educational contexts, in linguistic theorizing, or in popular common sense. Today, in the age of multimedia, it can suddenly be perceived again.

quotes


semiotic landscape


the place of visual communication in a given society can only be understood in the context of, on the one hand, the range of forms or modes of public communication available in that society, and, on the other hand, their uses and valuations.

...



sonic act(s)


what is the meaning of meaning in apparently meaningless expressions

modality


one of the crucial issues in communication is the question of the reliability of messages. Is what we see or hear true, factual, real, or is it a lie, a fiction, something outside reality? To some extent the form of the message itself suggests the answer.

coding orientation


  • technical/scientific -- effectiveness, blueprint
  • sensory -- pleasure principle is dominant
  • abstract -- used by socia-cultural elite
  • naturalistic -- dominant common sense paradigm of realism

aesthetics of shock


it is within the realm of probability that the shock, which Walter Benjamin diagnosed as being film's aesthetic innovation, will undergo renewal and intensification with far more sophisticated means.

voyeurism


the most obvious symptom of this loss of distance will be a voyeuristic, dissecting penetration of representations of objects and bodies.

TV


for the first time in the history of man's striving for understanding, simultaneity can be experienced as such, not merely translated as a succession in time.

sensory stimulation


although the new victory over time and space represents an impressive enrichment of the perceptual world, it also favors a cult of sensory stimulation which is characteristic of the cultural attitude of our time.

direct experience


proud of our inventions -- photography, film, radio, ... -- we praise the educational virtues of direct experience.

communication


when communication can be achieved by pointing with the finger, however, the mouth grows silent, the writing hand stops and the mind shrinks.

channels


... the decisive questions remain: who controls the channels, who distributes right of access, and who exercises economic and political authority over the networks?

visions


... the history of technological visions is the history of our dreams, our vagaries and our errors. Media utopias fluctuate, often occurring in a magical or occult ambience.

synopsis course(s)


... the curriculum should emphasise basic principles, and to the extent possible employ open standards and open source. Practical assignments must be centered on local culture, and stimulate the young talent to explore innovative applications for cultural heritage, serious games and artistic expression.

where, what & why


  • where -- Ethiopia & VU
  • what -- introduction multimedia
  • why -- to develop curriculum

environment


  • low end computers -- windows, linux
  • elementary skills -- programming, design

assumptions


  • open source -- flex 2 sdk, Delta3D
  • open standards -- XML, X3D
  • basic principles -- exploratory development

targets


  • local -- present local cultural heritage
  • serious -- develop serious game(s)
  • benefits -- promote local culture and commerce

presentation


  1. philosophy -- pathos, ethos, logos
  2. trailer -- drama, apocalyptic, appeal to player
  3. climate star -- scientific issues & game play
  4. game development -- architecture and project plan

phrase(s)


  • art -- rethorics of the material
  • technology -- solving problem(s)
  • science -- establish a theory

perspective(s)


  1. transcendental -- abstract form of experience,  [Kritik]
  2. speculative -- criteria for beauty,  [Urteil]
  3. phenomenological -- self-conscious subjectivity,  [Phenomenology]
  4. psychoanalytical -- sub-conscious meaning,  [Witz]
  5. pragmatical -- art as experience,  [Pragmatics]
  6. hermeneutical -- understanding of the senses,  [Hermeneutics]
  7. semiotics -- social construction of meaning,  [Semiotics]

dimensions of aesthetic awareness


  • spatial -- topological relations, layout of image
  • temporal -- order, rhythm, structure
  • dynamic -- interaction, reflection, involvement

projects & further reading

As a project, explore the ways narratives may be constructed from a collection of images. Deploy the various editing facilities for providing flashbacks, flashforwards, and other (temporal) relations within storytelling.

You may implement this using flash, VRML, or even try to embed such a narration facility in a game level developed with the Delta3D or the Source SDK.

For further reading I suggest you to take a look at more theoretical material from media theory, such as  [Remediation]. Also there is a large collection of books from MIT Media Press that is of relevance for our new visual culture.

the artwork

  1. einzelganger -- walking man of Alberto Giacommeti, taken from an aanouncement of the Ives Ensemble, Amsterdam.
  2. game component framework, from  [GamePatterns].
  3. diagram MIME
  4. diagram experience as meaning
  5. Roy Lichtenstein, 1962, (a) Kiss II, (b) Masterpiece, (c) Forget it, forget me.
  6. edgecodes -- showing George Lucas and his editoroid.
  7. El Lissitzky, suprematist works
  8. El Lissitzky, suprematist works
  9. Roy Lichtenstein, 1999, Still lifes with brushstrokes
  10. Les Demoisselles dAvignon, Picasso, 1908, regarded as the start of Cubism, and Le Goutier, Jean Metzinger, 1911, often referred to as the Mona Lisa of Cubism.
  11. poster for exhibition of dutch china work.
  12. signs -- abstract,  [Signs], p. 228, 229.