Conversations is an experimental Distributed Multi-User Virtual Environment forming the central focus of an Australian Research Council funded study investigating the reformulation of narrative within digital cinema. It offers viewers an immersive multi-modal interactive narrative experience dealing with the events leading up to the escape, recapture, trial and hanging of Ronald Ryan at Pentridge Prison, Melbourne in 1967, the last person publicly executed in Australia. It allows viewers “to inhabit the landscape of the escape, to experience the confusion of the crime scene first hand; then it enables us to encounter the key protagonists of the subsequent trial. Instead of being presented with ‘facts’, viewers are asked to negotiate the historical landscape themselves, to draw conclusions from their encounter with both the scene of the disputed events and the opinions of the characters who dominated their aftermath”.* It has been developed in the form of two demonstrators. The first for the Powerhouse Museum, Sydney where it was premiered in November 2004 and the second at iCinema’s Scientia Laboratory where it is available for presentation as part of a portfolio of experimental works. The project has been developed in collaboration with the University of Technology, Sydney, the University of Sydney and the ZKM Centre for Art and Media, Germany.

 

* McQuire, S. and Papastergiadis, N. (2006). Conversations: The Parallax Effect. iCinema: Sydney.

Axonometric view of Conversations 3D, showing a woman wearing VR goggles viewing the virtual environment.
3D axonometric view

Background

The rise of digital media has led to a decline in the use of traditional single-layered narrative and the corresponding loss of a major instrument of cinematic representation. This study investigates the reformulation of narrative within digital cinema through the integration of three models of interactivity so as to produce a new emergent digital narrative form. The study tests the proposition predicted in the cinematic theory of Gilles Deleuze and the systems theory of Manuel DeLanda that narrative, when generated as a complex of digitally interactive forms, provides the opportunity to recapture the significance of narrative within digital cinema, through its enactment within a multi-layered, emergent virtual space.

Objectives

Effect the synthesis of three prior experimental interactive narrative models: the multi-branched, the navigable and the algorithmic.
Demonstrate the expressive structure of a digital narrative as reformulated within the synthesis of the three prior models through its experimental application in the Distributed Multi-User Virtual Environment entitled Conversations.
Test through critical interpretation the representational character of these new forms of narrative emergent within a distributed multi-user virtual environment as a type of social dialogue emergent within a digital space.

Demonstrators

Conversations is composed of three electronically linked installations that are positioned at remote locations within the museum. The visitor at each installations wears a head mounted display (HMD) whose twin LCD screens present an immersive stereoscopic display. A gyroscopic tracking device, (Intersense) attached to each HMD, monitors the movements of the viewer’s head, so that it can generate images which correspond to the direction and angle of view. Microphones and headphones attached to the HMD enable the viewers at the three remote locations to hear and speak to each other. The microphones are also linked to a software based voice recognition system that enables the viewer to also interact with the environment by means of spoken commands. Each of the installations is equipped with a PC and high performance real time computer graphics hardware (nVidia). Custom application graphics software enables the construction of a shared distributed virtual environment in which the three viewers are simultaneously immersed and can individually navigate and interact with the features of this hybrid space of representation.

Conversations installation at The Powerhouse Museum, showing people wearing headmounted display glasses.
Installation at the Powerhouse Museum, Head Mounted Display

Its main attributes are:

A contemporary stereoscopic panoramic photograph of the Pentridge Prison and its environs. The viewers, wearing the HMDS, find themselves located within this scene and can freely look about it in any direction. The viewers also move between two different renditions of this scene – one is a daytime setting where the sequence of Ryan’s escape from prison is enacted, and the other is a nighttime setting inhabited by the ghosts of the protagonists in Ryan’s story.

Actors, who are the central characters in Ryan’s escape and subsequent murder trial, are three dimensionally placed within this photographic setting. These narrative agents are video-graphic entities that have been previously recorded in a Chromakey studio. In the Pentridge daytime setting, these narrative agents have been scripted and composited into the virtual environment and present the viewers with a linear narrative experience. In the nighttime setting, where Pentridge has become a ghost world, these ethereal narrative agents float about and individually respond to each of the three visitors, both when they are purposely looked at, as detected by the tracker on the HMD, and when they are talked to, as interpreted by the voice recognition system.

Computer generated avatars of the three visitors, who become aware of one another, are positioned within the Pentride setting. Each is identified by a different color, with which each corresponding installation is also coded. The real time behavior of these avatars depicts the viewers’ navigation of the virtual environment, their interactions with the narrative agents, and with one other. The trackers on the visitors’ heads control the corresponding behavior of their avatars’ heads, and when voice detection is activated, these avatars are animated by typical conversational gestures.

A panoramic view of the visualisation showing Pentridge Prison in daytime.
Panoramic view, Pentridge in Daytime
Panoramic view of Pentridge Prison at night time. There are holograms of priests and figures wearing VR goggles in the foreground.
Panoramic view, Ghost world – Pentridge at Nighttime

In this configuration, an innovative new component has been added to the three HMD installations. It is a large immersive hemispherical projection, set within a three-meter diameter half dome, mounted vertically in front of the viewer. Whereas each of the HMD viewpoints expresses the natural point of view of a visitor within the shared virtual environment, the hemispherical projection presents a global view of this environment, where one can simultaneously see all the multimedia assets – the Pentridge prison setting, the videographic narrative agents and the three visitor avatars. Using simple trackball control the user can rotate this scene and move back and forth between distant to close-up viewing. This immersive visualisation system provides a highly effective technique for rendering a complete audio-visual observatory of the Conversations multi-user, multi-modal, virtual environment with all of its narrative and interactive components, and is a significant advance on the multi-screen monitor setups that have been used until now to elucidate the workings of such complex mixed reality situations.

Research Outcomes

ARC InvestigatorsDennis Del FaveroRoss GibsonIan Howard, Jeffrey Shaw
Programmers: Matthew McGinity, Keir Smith, Joachim Tesch, Teewon Tan, Craig Jin, Andre Van Schaik, Dennis Lin, Alan Kan
Project Funding: ARC DP0209550

2004-06

Distributed Multi-User Virtual Environment

  • Scientia, UNSW, Sydney, 2006
  • Powerhouse Museum, Sydney, 2004/05

Directors: Dennis Del Favero, Ross Gibson, Ian Howard, Jeffrey Shaw

Writer and Video Director: Ross Gibson

Video Editor/Compositor/ Cinematographer: Greg Ferris

Project Manager: Kate Dennis

Production Manager: Damian Leonard

Production and Technical Co-Ordinator: Volker Kuchelmeister

Production Assistant: Allison Bisshop

Software Developer/Co-Ordinator: Matt McGinity

Software Developer: Keir Smith, Joachim Tesch

Audio Engine Software Developer: Teewon Tan, Craig Jin, Andre Van Schaik, Dennis Lin, Alan Kan

The Computing and Audio Research Laboratory: CARlab, University of Sydney

Character Animator/3D Modeller: Guido Wolters, Steve Weymouth, Andy Homan

Sound Designer: Robert Hindley

Panoramic Still Photographer: Peter Murphy

Technical Director/Sound: Max Harrison

Props and Costumes Designer: Imogen Ross

Hair and Makeup Stylist: Sue Carroll

Line Producer: Sue Midgley

Camera Operators: Katharina Klodt, Jodie Smith

NIDA Team: Amanda Morris, James Tait, Lisa Lipson, Jeavons

Ronald Ryan: Tim Burns

Peter Walker: Scott Williams

Ryan’s Mother: Mercia Deane-Johns

Prison Chaplain: Craig Coventry

Premier Bolte: Tony Girdler

Justice Starke: Ted Bennett

Leader Of The Prosecution: David Robbins

Leader Of The Defence: Stephen Holt

State Hangman: Damien Parker

Prison Officer: Kristen Throw

George Hodgson’s Ghost: Simon Mills

Anti-Hanging Campaigner: Kath Parle

Extras: Daniel Heckenburg, Casey Whitelaw, Nick Mariette

 

The iCinema Centre gratefully acknowledges the financial assistance of:

The Australian Research Council;

University of New South Wales – College of Fine Arts and Faculty of Engineering, School of Computer Science and Engineering;

National Institute of Dramatic Art;

CARlab at the University of Sydney;

ZKM, Centre for Art and Media, Karlsruhe, Germany;

the Powerhouse Museum.

 

It acknowledges the support of:

dedece;

Xenon Systems;

Harbour Street Management;

3 Arts Make-Up Centre;

Penny Hagan;

Anthony Speed;

Jackie Farkis;

Milvia Harder;

Chameleon Touring Systems.

 

Funded by:

Australian Research Council Discovery Grant: The reformulation of narrative within digital cinema as the integration of three models of interactivity.