7/97

San Diego, CA

Published in ACM SIGGRAPH Computer Graphics, Volume 4 Number 4, November 1997.

Virtual reality is irrupting all around us. It is coming from a number of directions: popular culture, the military, scientific visualization, entertainment systems, digital cinema, computer networking, fast computers and the economies of technological consumption. So what of art? How should artists respond to virtual reality and how are the developments of virtual reality (VR) responsive to art?

All aspects of VR are still in a formative stage. While there are specific applications that are fairly well implemented, such as military simulation systems, their existence is suggestive of the potential of VR as a transformative medium for cultural discourse. These systems have spurred on a speculative popular fantasy about the future of media and the reformation of communication and entertainment environments by the construction of sensory rich realms for mediated experiences that have an equivalency for sensory experiences based in a real world.

So far, VR based entertainment systems have offered glimpses of this promised future, with little actual delivery. VR's significant cultural operation is in its fantasy, rather then through people’s actual VR experiences. It has captured the interest of artists for a variety of reasons, and several artists are now creating works with VR characteristics. This paper identifies a mode for artists working with VR by providing an overview of artists' engagements with previous media forms, and looks to identify some of the key formal structures of VR that suggest connections to art discourses which can provide alternative strategies for constructing artworks with virtual reality systems.

Artists are provoked by the promise of VR for several reasons. One is its current cultural operation as a fantasy to play out visions for post-reality experiences. The other is that VR, by bringing together several technological developments, provides a significantly different set of formal parameters for articulation then have existed before. Both of these are interesting aspects to explore, and VR works will at the very least be approached with the expectations and desire that the first condition describes, however, in the long term, the cultural operation of VR will be reformed by the actual expressions arising out of the second set of conditions.

Artists working in previous media forms have worked with a similar set of dynamics. It is useful to note some methods that artists have engaged with in these media forms in relationship to the broader cultural role of that particular media form. Photography and film have been accorded as having provoked profound cultural and social transformations. They came into existence embodying the technological and conceptual forms of their time. VR's introduction is often presented as having a similar potential transformative effect. However, as the most recently introduced cultural medium, video offers an interesting model to examine the relationships of art and culture, from which comparisons and contrasts can be drawn to VR based art practice.

Video is the most recently precursant media form to VR that artists have engaged with. Video became a significant artistic medium beginning in the late 60's; a full 25 years after video began as a cultural forum. Computers were invented around the same time that television was introduced to the public, yet only recently have computers had the capabilities to be a tool for cultural production. When video was introduced to the public, it had its major components in place. The addition of color a few years later has been being the only major development in the basic structure of video since its introduction.

When artists began working with video in the late 60's, television had achieved a ubiquitous level of cultural penetration. A semantic structure for video had developed over the previous 25 years which was distinct from the cinematic model of film, although it was able to contain a filmic model as an aspect of its structure (something that each progression of media forms is able to do). With a much lower resolution and a squarer format, video did not emulate the compositional aesthetics of film. It built upon its unique temporal and spatial qualities. Video is a real time medium. Television is a live space. Most early television took place live, much still does. (Although more live events are becoming pre-recorded, i.e., 1996 Republican National Convention.) The later introduction of videotape, only displaces the liveness of television. Tape is a continuously active, continuously alive, recording medium, unlike film, which fixes the shadows of the lens via the chemical reactions imbedded in its immutable series of photographs.

Televisions' liveness also comes with its ubiquitous presence. We don't go to television, it comes to us. Television isn't an event, it is a condition. It is continuous and omnipresent. We turn on and off TV receivers and they display the current status of the television environment. Much like opening a curtain will reveal the condition of the weather. This quality of liveness also is the basis of its spatial form. Video contains images in a substantial object, which melds into the space of our lives. It is the scale of furniture vs. films architectural scale. It occupies our lived space, yet it is at a scale that is less then life size. We pretend that we can turn it off and that it isn't there, yet its presence has changed everything about how we structure our environment.

When artists first began working with video, the work had little to do with the types of narrative structures that had developed in television. Artists engaged the temporal and spatial qualities particular to video to express the type of formal and conceptual relationships that video offered, derived from the same set of concerns that developed out of minimalism, performance art, earthworks, and conceptual art of the time. Video art was distinct from its two closest cultural cousins -- television and avant-garde film art, which itself was quite distinct from commercial cinema. Both video art and film art, used the unique formal capabilities of their mediums to engage in the exploration of expressions, representations and abstractions that were in a discourse with developments in the broader art world of their time.

Both of these modes of artistic practice have shown to have considerable depth and range over time and have continued to be important modes of contemporary art practice. The concerns of artists working with them have changed over time, moving from the formal engagement of structural qualities to the deconstruction of cultural codes, expressions of personal narrative and the engagement of identity politics, in accordance with the explorations of the artworld at large.

One primary difference exists between the artists who have engaged with video and film in the last few decades, and artists that are now working with VR. With these other mediums, artists were working with forms that had already achieved a high level of cultural dissemination and had established semantic modes for the articulation of content. Artists working with these media were often working against the expectations of their audiences. Audiences had a culturally ingrained literacy in reading films and video, and often artists worked to provoke the audience to question this unconscious mode of understanding. The art film was often seen to have an analogous relationship to commercial film as poetry has a relationship to prose. In video, the form was an extension of the modes that had arisen out of extended sculptural practices, along with an intimate engagement with personal and social concerns.

With VR the artist does not have this situation. There may exist a wide spread notion in culture of what VR might be, perpetrated by an economy whose growth is now dependent on the continuing deployment of high technology, but as a mode for cultural experiences, it doesn't have a culturally dispersed, significant semantic mode for its operations. What it does have are some identifiable qualities which resonate with some artists concerns at this time. Its possibilities are enticing enough for some artists to overlook the difficulties of working with a medium about which so much is yet to be described (not to mention its enormous expense, and difficult tool set).

VR is an important form for artists because it is the medium that best engages the qualities of our particular cultural moment. VR choreographs images in time. However, both images and time have different modes in VR. These differences can be identified in isolation and seen in combination to provide the significantly unique aspects of this medium. Images in VR are derived from a system of spatialized simulation. Time in VR is structured by user interactivity. The intertwined relations of these two create its unique mode. It engages both space and time in a manner that is not only inclusive of previous media forms, but is extensive in its strategies for containing them. As photography is instanced in film, and film is instanced in video, all of these media are instanced in VR as a portion of the structure of VR.

As VR comes from the collective development of several different areas of computing, there are many variations on what constitutes a VR system. One element that often gets discussed as being crucial to VR is that of sensory immersion. Immersion arises when the vision of the user is sufficiently contained by the mediated images such that movements of a user's vision apparatus result in changes of viewpoint in the images of the mediated environment. This phenomenon has several manifestations in pre and post-cinematic forms, such as panoramic painting, phantasmagoria's and amusement park rides that couple motion controlled platforms to movies. Immersion is not dependent upon the fundamental characteristics of VR, but can arise out of them. Immersion is therefore a secondary characteristic of VR, although it provokes much of the popular fantasy about VR, offering the sensory substitution of the mediated for the real. However, the more profound interchange between the mediated and the real is an ongoing process of culture, and is a more complex situation then a simple one to one substitution. VR enables the articulation of this condition by providing a form that can engage in an active exploration of this situation.

VR can contain cinema, it can contain video, it can contain sculpture and architecture, but it doesn't simply replay them. It re-stages them in a metamedium, which is experienced as the interplay of representations. The images that display VR are all synthetic, post-optically derived from computer algorithmic processes. These images begin by creating spatial relationships between dimensional forms. These forms become surfaces for the application of images. VR is a faceted space of image planes. These images can have an initial connection to an optical source, but they also come about as a simulation of the structural rules of a subject that is algorithmically described. Images can also be used to create the initial spatial form. Geographic simulations use object shadows on satellite images to determine the z-axis of objects from which computer models can then be automatically derived. In these systems the VR is an image of a space of images derived from a spatialized image. The organization of these convolutions of representations and spaces gives rise to the framework of simulation.

Simulation is a break down of the simple one to one relationship between referent and sign, such that the deployment of power and the discernment of meaning are caught up in a complex relationship where the sign of the real can proceed or interchange with the real itself. This is the representational structure that VR overtly engages.

VR transforms cinematic representation by adding a spatial dimension to the image. This positions VR to cinema, as cinema is the photograph. This spatial dimension isn't simply an optical trick that fools the brain into imagining that a red and green Frankenstein is coming out of the movie screen (although this is the trick used in immersive systems), it is as important a component of the image plane as up/down or left/right. This extension of the image necessitates the interactive engagement of the viewer, in order for the image to be read. Our perceptual apparatus only sees images. We have lenses that project a focused, 2-D image onto our retina. Our brains are very clever in taking two of these images, separated by a couple of inches, and discerning 3-dimensional form. It is an easy thing to engage the brain with trickery to mimic the effect. But the important thing to realize is that we only see 2-D images of the world. However, we understand the relationships of these 2-D images by moving through the world and experiencing it in three spatial dimensions. Our visual perception of the world is created by 2-D representations given spatial coherency through our active engagement with dimensional space. Seeing and understanding are in a continuous relationship between representation and spatial experience.

We can't see VR without interacting within it. Interactivity is the mechanism through which VR has temporal form. While interactivity can be seen as a naturalistic description of our temporal operation in the world at large, or with the cultural forms of architecture and sculpture, it is a new mode for the engagement of media experiences.

Certain aspects of interactivity and media have reached a general level of functionality through the development of video games over the last 20 years. Particularly as a format for the engagement of youth (and primarily boys), video games have created an expertise and ease of engagement with media in an active way. This provides a functional literacy for a type of interactivity in which there is a physical responsiveness to stimulation whose actions are consequential for the continuation of a narrative experience. In other words, a functional literacy, or facility for hand-eye coordination has developed in which users understand that they have subjectivity embodied in a consequential relationship to a thematic structure. This embodiment is articulated through the connection of a physical interface to a limited set of possible actions that the embodied subject can undertake.

The interactivity in the video game system establishes some crucial parameters for further development in VR systems. First of all, it creates the facility for exchange between action and representation. The coordination of screen, eye and hand in the creation of consequential action in a media based system has been an important cultural legacy of video game systems. In many ways, the general cultural engagement of this aspect of interactivity spurred on the implementation of graphical user interfaces in computer systems, which had actually been theorized and developed long before they were ever coalesced or widely deployed.

Video games provide one other shift, which is potentially the most profound one, locating the viewer as a subject within the narrative of the game. This development creates a distinct difference from film and video for the expression of narrative structure. Film and video create subjectivity through a displaced voyeuristic eye or via literary models. These models are quite facile and have extensive scope. However video games and VR create subjectivity through experiential models. This is a key point for the interest in, and the difficulties of, VR works. As engaging a form as interactive video games are, they do little to engage narratives of any complexity. Video game narratives are successful for pre-adolescent males. They engage in narratives of very simple structure. Hunt and kill or die, is the most popular, followed by, drive fast until you crash. As soon as elevated testosterone levels begin to subside, an engagement with other narratives becomes of interest. However the successful engagement with narratives of greater complexity has a spotty record within highly interactive media. This is in part due to an insufficient understanding of the representational structure of simulation.

The proliferation of systems of representations is one irruption of this condition that necessitates the type of qualitative change in strategies of engagement with cultural forms that VR offers. In traditional media realms such as television, an increase in the quantity of narrative streams of 2 orders of magnitude has occurred in the last 10 years. The capabilities of digital signal compression and switching bring 100's of simultaneous narrative streams into people’s lives. Current systems of indexing and organizing these streams are insufficient, but lead to the self-construction of a larger narrative space, a simple, primitive form of interactive media space. The next stage of this development is lurking around the corner, as this becomes a part of the extensive system of 2-way, computer communications networks.

This is an interesting turn of events. The fact that a communication form that came into being by a serendipitous series of events, beginning with a military experiment, becoming an obscure tool for academics, and then becoming a sub-cultural forum, could even be considered as the primary paradigm for the future of communications exchange, superseding broadcast media, telephone services and the mail system is a fascinating development. If this system is to achieve its potential, it will necessitate another level of formal transformation that will allow for the multiple articulations of information structures in coherent forms. The key to this transformation may be the self-conscious location of the subject within the representational matrix.

VR overtly engages this structure. It locates its subject/viewer at the focal point of its spatialized representations. Architecture has some similar engagements, yet VR is not limited to the underlying constraints of architecture (utility, gravity, and material strength) or even the constraints of an Euclidean dimensional space, as we know it. Dimension in VR space can have an associative, malleable form that can be contextually dynamic. Space can have a thematic structure, just as time has in cinematic narrative, and this structure can actively reconfigure itself as context requires.

Understanding the formal structure of VR is a necessary step in devising a syntactical structure for the creation of meaningful VR works. Artists coming to work with VR at this time, find a potential form whose mechanisms have so far been deployed in simplistic expressions of one to one relationships between action and response, referent and symbol, subject and object. For VR to develop into a significant cultural form, the complexities of interrelationships between these aspects will have to be engaged with. For artists, the development of VR as a cultural form will provide a point of reference for works, but artists' intentions with VR will likely extend into realms that are quite different then its general cultural deployment. However, until there is a general cultural mode of VR, artists’ activities will be involved to some extent in devising mechanisms for creating the syntax of VR.

There can be seen to be a potentially rich connection between the modes of artistic inquiry and the potential mode for VR expressions. Art is the exploration of articulating the pluralities of meaning and the space of interpretation. Art engages a highly interactive form for the creation of meaning. As Duchamp stated, "the spectator makes the picture". Art exists in the space between intention and interpretation. The artwork is a system for the production of meanings, and this system is dependent upon the viewer setting in motion the apparatus of signs that comprise the whole work. This mechanism by which art engages meaning provides a viable model by which the experiential and the representational coalesce into an active structure for the complex engagement of meanings, and a matrix for the location of producer, cultural product and subject.

VR brings us back to an engagement with creating a system of meanings that has always been a part of the cultural role of art. It is therefore quite natural that a number of artists are among the first cultural producers to be actively engaged in the development of VR works.

In my own work, VR is an area that I both work directly in and address aspects of in works of other form. This paper maps out some of the terrain that I operate in when I am working. Some of these ideas have an obvious apparency in the works, some do not, and there are a multitude of threads in the works that aren't articulated here. Several works that are representative of these concerns are outlined below.

In The Video Wind Chimes the viewer encounters a an interplay between the cultural and the natural. This public artwork that operates by projecting multiple video projections on the ground of signals from the broadcast television spectrum. The specific signals that are shown are chosen by the wind blowing upon the projectors themselves, which are inside of winged housings, suspended by ball joints from light poles. As the projectors sway in the breeze, they scan the broadcast television spectrum, and articulate patterns of natural and cultural forces, describing a condition in which the role of the cultural and the role of the natural have reached a point of sublimation. The piece is dependent upon the visceral engagement of the viewer to begin understanding its engagements. The feel of the wind on your skin, in conjunction with the vertiginous shifting of the image planes beneath your feet, as multiple narrative streams spew from the night, is a revelatory experience of the atmospheric condition of culture. The projection of these images upon the skin, makes ones skin crawl, as they leave the projection cone and realize they can't leave the projection cone, either in its physical or its cultural form.

In the Event is a public artwork, commissioned by the Seattle Art Commission for the Key Arena at Seattle Center. It consists of 28 monitors set in a wall 60’x8’ of cast aluminum relief panels. The monitors show 4 video sources being processed in real time by a network of 9 computers. These computers transform the incoming video such that the viewer must transform their own vision into a cinematic apparatus in order to perceive the mediated imagery. By doing so (or by not doing so) they are actively interacting with the mediascape with which the piece operates. This interaction creates an alternative space that occupies a zone in-between the physical and the mediated environment. This is the zone that I consider to be the realm of the virtual.

The last artwork to be discussed here is a virtual reality installation titled Apparitions. Apparitions constructs a physical space with three large video projections, a viewing environment of 3 chairs (red, green and blue), with one chair being a user interface constructed out of a reclining chair, a wheelchair, a toy gun and a joystick. The interactive viewer becomes a subject in a clinic that examines the various manifestations of their subjectivity, through an examination of virtual reality. The viewer is taken through various stages of the clinic, where their subjectivity is continually transformed through various examinations, and where the viewer has various degrees of control in the simulation environment. The piece uses an interesting engagement of viewer interactivity and structured narrative; the viewers' control opens up and closes down at times throughout their experience, providing a linear narrative structure, with a multiplicity of paths. Montage sequences, triggered by user actions, flank either side of the interactive computer space. The computer space itself has a basis in architectural space, which it departs from and returns to, providing spatial continuity and the key tool in narrative structuring.

My own practice as an artist has been in anticipation of VR sense the early 80's. My engagements with video, film and computer controlled installations have all been in the context of the increasingly complex interrelationships between notions of the real and the mediated. The congruence of these concerns in the form of virtual reality creates a media form that overtly engages this question. The task for the moment is gaining an understanding of how this form actually functions in its operations, in order to create the type of works that will lead to a further understanding of our particular cultural condition.