Disquiet in the plasma

This art icle was downloaded by: [ St ockholm Universit y Library] On: 19 February 2015, At : 05: 15 Publisher: Rout ledge I nform a Lt d Regist ered in England and Wales Regist ered Num ber: 1072954 Regist ered office: Mort im er House, 37- 41 Mort im er St reet , London W1T 3JH, UK Digital Creativity Publicat ion det ails, including inst ruct ions f or aut hors and subscript ion inf ormat ion: ht t p: / / www. t andf online. com/ loi/ ndcr20 Disquiet in the plasma Andrew Morrison a a , Geska Helena Andersson b , Robert Brečevi ć b & Synne Skj ulst ad a Int erMedia , Universit y of Oslo , Norway b Int eract ive Inst it ut e , Sweden Published online: 01 Dec 2010. To cite this article: Andrew Morrison , Geska Helena Andersson , Robert Brečevi ć & Synne Skj ulst ad (2009) Disquiet in t he plasma, Digit al Creat ivit y, 20: 1-2, 3-20, DOI: 10. 1080/ 14626260902867782 To link to this article: ht t p: / / dx. doi. org/ 10. 1080/ 14626260902867782 PLEASE SCROLL DOWN FOR ARTI CLE Taylor & Francis m akes every effort t o ensure t he accuracy of all t he inform at ion ( t he “ Cont ent ” ) cont ained in t he publicat ions on our plat form . However, Taylor & Francis, our agent s, and our licensors m ake no represent at ions or warrant ies what soever as t o t he accuracy, com plet eness, or suit abilit y for any purpose of t he Cont ent . Any opinions and views expressed in t his publicat ion are t he opinions and views of t he aut hors, and are not t he views of or endorsed by Taylor & Francis. The accuracy of t he Cont ent should not be relied upon and should be independent ly verified wit h prim ary sources of inform at ion. Taylor and Francis shall not be liable for any losses, act ions, claim s, proceedings, dem ands, cost s, expenses, dam ages, and ot her liabilit ies what soever or howsoever caused arising direct ly or indirect ly in connect ion wit h, in relat ion t o or arising out of t he use of t he Cont ent . This art icle m ay be used for research, t eaching, and privat e st udy purposes. Any subst ant ial or syst em at ic reproduct ion, redist ribut ion, reselling, loan, sub- licensing, syst em at ic supply, or dist ribut ion in any form t o anyone is expressly forbidden. Term s & Condit ions of access and use can be found at ht t p: / / www.t andfonline.com / page/ t erm s- and- condit ions Digital Creativity 2009, Vol. 20, Nos. 1 –2, pp. 3 – 20 Disquiet in the plasma Andrew Morrison1, Geska Helena Andersson2, Robert Brečević2 and Synne Skjulstad1 1 InterMedia, University of Oslo, Norway Interactive Institute, Sweden 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 andrew morrison@intermedia.uio.no; synne.skjulstad@intermedia.uio.no; geska@tii.se;robert.brecevic@tii.se Abstract 1 The metaphor of plasma is taken up to present and discuss movement and engagement by participants in mixed reality installation arts. Two works involving full body video portraits exhibited through large plasma screens in a variety of public settings are covered. Machinic mediations of video realism are considered in terms of embodied interaction in which viewer-participants contribute to the ‘disquiet’ of gendered figuring. Processural, proximal and personal aspects of responsive engagement are discussed. This is extended to performativity that may lead us to critical reflection of our own actions and responses in mixed reality arts. Keywords: mixed reality, video art, plasma, participation, proximity, responsive video DOI: 10.1080/14626260902867782 Introduction 1.1 Plasma in the plasma There is an elemental, life-enabling quality to plasma. It is through plasma that our human blood cells are suspended and channelled, at once enwrapped and thereby also transportable through our circulatory system. Constantly in motion, yet unseen, plasma courses within the pathways of our moving selves. Yet plasma refers to more than the liquidity of our living beings. It is also the stuff of whey, the visible liquid remainder of the process of curdling and straining the colloid milk. In physics, plasma refers to a different, distinct state of matter, also known as the fourth one, typically taking form as an ionised gas. This state allows the conduction of electrical charges in electromagnetic fields. Perhaps more widely known at the level of contemporary electronics, a plasma screen is a large flat-panel electronic visual display technology that allows high definitional views. In this article we take up these different senses and states of plasma with reference to the domain of mixed reality arts1 in which affection is liberated from the cinematic image and embodied within the viewer-participant’s active engagement with a potentially dynamic work. In such a move, Hansen argues, if we adopt a Bergsonian ontology of images in which the body is the centre of indetermination, . . . the focus must shift to the post-cinematic problem of framing information in order to create (embodied, processural and affectively constituted) digital images (Hansen 2004a, p. 270). In following this approach in teasing out relations between art, design and research (e.g. Brouwer et al. 2005), we reflect on two specially developed works that use large flat screens in the realisation of mixed reality art Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Morrison et al. Figure 1. Women That Turn by Performing Pictures at ‘Balta nakts’ Galerija Centrs, Riga Old Town. designed to engage particpants in the interplay of different embodied and affective senses of ‘plasma’. Sensors make it possible to sense the plasma; they enhance our access to the sensory while not intruding into the wider aesthetics of engagement and expressivity in the mix of the im/material as cultural communication (e.g. Udsen and Jørgensen 2005; Redström 2008). In these two pieces – via video and sensors – we explore machinic mediations of video realism, questions of gender and representation, and processural realisations of embodied identity, now in the screen, now in our own proprioperceptive activity. In these works, metaphorically, there is plasma in the plasma; it is also present in our own responses to engaging with electronically mediated art. The medium, the conductivity of the plasma and our own embodied interactional engagement are intertwined. The conduction of digital ‘matter’ is ionised in the representation of large human figure-sized video portraits as well as the potential engagement of audiences by way of zones of embodied responsivity. However, we address more than the physics or digital materiality of flatscreens. ‘Disquiet in the plasma’ also concerns the estrangement or uncanniness (after Brecht and Freud) that lies 4 both in becoming aware of the fabrication of the seemingly realist video representations and their transformation through the movement of the viewer towards them. This disquiet continues as the participant subsequently engages with the works in negotiating given patternings and her own sense of embodied self and relations to the mediated responsivity she effects and affectively meets, senses and considers. Here disquiet is to do with the unsettling of a sense of familiarity and apparent tranquility that may arise in digital video mixed reality installations (Figure 1). Referring to mediatheoretical and technical-aesthetic categories, Spielmann reminds us that video is processural and contextual and that it may be defined . . . as a reflexive and genuinely audiovisual medium that can demonstrate its mediaspecific transformativity and multiplicity. . .(Spielmann 2008, p. 17). In terms of mixed reality arts, Hansen (2003) argues that through our engagement with electronic installation works we encounter an affective correlate that arises precisely from our embodied interaction with the digitally mediated. Thus: Instead of a static dimension or element intrinsic to the image, affectivity thereby becomes the very medium of interface with the image. What this means is that affectivity actualizes the potential of the image at the same time it virtualizes the body; the critical element is neither image nor body alone, but the dynamical interaction between them (Hansen 2003, p. 208). In a slightly less phenomenologically framed approach than this one, we argue that one additional turn is needed, namely a selfreflexive and critical one through which the participant also develops some distance and reflection on their experience. This too is an additional level of disquiet — an ‘experience’ as Brecht said of the theatre also lasts after the curtain falls; the processural character and qualities of mixed reality arts endure beyond the session with its sensors and plasma displays. 1.2 ‘Public plasma’ Plasma screens now appear in a host of private and public spaces (Figure 2). They function as digital canvases, that is both as windows and mirrors (Bolter and Gromala 2003). They loom large in living rooms, corporate foyers, airport concourses and train terminal lounges.2 These high definition screens allow us to see representations of our own lives, such as holiday photos, as well as those served to us as information such as details of flight departures. Often we encounter a hyper-realist aesthetics in the publicly screened media these large surfaces enable. More recently ‘high-def’ screens have begun to be used in shop window displays with human figures as digital mannequins, such in the multimediational branding by the Swedish fashion chain H&M. In a more active and engaged view, as consumers and citizens we are aware that these plasma screens are sites for the production and mediation of digital media and representations. Into them we plug our cameras and computers, breaking the screen divisions between television and laptop, watching holiday videos and playing computer and arcade-like games together at home through the same plasma space. These large, high quality screens also allow our active and embodied participation in games, such as Nintendo’s Wii. These are transformative screen spaces. They are mediational and communicative: they are designed not only to serve information but also to engage us actively in generating our own mediated meanings. 1.3 Focus of article Public Mirrors – Responsive Video is a cluster of artistically driven projects that examines relationships between the initial selflessness of the spectator and elusive reflections of the ‘other’ that are mediated through plasma screens and responsive digital video. Public Mirrors – Responsive Video are artistic interpretations of responsive video as a tool to involve the viewer as part of a narrative. The pieces are included in a research project within the field of media in public spaces. They also serve as a statement about the power of the image and modes of subjectivity that are realised in multiple contexts of public display and engagement. In this paper we investigate two pieces, namely Men That Fall and Women That Turn. These two installation pieces are shaped through with large plasma screens, sensors and micro-controllers, located in public spaces as well as gallery settings, and centred around movement and reflection. The works comment on issues about proximity in both time and space. The increasing convergence of media throughout different channels of display is investigated by the interpretation of media artifacts. The ubiquitous presence (as well as the resulting transparency) of media and its seeming disenchantment are broken, with the result that the viewer relates to media as an object. 2 2.1 Digital video, aesthetics and identity Hyper-realist aesthetic tendencies Across these artistic, informational, commercial and consumer contexts and collaborations, 5 Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Disquiet in the plasma Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Morrison et al. a form of ‘plasmatic materiality’ may be said to be realised. The high definition image quality of such screens along with the digital tools used to design and generate their content often result in an accentuated, hyper-realist aesthetic. In commercial broadcast media, digital television signals and conversions and high-end and mobile video cameras and software allow the reproduction of a realist imagery, even where these same tools can be applied to modify and manipulate visual representations. Mainstream commercial cinema hyperbolised this by way of special effects and digital image manipulation that has mimetic, realist aesthetic at its core. In contrast, in the interdisciplinary domains of electronic art, digital technologies made it possible for artists to encapsulate digital tools and technologies as part of an unfolding processural aesthetic (Paul 2003). This is one that includes the transitory, fragmentary and dynamic as part of its ‘materialist’ energies (Fuller 2005, 2008). Modes of making and mediation are intertwined not just in the generative computations of ‘software art’ (e.g. Manovich 2001, Mignonneau and Sommerer 2003) or aesthetic computing (e.g. Fishwick et al. 2005), but in the unfolding of art works in which recordable elements, loops and manipulations may themselves be central to the works’ character (e.g. Manovich 2003). This is also to situate the sensors, software and code as both means and material for expression (e.g. Reas 2006). In his large plasma screen work Splash (2001) Jeremy Welsh encapsulates many forms of underwater activity in video drawn from a public swimming pool in Trondheim, Norway. High definition digital video allows a transparency of the aquatic movement of swimmers, of waterpolo-like ball play and an overall bouyancy in a what may be called a ‘plasmatic aesthetic’.3 2.2 From video art to electronic installations A plasmatic aesthetic stands on the shoulders of earlier experimental electronic art in media 6 (Grau 2007) that themselves took time to be given legitimacy in the art world. Despite their appearance in the 1960s, video works were gradually included in the formal repertoire of art (Martin 2006); video has featured strongly, for example, in the recent Whitney Biennale of 2008. Video art began by experimenting with magnetic tape and was bound by editing and screening tools. It has shifted from being eventand time-driven (despite making extensive use of loops and splits) to also being spatially realised and computationally served. Here shifts have taken place in conceptualising installations not so much in televisual terms, but in stretching, investigating and reshaping notions of the cinematic imaginary (Reiser and Zapp 2002, Shaw 2003), such as the installations of Shirin Neshat. For example, in her work Rapture (1999) Neshat places large black and white filmic projections of women and men opposite one another; the scale and gulf of gendered distance between them is the space the viewer experiences. The works of Bill Viola, for example, have at times employed markedly slowed movements to create a sense of stillness yet also disquiet. Installations, including videoinflected ones (e.g. by Pipilotti Rist), now transverse a span of possible structures, environments and means of mediated engagement (De Oliveira and Oxley 2004), frequently being digitally composed and enacted (Leggett 2007). Digital video has featured in installation works, exploring the potential shifts from sequential film to user-driven narratives (e.g. Hales 2001, 2005). Large projections have been devised, such as in Jeffrey Shaw’s Legible City (1990) where monumental alphabetic structures are negotiated and navigated by the game-like traveller. For the viewer, other works involve physical engagement, though this may not necessarily involve direct manipulation of the computational, but a sense of coalescence of the viewer’s actions and the screening of a related but not entirely corresponding effect (Huhtamo 2007). Projection, loops and sensors have been widely used to achieve such a choreography of participation, including ones that involve dance and kinetic delays and engagements for audiences (Morrison et al. in press), such as in Bill Seaman’s Exchange Fields (2000). In the ‘dance of the audience’, relations between the participant self and the mediatised other and artifacts are inscribed as part of the dynamic and unfolding work. 2.3 Identity and kinetic portraiture Relations of self to mediated other are played out in works that engage the first person viewer with representations of themselves and pictured and simulated subjects that are both realised in a kinetics enacted through the participant’s activity. Explorations of identity are central to works that include embodied interaction (Dourish 2001) and reflexive responses, not necessarily haptic, but at times also purely in the imagination of the participants. This extends to facial and whole body processural portraitures, the focus of this article. Hansen (2003, 2004a, 2004b) addresses issues to do with faciality in digital art as the ‘digital-facial-image’, one that “. . . comes to encompass the entirety of the spectatorparticipant’s embodied activity as she seeks to negotiate the perceptual and sensorial challenges posed by the new media environments”. (Hansen 2004b, p. 363, original italics).4 Hansen (2003, p. 211) argues that “. . . new media art works turn attention back on the process of affective atunement through which facial signals spontaneously trigger affective bodily responses”. He goes on to argue that “the “autonomy” of the affection-image is shown to be indelibly bound up with the connotations of the digital medium as these are mediated through embodied response”. (Hansen 2003, p. 215, original italics). In the works we present here, however, the focus is on whole bodily portraiture that is designed to engage viewers not through facial signals but by way of their own ‘embodied interaction’ with full figures. We now refer to a number of works that specifically take up matters of ‘faciality’, kinetic portraiture and our proximal, embodied engagement with the mediated through which affectivity is reflexively revealed.5 What is important in engaging with the ‘digital-facial-image’ today is that, unlike earlier cinematic imageries, it selects from what Hansen (2004b, p. 363) refers to as unformed information that needs to be filtered by the viewer as participant. This is not to argue for the removal of a critical perspective, but rather a shift towards the realisation of the visual via embodiment in mixed reality works in which information and meaning are not separated. In Portraits (1990 – 1993)6 Luc Courchene has worked with movement in facial representations that allow the seeing visitor to engage with lifesize mediated artifacts. The creation of a sense of immediacy and intimacy are central to these works even though they make use of the computer mouse as an interactional device to build relations between the self and the mediated other in play on mirroring and recognition of similarity and difference. Portrait One, once a stand-alone piece, is now accessible online and uses options in a dialogue to simulate a conversational process. To some extent this bridges the gap between the painted or photographic portrait and the proximity and connections of the digitally enacted speaking other prompted by the user’s selections in this work. Two works by Catherine Ikam and Loius Fléri, Elle (2001) and Lui (2003), also grapple with our expectations of portraiture (Popper 2007). These artists use digital tools to create plasticity in their portraits, ones that change their composition, where the object’s digital materiality is transformative and where the viewer’s notion of a still image of another person’s face is represented dynamically. Popper writes that 7 Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Disquiet in the plasma Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Morrison et al. The work becomes a portrait in depth; the face becomes an empty envelope. The lacquered texture of the skin acts like a mirror on which a virtual light source is reflected, thereby creating an additional distance (Popper 2007, p. 251). Reface (2007)7 by Golan Levin and Zachary Lieberman uses visitors’ blinking to select parts of their faces that are mixed to generate a dynamic portrait. These are then collected in a group and allow visitors to see their own faces remixed along with those of others. Personal and collective identity are thereby presented through edits, combined elements and a generic linking of sorts across a spread of faces. The ‘generative’ potential (Morrison et al. in press) of mixed reality installation works, as well as potentially ‘envisaged’ in digital portraiture is taken up in a number of public art projects by Lincoln Schatz.8 Across Time (2005)9 features plasma screens with a camera between them that captures the immediacy of public activity in the foyer of the Spertus Institute building in Chicago, USA and building progress of a new, adjoining museum space filmed from above, over time. The work shows a dynamic process: a portrait of the activity in the old building and the making of the new structure. Momentum and memory are juxtaposed on these two screens, showing horizontal activity across ground level in the foyer and the developmental, layered construction of the new edifice. Schatz takes temporal and self-representational aspects of dynamic portraiture even further in his recent piece Cube (2007).10 This work consists of the generative selection of random images from an hour long sitting by a variety of people, in collaboration with the artist. The sitting is captured by 24 video cameras and then served to a plasma screen across 24 hours. Both the time of capture, the process of delivery and the time of presentation challenge the notion of the fixed image of the portrait. 8 This last example, together with the proceeding ones, refers to video and process, faciality and embodiment. By materializing the virtual elements of the object or work, new media artworks recast them as the catalyst for the viewer-participant’s own virtualization (Hansen 2003, p. 217). However, in the examples which we now present and analyse in detail, this virtualisation on the part of the viewer-participant is a matter of engagement with the mediated via embodied interaction (not merely a digital facial portrait). Embodied interaction catalyses new affects and affective relations “that virtualise contracted habits and rhythms of the body”. (Hansen 2003, pp. 217– 218). This catalysation needs to be understood as helping create meaning, not simply a vacant state of affect or naı̈ve experience (Hansen 2004b). 3 Analysis 3.1 Two mixed reality installations The two responsive digital video installations, Men That Fall and Women That Turn cross domains of the private and the public. They feature video of individual figures, for example, in their homes, en route to and from work, or in open public spaces. The works entail viewers in these spatial representations and play with a sense of private and public corporeal boundaries. As participants, audiences’ actions are transformed through experiencing the strange effects of their own movement and further possible interaction. The works have been exhibited in a variety of venues, in art galleries, shopping malls, libraries, stations and have been placed in street-facing windows. The works manifest a mix of realities: they are digital video representations of staged, theatrical events that involve sets and costumes in the material and physical world; they are digitally processed media files that are programmed to mimic hyper-realist representations; viewers encounter the works and, as we show, participate in various physical settings in activating, experiencing and completing the pieces. The works highlight the hyperreal by presenting unnatural movements that appear to be natural but as a result accentuate the human figure and its movements as opposed to a mechanical marionette. As a consequence the viewer experiences a sense of estrangement, such as in Artaud’s conceptualisation of the theatrical (1958) and audience participation in a transformative experience, along with one with familiarity of recognising a moving, full human form. The viewer as kinetic participant prompts movement in the digital figures. These movements are a means of revealing what lies behind the ‘masked’. In turn, the participant may then read emotional values into the resulting, constructed marionette-like actions of the figures. So too may they reflect on the potential significations of their own embodied collusion in producing the results played out before them. 3.2 Men That Fall 3.2.1 Masculinity deflated Men That Fall (2004 – 2006) encourages the viewer to question matters of gender and masculinity through their own relation to pictured male subjects. The writings of Susan Faludi, especially her book Stiffed: the betrayal of the American man (Faludi 1999) provided some of the stimulus behind this work. Stereotypes of male reserved distance and fixedness are visualised in the staged almost full-size imagery of 9 men in different situations: a businessman in a snowy field, Santa Claus with a crow bar, a policeman with an orange, etc. Ultrasonic range sensors that react to the viewer’s distance to the screen fire off three short video sequences. The distance and stiffness of the characters is easily deflated by the viewer getting too close and too intimate. What is actualised is a relation of the repelling forces of bodies, integrity and frailty, presence and stiffness. The work is a comment on free falling masculinity. Figure 2. Screenshot from the installation Men That Fall showing three of the nine male characters, each in the process of falling backwards. 9 Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Disquiet in the plasma Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Morrison et al. In the first sequence, facing the viewer is a man standing up straight up. When the viewer takes a step or two towards the screen, this movement instigates a second sequence in which the man then looks at the viewer. A third sequence is activated when the viewer takes another step towards the screen. In effect this crosses a line: when the viewer gets too close, the man falls down. A border to intimacy is suggested. As a result, the image of upstanding, immobile masculine authority crumbles through the presence and movement of the real, human body of the participant-viewer. When the man falls, he maintains his style and his posture. He falls stiffly, reminding one defencelessly on a variety of surfaces. Their loss of power, the sense of potential pain and damage and yet the power of the viewer to fell them, produces a sense of accomplishment but also, potentially, some disquiet. This is achieved through loops that result in the display of cycles of different men, all falling in the same fashion. Each patterning of this is dependent on the participant’s actions. Disquiet arrives as one engages with one’s own proximity to the unnatural but apparently similar fallings. Mediation thus becomes the interplay of the participant’s bodily movements and those seen, experienced and anticipated. The viewer-participant is confronted with the 3.2.2 Proximity, loops and activity Figure 3. Showing a sequence from Men That Fall where one of the male characters (1) looks down in a loop (2) reacts to the viewer/trespasser taking a step forward by looking up and (3) falls down when the viewer gets even closer. of a tree being felled. No facial expressions are seen nor does he make any attempt to prevent himself being ‘cut down’. The men are presented in various attires, yet the same action of falling reduces them to a similar defencelessness. After several iterations of the work, this is mixed with a sense of impotence, perhaps even reservation. These male figures, clad in carefully chosen archetypical costumes, are reduced to a common humanity as they lie 10 effects of their actions and their collusion in effecting their own disquiet. Men That Fall delivers a set of men that stand in front of the viewer, looking down to the ground whilst the viewer ‘crosses the line’. When waiting, a loop of the man seamlessly shifts to another sequence where the man looks at the ‘trespasser’. There is a sort of stand-off at this point – if the viewer chooses to go away, the man looks down at the ground again. If the viewer chooses to go even closer, the man falls down. Then a video loop of a waiting man marks a shift to another segment where there is a short dolly movement towards the man, who then falls straight down. This is quite a violent action, performed right in front of the viewer who has inflicted this action on the man by taking one step-to-many towards the screen. The effect is convincing; a dramatic moment occurs, expressed in real-time by the action and reaction of the viewer. This setup of short movies expresses the same thing: a small but very distinct change in the state of a fictional character. In terms of forward by their very physical presence, there is a possibility of empathy or even identification with the falling man. Single-point actions can be invoked as something that conveys greater meaning and do not need to be a part of a larger construct or narratives and background information; nor is there any need for complexity in order to make what appears on the screen interesting. Men That Fall was first installed in a shopping mall as a part of a larger artistic event with the ambition to bring different cultural artefacts and performances to places that are not usually associated with them. As a physical installation the work mimicked the information system 3.2.3 Contexts of display Figure 4. Photo showing the installation of Men That Fall at Bergvik’s shopping mall, Karlstad, Sweden, December 2004. The shopping mall as a backdrop for responsive video was examined during the frenzy of Christmas commerce. empathy, as conventional movies these events are hardly convincing - too little is happening and at too small level of complexity to identity with the situation of the falling man. As the viewer him/herself follows the shift from one state to the other while pushing ‘the story’ present at the wider site. The main difference between the screen of Men That Fall and the other more conventional displays (commercial information as well as life-style content such as music videos and advertisement videos) was the placement of the plasma screen. It was set 11 Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Disquiet in the plasma Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Morrison et al. up in the only spot in the whole shopping mall that had a constant flux of passers-by – near the emergency exit. In addition, the screen was hung on the wall in portrait format as opposed to the landscape format prevalent in most other sites in the mall. A small section of the floor in front of the screen was ‘sealed off’ with black-yellow tape. This gave the public a sign that one should think once or twice about stepping over the line. Otherwise there were no markers, nor any information about the piece being any different than other screens at the venue. In the mall setting, visitors approached the screen over and over again – commenting on the characters-on-screen and personifying them even in cases where malfunctioning electronics made the behaviour of the sensors non-coherent and viewers accounted for this as personal traits of the fictional characters. Altogether there were interesting examples of identification with characters at a meta-level. In one 3.3 instance a group of young girls in their teens rushed towards the screen repeatedly, while another group of elderly men watched them from a distance, laughing smugly: men watching girls that make men fall. Men That Fall is a study in movement— on the screen as well as the front of the screen. As a mixed reality piece, it is partly film, partly installation. All in all, it is a series of men, impressive while they stand, then without power as they fall, and, finally, very human as they lie in state. We know they are alive — we have seen them move — but it is we who have changed their state and status. This shift in materiality leaves us with questions about our new power to manipulate the plasma screen installation further. We find that when we repeat these actions, a repetition in reaction is achieved so that we too are now bound within both plasmatic loops of motivation, enactment and interpretation (ones that are both screen mediated and grounded in our bodies). Women That Turn 3.3.1 At a distance Figure 5. Screenshots showing three of the female characters in Women That Turn, with varied public space settings. Seen as objects, women are ‘tied down’ in a set of fantasies about how they should present 12 themselves, In Courshene’s Portrait One the main female figure looks away at the close of each simulated dialogue (Popper 2007, p. 288). She shifts from her conversational self to one that is more contemplative, pausing the enactment of the viewer holding the mouse through her gaze, at once distracted, disinterested, watchful and waiting. These moods thus create a sense of the user’s own reflections on this reflective piece. There is a strong sense of the converse to the bright and appealing smiles of commercial advertisements. In Women That Turn, a series of female figures distance themselves from the viewer who walks in too close to the plasma screen video representation. In this way, each woman escapes being the canvas for the controlling male glance that has been institutionalised in both men and women. The woman who has been seen, who you have seen, turns away from you; her back stares back at you, accentuated with a twitch. If you move in too close, you are caught in this response. The piece is a study in the shift between twitches and movement; it is an illustration of women that sense our glance and react with small movements of discomfort. At this point, there is no turning around; there is no act of symbolic meaning facing you. Women That Turn is the continuation of the installation performed at the shopping mall in Men That Fall. It was a part of a commissioned art and technology project for an exhibition in a larger gallery in the countryside in Sweden (Avesta Art/Verket). This is a monumental scale gallery space that is lodged in the historical setting of an 18th century blast furnace. In this exhibition context, there was no means of disseminating the video pieces in a seamless or transparent way, as in the case of Men That Fall in the mall. Instead, there was a need to make a firm and clear mark in the surroundings, articulating a claim for the very existence of the work on site. The video installations were not to be mounted on the walls, but on the contrary stood as distinct objects on the floor, in the middle of the large hall. The significance of such objects is stated in terms of sculptural values, a design statement of sorts carried by the object in itself. The content on the screen and the ‘thing’ surrounding the screen itself is equally important and is filled with significance. A screen hanging on the wall is more a discrete object than one that is part of a cabinet standing in the middle of the room. The slick, monumental values of ‘black boxes’ resonate with architectural or fine arts values. They deviate attention away from the fictional film characters on the screen. There was clearly a need for some kind of a construction to carry this out. What was chosen was that of the flight case. This is a convention within the AV-industry for transporting PA-systems, monitors and similar equipment. Subsequently, the 42-inch screens were built into large flight cases with aluminium edges and handles, equipped with wheels and tailormade solutions for lids and encapsulation. The overall statement of such an object was that it was merely a visitor and a highly transportable object whose sole purpose is to be on tour. 3.3.2 On your turn . . . In this specific setting, Men That Fall was included alongside Women That Turn. Yet, the syntax of the Women That Turn work was clearly different to that of Men That Fall. A set of female characters awaits the viewer with their backs turned against him/her. The women react with even more distance when the viewer comes too close. A video loop showing a woman standing firmly shifts seamlessly to another segment where she clearly demonstrates her discomfort at the presence of a viewer. This is achieved without her uttering a single word. She simply signals her disquiet by making a shift in her posture and without looking back. The woman is then ‘locked’ in that state. 13 Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Disquiet in the plasma Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Morrison et al. Figure 6. Photo showing (from left to right) the installation at the gallery Avesta Art/Verket, Sweden, Men That Fall, Delay Mirror and Women That Turn. The 42-inch screens were built into tailor-made flight cases. Sections marked on the floor were developed with a light designer to demarcate the zone for engagement. Viewers who have already experienced Men That Fall might experience an anti-climactic conclusion in that nothing else happens as long as they stand there and wait. Yet, here there lies a difference. In Women That Turn, the very moment the viewer decides to walk away from the screen, the woman on the screen reacts. The video seamlessly shifts into another segment where the camera makes a small dolly movement backwards and the woman turns around, but in an odd fashion as if someone (or something) is turning her around rather than she has done this voluntarily. Without lifting her feet from the ground the woman faces the viewer when s/he walks away. Her body expresses a certain wooziness yet tautness; her face expresses a kind of judgment of you as viewer. Finally, as in the case of Men That Fall, there is a shift into the next character in the suite. Figure 7. A sequence (from left to right) from Women That Turn showing one of the female characters as she (1) awaits the viewer with her back turned (2) reacts with greater distance when the viewer comes too close and (3) reacts to the viewer’s moving away from the screen by turning around while the camera performs a small dolly movement backwards. 14 Disquiet in the plasma Downloaded by [Stockholm University Library] at 05:15 19 February 2015 In personal relationships viewers are likely to have encountered similar twists and turns in reaction and responsibility. A partner’s back is turned against you until the very moment you walk away. At that point, they turn around and confront or blame you. These two works engage viewers in such scenarios but non-verbally. Women That Turn and Men That Fall express small shifts in the behaviours of fictional characters and where viewers of the exhibition also start silently debating their own behaviours or perhaps discuss the scenarios with other viewers, partners or friends. As video installations, importantly, these works also wait for a certain dialogue to occur. While viewers who approached Women That Turn might expect something spectacular to happen, as takes place in Men That Fall, they 4 in fact conclude the exchange when they walk away. As such they become counterparts in an evolving ‘mini-plot’. Being the estranged and also embodied ‘other’ may make you feel as if you are the only person present. No personal traits, no ‘cookies’, no personal tracking and no RFID tags are provided that would that make the ‘experience’ match your preferences and past behaviour. The involvement is equally non-personal as is the involvement of the camera in the cinematic convention of the vast majority of films. As opposed to the audiovisual spectacle of video gaming where users mostly identify themselves with the point-of-view of the game (as in so-called first person shooters), the viewer as the other forgets about himself/ herself and reacts to what happens to/with the fictional characters on the screens. Conclusions Figure 8. The two works by Performing Pictures at Balta nakts, Galerija Centrs, Riga Old Town, Latvia. 15 Digital Creativity, Vol. 20, Nos. 1 – 2 3.3.3 Fictions of dialogue Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Morrison et al. Figure 9. Men That Fall. Gallery setting, Digital art and culture, Konstfack, Stockholm, Sweden. Figure 10. The work ‘on display’, public concourse, PUB, Stockholm, Sweden. 16 4.1 Proximal relations Shifts between modes of representation, reaction, responsivity and reflection are what mark these two works — Men That Fall and Women That Turn — as mixed reality art installations. The works inscribe our experiences and expectations of video as time-based. While they clearly operate as affective pieces, they also challenge us to think about spatial relations between the people screened, to reconsider proximity, to reassess our own senses of the private and the public, and our internalised and external comfort zones and their violations. To return to the several sense of plasma outlined earlier, in mixed reality arts, and in the two examples given here, tensions between the naturalistic and the uncanny meet one another in the mediation of a ‘plasmatic materiality’. Men That Fall and Women That Turn aim to achieve relatively simple and direct means of interaction. In these works, the interaction model was constructed around two separate, yet similar sensor-based arrangements. Locus, modus, status and behaviour are terms that we have applied in reflecting on the works and the activities of their production. In terms of film, the installation-based explorations were made in the field of parallel film montage and real-time interaction with video footage in relations to gesture and characterisation. The works are part of on-going attempts to establish personal syntaxes for responsiveness in films in relation to specific places. Concerning locus we have found that the object in itself has great significance; displaying Men That Fall in a shopping mall, the approach of a discrete object with a certain DIY (do-it-yourself) approach correlated with the surroundings. An apparently over slickly designed artefact would have lost its charm in such a locus. When framed as a console in the shape of a highly transportable flight case, the object carrying these films took on an entirely different meaning. It became a video sculpture at a gallery that hosts visitors. In both cases the status of the viewer is that he or she is standing, engaging with the work through backwards and forwards movement, as well as a wider process of walking to and fro in the mall. In contrast to the larger consumer-driven activity of the public mall, the impact of falling men and turning women is direct - it requires only a small set of inputs, the viewerparticipant now moving in closer, now stepping back, perhaps several times and in different patternings and then moving away. Proximal relations are thus central to these works and to the achievement of embodied, affective engagement. 4.2 The disquiet is ours A plasmatic aesthetics may be said to be polysemous, that is technically, experientially and interpretatively. In the two works we have presented and discussed here, our attention has been on full body portraits. In these pieces, the supplementary analogical connection to the digital is realised through our own embodied movement as viewer-participants. Disquiet in the plasma flickers, here in the gaseous screen, there shifting around our own corporeal selves as we relate to it and conduct it once again back towards the figure sculpturally facing us. The character that is waiting for us, that seems to be present, represented, but silent, is one of a similar set. So it is that we encounter a figure that moves because we do, a figure that in a sense moves us into a mixed reality relation where the disquiet is ours. 17 Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Disquiet in the plasma Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Morrison et al. Figure 11. Men That Fall and Women That Turn on display in shop windows, Stockholm, Sweden, November 2006. Acknowledgements This article is an outcome of an art and research project, GRIG (Guild for Reality Integrators and Generators), funded by the EU Culture 2000 Programme. The article is the result of ongoing collaboration between two project partners: the Interactive Institute and InterMedia, (Communication Design Research Group), University of Oslo, Norway. For more information on GRIG: http://www.intermedia.uio.no/display/Im2/grig; see also on processes in the project http://www.intermedia.uio.no/display/grigsite/Process, Accessed 1 September 2008. The art works Men That Fall and Women That Turn were created as part of a wider artistic research platform called Performing Pictures at the Interactive Institute, Stockholm, Sweden. For details: http://www.tii.se/ and www.performingpictures.se. This was partly also their contribution to the EU Culture 2000 project WAVES and the project Cinésense funded by the Swedish Science Council, and exhibited and researched as part of GRIG. 2 3 4 5 6 7 8 Notes 1 The term ‘mixed reality’ is chosen in preference to augmented reality or virtual reality. It sits on a 18 larger continuum of im/materiality (Krüger 1993, Milgram and Koshino 1994). From being sites for mediating television and computer content they may be changed to display famous art works downloaded from the web, ones that may in turn be replaced, in our homes (http://www.vat19.com/dvds/ ambart.cfm?ADID ¼ AMBARTLCD1&gclid ¼ CLm3rNX33pICFQs XQgod9kBo-Q) and in public spaces such as the Eurostar departure lounge at the new St Pancreas station in London (http://news.bbc.co.uk/2/hi/ uk_news/england/london/7360314.stm (Both accessed 1 September 2008). This is to coin a term applied within reconstructive and aesthetic surgery. He inverts arguments originally advanced by Deleuze and Guattari (1987) to do with the ‘faciality machine’, a mechanism by which the embodied is encapsulated in the face. We concur with Hansen (2004b) who counters a critique of his approach to affectivity by Rushton (2004) who argues it merely perpetuates the very capitalist erasure of critique that Deleuze and Guattari aimed to resist in their analyses. http://www.din.umontreal.ca/courchesne/, accessed 1 September 2008. http://www.flong.com/projects/reface/, accessed 1 September 2008. Artist’s website: www.lincolnschatz.com, accessed 1 September 2008; Schatz featured in Rhizome: http://rhizome.org/ profile.php?1002208, accessed 1 September 9 10 2008; featured in the New York Times (14 October 2007): http://www.nytimes.com/ 2007/10/14/fashion/14noti.html?n¼Top/ Reference/Times%20Topics/Subjects/C/ Cameras&_r¼2&adxnnl¼ 1&oref¼slogin& adxnnlx¼1210057879-WBa/0UtpOTbXudji Lv2xjQ, accessed 1 September 2008. http://www.lincolnschatz.com/work/present/ 05/acrosstime.php, accessed 1 September 2008. http://www.lincolnschatz.com/work/present/ 07/cube_main.php, accessed 1 September 2008. References Artaud, A. (1958) The theatre and its double, Grove Weidenfeld, New York. Translated from the French by Mary Caroline Richards. Bolter, J. and Gromala, D. (2003) Windows and mirrors: interaction design, digital art, and the myth of transparency, The MIT Press, Cambridge, MA. Brouwer, J., Fauconnier, S., Mulder, A. and Nigten, A. (eds) (2005) aRt&D: research and development in art, V2_Publishing/NAI Publishers, Rotterdam. Deleuze, G. and Guattari, F. (1987) A thousand plateaus, University of Minneapolis Press, Minneapolis. De Oliveira, N. and Oxley, N. (2004) Installation art in the new millennium, Thames and Hudson, London. Dourish, P. (2001) Embodied interaction, The MIT Press, Cambridge, MA. Faludi, S. (1999) Stiffed: the betrayal of the American man, William Morrow and Company, New York. Fishwick, P., Diehl, S., Prophet, J. and Löwgren, J. (2005) ‘Perspectives on aesthetic computing’, Leonardo, vol. 38, no. 1, pp.133–141, accessed 1 September 2008, http://www.mitpressjournals.org/doi/abs/10.1162/0024094053722372 Fuller, M. (2005) Media ecologies: materialist energies in art and technoculture, The MIT Press, Cambridge, MA. Fuller, M. (2008) Software studies: a lexicon, The MIT Press, Cambridge, MA. Cinema/art/narrative, eds M. Rieser and A. Zapp, BFI, London/ZKM, Karslrue, pp.105–119. Hales, C. (2005) ‘Cinematic interaction: from kinoautomat to cause and effect’, Digital Creativity, vol. 16, no. 1, pp. 54–64. Hansen, M. (2003) ‘Affect as medium, or the “digital-facial-image”’, Journal of Visual Culture, vol. 2, no. 2, pp. 205–228. Hansen, M. (2004a) New philosophy for new media, The MIT Press, Cambridge, MA. Hansen, M. (2004b) ‘Communication as interface or information exchange? A reply to Richard Rushton’, Journal of Visual Culture, vol. 3, no. 3, pp. 359–366. Huhtamo, E. (2007) ‘Twin-touch-test-redux: media archaelogical approach to art, interactivity and tactility’ in MediaArtHistories, ed. W. Grau, The MIT Press, Cambridge MA, pp. 71– 101. Krüger, M. (1993) ‘Environmental technology: making the real world virtual’, Communications of the ACM, vol. 36, no. 7, pp. 36–51. Leggett, M. (2007) ‘Generative systems and the cinematic spaces of film and installation art’, Leonardo, vol. 40, no. 2, pp. 123–128, accessed 1 September 2008, http://www.mitpressjournals. org/doi/abs/10.1162/leon.2007.40.2.123 Manovich, L. (2001) The language of new media, The MIT Press, Cambridge, MA. Manovich, L. (2003) ‘The poetics of augmented space’ in New media: theories and practices of digital textuality, eds A. Everett and J. Caldwell, Routledge, New York, pp. 75 –92. Martin, S. (2006) Video art, Taschen, Köln. Mignonneau, L. and Sommerer, C. (2003) ‘From the poesy of programming to research as an art form’ in CODE: the language of our time, eds G. Stocker and C. Schöpf, Hatje Cantz Verlag, OsterfildernRuit, pp. 42 –249. Milgram, P. and Koshino, F. (1994) ‘Augmented reality: a class of displays on the reality-virtuality continuum’, SPIE, telemanipulator and telepresence technologies, vol. 2352, no. 34, pp. 42 –48. Grau, W. (2007) (ed.) MediaArtHistories, The MIT Press, Cambridge, MA. Morrison, A. Sem, I. and Havnør, M. (in press) ‘Behind the wallpaper: performativity in mixed reality arts’ in Inside multimodal composition, ed. A. Morrison, Hampton Press, Cresskill, NJ. Hales, C. (2001) ‘New paradigms ¼ new movies: interactive film and new narrative interfaces’ in Paul, C. (2003) Digital art, Thames and Hudson, London. 19 Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Disquiet in the plasma Digital Creativity, Vol. 20, Nos. 1 – 2 Downloaded by [Stockholm University Library] at 05:15 19 February 2015 Morrison et al. Popper, F. (2007) From technological to virtual art, The MIT Press, Cambridge, MA. Reas, C. (2006) ‘Process/drawing’ in ‘Programming cultures: art and architecture in the age of software’, Architectural Digest, vol. 76, no. 4, pp. 26–33. Redström, J. (2008) ‘Tangled interaction: on the expressiveness of tangible user interfaces’, TOCHI: ACM Transactions on Computer-Human Interaction, vol. 15, no. 4, pp. 1–17. Reiser, M. and Zapp, A. (eds) (2002) New screen media, BFI, London. Rushton, R. (2004) ‘Response to Mark B.N. Hansen’s ‘Affect as medium, or the “digital-facialimage”‘’, Journal of Visual Culture, vol. 3, no. 3, pp. 353 –357. Shaw, J. (2003) ‘Introduction’ in Future cinema. The cinematic imaginary after film, eds J. Shaw and P. Weibel, ZKM, Karlsruhe, The MIT Press, Cambridge, MA, pp. 19–27. project leader for InterMedia’s contributions to the EU Culture 2000 funded project into mixed reality arts called GRIG (Guild for Reality Integrators and Generators). Geska Helena Andersson is an artist with a background in film and theatre. She heads the Performing Pictures platform (www. performingpictures.se) within the Interactive Institute (www.tii.se) – a Swedish experimental IT-research institute that challenges traditional perspectives through combining art, design and technology. She is project leader for the institute’s contributions to the EU Culture 2000 project GRIG and manages the Cinesense-project funded by the Swedish Science Council. Zielinski, S. (2006) Deep time of the media: towards an archaeology of hearing and seeing by technical means, trans. G. Custance, The MIT Press, Cambridge, MA. Robert Brečević is an artist and a film-maker who works with enhanced video as part of Performing Pictures of the Interactive Institute. His work facilitates research within the field of public spaces and enhanced media. Prior work includes commissioned scriptwriting at SF (Svensk Filmindustri), production of independent short films as well as commissioned feature film production and screenwriting by the Swedish Film Institute. Together with Geska Helena Andersson he is working with the GRIG and Cinesense-projects. Andrew Morrison is an Associate Professor at InterMedia, an interdisciplinary research centre at the University of Oslo, where he leads the Communication Design research group (www.intermedia.uio.no). He is also a Visiting Professor in Interaction Design at the Oslo School of Architecture and Design (www.aho.no). He is Synne Skjulstad has a doctorate in Media and Communication Studies from the University of Oslo on textual analysis of designers’ websites and is taking a related post-doc there on digital advertising, the web and mediation. She has published on advertising, new media arts and communication design amongst other topics. Spielmann, Y. (2008) Video: the reflexive medium, The MIT Press, Cambridge; MA. Udsen, L. and Jørgensen, A. (2005) ‘The aesthetic turn: unravelling recent aesthetic approaches to human-computer interaction’, Digital Creativity, vol. 16, no. 4, pp. 205 –216. Welsh, J. (2003) The transit zone, Kunsthøgskolen Bergen, Bergen. 20