Island Design Camps—Interactive Video Projections as Extended Realities

Abstract

Over the course of seven years during ten events, the author explored real-time interactive audiovisual projections, using ad hoc and portable projections and audio systems. This was done in the specific location of Cockatoo Island in the waters of a part of Sydney Harbour, Australia. The island offers a unique combination of the remnants of a shipyard industrial precinct, other buildings, and increasingly restored natural environment. The project explored real-time audiovisual responses through projected overlays reminiscing the rich history and past events, interactively resonating with the current landscape and built environment. This included the maritime industrial history, as well as other historical layers such as convict barracks, school, and the significance of the location for Australia’s original inhabitants before colonisation by the British started in 1788. But most prominently, the recent use of the island for large scale art projects (such as the Outpost street art festival in 2011, and over a decade of use as part of the Sydney Biennale of Art, and the use of the island for film sets). This was a rich source of image material collected by the author and used to extend and reflect on current realities. By using the projections, overlaying and extending the present reality with historical data in the form of sounds and video, dialogues were facilitated and a conflation of past and present explored. The main activity were the VideoWalks, where the author, using a custom built portable audiovisual projection system and a bank of audiovisual material was able to re-place sound and video of previous events in the present context, in some instances whilst delivering a performative lecture on the way. The explorations are part of the author’s Traces project, exploring traces and remnants of past events and how these can inform design approaches. The project over the years also developed an element of recursion, by using footage of an earlier projection into the current, the footage of which was then used in the next event, and so on—up to five layers of extended reality.

1. Introduction: Audiovisual Projections as Extended Realities

The research presented in this paper investigates audience interactions with audiovisual projections. The main research question is how audiences can be involved in interactive media installations and performances. This is done through the research methodology of creative exploration and artistic expression [1]. This methodology includes creating interactive artworks with the specific aim to enable and facilitate the study of audience interactions. This research methodology also encompasses the study and exploration of existing interactive audiovisual artworks and immersive installations in exhibitions such as teamLab from Japan (at the Powerhouse in Sydney, Australia in 2018) [2] and Rafael Lozano-Hemmer (at the Museum for Contemporary Art (MCA) in Sydney in 2012) [3], or the ZKM Media Museum in Karlsruhe, Germany and the Ars Electronica Center in Linz, Austria, and many others [4]. These works are often immersive, extended realities but usually not with Head-Mounted Displays (HMDs), more commonly through using projections.

In the author’s own explorations, solo or in collaboration with others (musicians, visual artists, architects), audience interactions with audiovisual projections and displays are studied with an emphasis on physical interaction, using a range of sensors which are very sensitive and picking up motion, pressure, and other movement-based parameters [5,6].

The work presented in the current paper builds on this work, as well as on the projects, theory and reflections presented in the companion paper Exploring Extended Realities in Environmental Artistic Expression through Interactive Video Projections [7] in this Special Issue on Augmented Reality (AR), Virtual Reality (VR), and Human–Computer Interaction (HCI).

1.1. Extended Reality Spectrum

In the Exploring Extended Realities paper [7] examples of Extended Realities are discussed not so much from Computer Science point of view, but from artistic expressions on spatial scales, such as Landscape Art, public art in urban environments, Street Art (graffiti), electronic music, live video performances, and interactive art. In that paper, an Extended Reality Spectrum is defined, to describe environments from the real world, to augmented (AR), mixed realities (MxR) and the completely virtual (VR) [7]. This is a common way of looking at continuum from reality to virtuality, as also discussed in the paper by Memmesheimer and Ebert in this Special Issue [8], who extend the continuum with two more dimensions, one axis for display size (HMD versus Hand-Held Displays (HHD), and one axis for the number of collaborators. Their ‘scalable extended reality’ (XR^s) model can be further extended, by including video-projections from the small scale (starting from where the HHD size ends) to the architectural scale, as in the examples presented in the Exploring Extended Realities paper [7], and even clearer in the studies presented in this paper about the Island Projections. The third axis of Memmesheimer and Ebert’s model, about the number of included collaborators or participants, is also applicable on the audiovisual projection projects presented by the author, as they always include an audience, who in many cases can interact with the mixed realities.

1.2. Interactive AudioVisual Projections

Some of the experiments with the interactive audiovisual projections were more structured and designed to collect data to quantitatively represent audience participations and appraisals, while others were more incidental, improvised, and ad hoc, allowing for exploration and serendipity. The project presented in this paper is an example of the latter, but the unique characteristic is that all explorations took place in the same location during ten three-day events spanning seven years: the Design Camps on Cockatoo Island in Sydney Harbour, some examples can be seen in Figure 1.

Although not intended to collect quantitative data, the advantage of such a longitudinal study is that experiments could be repeated, and explorations extended at each iteration, in some cases encapsulating previous experiments recursively. The outcomes are the experiences, and new insights through the pan-historical re-situations, juxtapositions, re-presentations. The audiences would vary, each year there were different students, and while there was variation in the staff that attended many would be involved over several years.

Audiences were specifically invited to participate and explore the interactive video projections, which were designed (as in other projects [5,7]) to allow and accommodate people’s input, enabling participants to co-create the visual responses through the range of sensors and interfaces provided, resulting in collaboratively produced unique outputs.

With the VideoWalker interactive portable audiovisual projection instrument and system [6,7], the author investigated the re-presentation of past events of this particular location.

1.3. Island Projections as Extended Realities

In this paper the focus is on the various audiovisual projections developed and undertaken by the author over a span of seven years in one location. This encompassed fixed interactive projections in specific ‘projector-genic’ locations and settings, as well as performative, mobile projections. Particularly the latter would have a spatio-narrative structure which had the aim to relate, resonate, and reconnect to the island and its many levels of history. The island became a canvas for these audiovisual projections.

One approach was to ‘re-situate’ past events and particularly artworks from past exhibitions on the island, an example of which is shown in Figure 2.

In order to base these activities solidly in the context, the background of the island needs to be understood at least to a level to make the projects relevant. Extending Reality is an endeavour that needs to be grounded and referencing the context, not replacing it or overpowering it with an audiovisual makeover. The reader is free to skip or skim in these historical background sections, and perhaps later return to it from the perspective of what has been added temporarily in the events of the projections.

The explorations took place as part of the Design Camps of the UTS School of Design, a three-day event where all first-year design students camped on the island, and the participating design teaching staff in residence, which allowed all participants to immerse themselves and completely focus on creating design interventions in the environment. In total ten Design Camps took place, over seven years. The author participated in all of them, as a side project to the scheduled teaching and workshop activities, experimenting with a range of public video projections: through a custom made portable projection set up, the VideoWalker, and through large scale stationary projections (often interactive too).

The author also participated on several occasions in the yearly situated architectural design event, Urban Islands [9].

Each Design Camp event offered opportunities for several ad hoc audiovisual installations (such as projecting on architectural structures, particularly the iconic water tower of the island, visible from the mainland), and a range of audience interaction techniques were explored.

The Island Extended Realities project draws on, and relates to, earlier and concurrent projects with large scale video projections by the author and team, in particular the interactive kaleidoscopic projections (the Facets pieces) [5] and the mobile projections (Video Walker events) [6], as presented in the paper Exploring Extended Realities in Environmental Artistic Expression through Interactive Video Projections in this Special Issue [7].

2. History of Cockatoo Island

In order to fully appreciate the effects of the projections, re-situating and re-contextualising image material of past events in this location, it is important to know a bit about the rich history of the island. In this section this location is presented.

Cockatoo Island is the largest island in Sydney Harbour, situated where the two rivers (Paramatta River and Lane Cove River) meet, as shown in the aerial view (from a plane) of the harbour in Figure 3.

The island always has had a cultural significance to the original inhabitants of the area, the Gadigal people of the Eora nation. Today it is mostly known for its industrial past as one of the largest shipyards in the country, and (like many sites in early Sydney) as a jail for convicts who helped developing the site through forced labour. Since the decommissioning of the shipyard in 1991, and the removal most of the ship building infrastructure such as heavy machinery and buildings, there was a period of neglect but eventually after the establishment of the Sydney Harbour Federation Trust in 2001 the island was redeveloped as a public space [10]. The Trust was formed after pressure from the community driven Headland Preservation Group, which was founded in 1996, and the Trust has since been involved in developing and maintaining a range of industrial and convict historical sites, nature sites, as well as preserving indigenous sites.

The history of Cockatoo Island was often described with a strong focus on developments and activities of the British colonisation of the Sydney area which started in 1788, and until recently often neglecting the pre-colonial significance. (In a book particularly about the history of the dockyard in the late 1990s the first chapter starts with “There is no evidence of Aboriginal use of Cockatoo Island” [11] (p. 1) Sites of cultural or practical significance to the Aboriginal people on the headlands in the area were often treated with disregard (nearby rock engravings of thousands of years old, now revered and protected, show marks of colonial structures that were build right on top of them), and the heavy use of the island since early colonial times would have erased any trace of Aboriginal presence, furthermore the lack of a water source is often mentioned as an indication that it wasn’t used for permanent inhabitation. However we now know that it certainly had a cultural significance to the local people, who called it Wa-rea-mah or warayama in the local Dharuk language, and it was used by various clans of the peoples of the Eora nation. The name refers to its role as a ‘women’s place’, thought to be used for ceremonial and cultural activities, but other cultural activities may also have occurred. In recent documentation and online resources about the island the Aboriginal history and cultural aspect are prominently represented [12].

The first occupation by the British colonists started in the early 1840s, when Governor Gipps chose the site as a jail to house twice-convicted criminals when it was no longer possible to keep them on the distant Norfolk Island [13] (pp. 513–514). As elsewhere in the Sydney region, the convicts were used as cheap labourers quarrying sandstone and building houses and barracks, and the first dry dock on the island, the Fitzroy Dock which opened in 1857 [11] (p. 9) (it is still present, in expanded form). Later the site was also used as a girls’ schools until 1913 (such as the Biloela Reformatory and Industrial School, for destitute and orphaned girls). In that period the island was known as “Biloela”, which means white cockatoo in an Aboriginal language (not Dharuk), in an attempt to remove the stigma of the name Cockatoo Island which had become connected to the convict prison practice.

2.1. The Island as Dock Yard

In the 20th century the island had an industrial purpose, increasingly developed as a large scale naval and commercial ship yard. During the peak of the activities around WWII the whole island was covered in buildings and machinery, some of which still remain. The original curved shape of the island was increasingly cleared by excavations and quarrying, for sandstone building materials and to make room for more buildings of the ship yard, and extended by reclaiming areas of the surrounding bay. This resulted in two levels, the low lying flat areas excavated and extended just above the water level, and the original top level of the hill, a plateau at about 18 m above the water level, reachable by stairs in two locations and a sloped, curved road on the north side of the island. Two tunnels were dug through the hill at the lower level, one straight one connecting the north and south aprons, and one with a kink (the ‘dog-leg tunnel’) connecting the eastern and the southern area.

The last managing director and chief executive John Jeremy had the task of organising the clearing of the majority of the heavy machinery and dismantling a number of the buildings after the closure of the dockyard in 1991. Some buildings were retained, particularly the less utilitarian ones that had historical and architectural significance, and some of the machinery selected for its value as industrial heritage [11] (p. 69), [14]. The rest was sold, often as scrap metal. A group of volunteers has a permanent machine workshop on the island, and the process of restoring the old machinery is still ongoing—including the recent completion of a steam driven crane which on some occasions is on public display. Some of the larger machines and mechanical structures have inherent sculptural qualities, particularly a set of massive iron presses (which, due to their rusted surface, resemble Eduardo Chillida’s sculptures).

2.2. Wolverine Film Set

Over the next decade (1990s), the island was mostly abandoned, eventually increasingly used for special events (music festivals, art events, a yoga event, motor cross events (!) and as film set—several scenes of the X-Men Origins: Wolverine movie were shot here in 2008. These scenes were all revolving around the personage of Stryker who used this as his ‘base’—acts such as the bonding of the Wolverine’s bones to metal, in the Powerhouse building at the far west point of the island, a fight between Wolverine and his brother, and the liberation of the mutants from their prisons in the area next to the massive space of the Turbine Shop. The walls made for this set look exactly like the real walls, one bit is still there but it is visually such a perfect simulacrum that one would need to touch it to reveal that it is not the real wall. The prominent remnants of this set were present until 2018, when almost everything was finally removed to make room for Ai Weiwei’s massive installation The Law of the Journey, a scaled up and elongated sculpture of a dingy with refugees, as part of the Sydney Biennale.

2.3. Recent Aboriginal Presence

In 2000–2001 the island was inhabited for several months by a group of Aboriginal people, who set up a ‘tent embassy’ on the island [15,16,17]. They claimed the island as ‘terra nullius’ (in reference to the false claim of the British colonists that the continent was uninhabited when they started settling on the unceded land since 1788), and that Captain Cook had not claimed the islands for Britain, only the mainland. Their case was brought before the High Court, but they did not win. It is widely known though that the lands were never ceded to the colonists.

They held a smoking ceremony as a cleansing ritual, to draw attention to the destruction and contamination of the site, and painted several buildings at the top level, such as the search light platform, and the east facing side of the timber drying building. These paintings are still mostly present, and now signposted for visitors to learn about this part of the Island’s history.

2.4. First Visits to the Island

After further development of the island by the Trust, the site was opened to the public in 2007, and has been accessible by regular ferry since 2008.

Since I moved to Sydney in early 2007, I lived in two different places in Birchgrove on the Balmain peninsula on the south side of the bay, with a direct view of the island (until the end of 2015). To emphasise the ever changing views of the island, even in its current form, a selection was made of my collection of sunset shots I took over time (part of an animation video) as shown in Figure 4.

A camp site was established on the apron on the north side, populated with pre-pitched tents with camp beds inside but people are also allowed to bring their own tents (for a fee). Several of the houses on the top level were developed for visitors to stay, enjoying the island overnight, and with spectacular views. Solar panels were mounted on the roof of the biggest building, the Turbine Shop, and rainwater capture facilities installed.

Since 2010 the island has UNESCO World Heritage status, however this is only for its convict past (together with a number of other sites around the country, mostly in the Sydney area and in Tasmania).

My first visit to the island was in August 2007, by invitation of my colleague, architect Tom Rivard who has been regularly organising the Urban Island two-week architecture event and workshop on the island [9]. This event presented international guest speakers and workshop leaders, and often former dockyard director John Jeremy (and myself in later years). At that time one needed to charter a private vessel to get to the island, which Tom had arranged to depart from the Woolwich dock north of the island. I checked later on the photographs that I took from the visit, that the Wolverine set mentioned above (and later in this article) and other structures were indeed not present at that time.

2.5. Biennale of Sydney and Other Art Festivals

Since 2008 the island has been used by the Sydney Biennale to present a range of large-scale and often site specific art works—however only during the months of the event. I would visit and capture the works on photo and video, and a lot of this footage was later used for the live projections I carried out at the island, particularly the site specific and monumental scale works. I would capture footage at every Biennale, including the last one held on the island in 2020 (due to the COVID pandemic it was shut down after 10 days). The latest one, the Sydney Biennale 2022, did not occupy the island; this is somewhat ironic as the theme is titled Rivus, it was all about flowing water.

A bit less prestigious but also presenting some great works, and often a source of useful video footage, were the Underbelly Arts Festival events that occasionally took place on the island (particularly the 2015 event).

A particularly rich source of inspiration as well as an opportunity to capture some extremely powerful footage was the Outpost—art from the streets festival in 2011 (from 4 November to 11 December). The whole island was covered in graffiti, murals, and sculptures, there were performances and video projections by a range of top international and local artists, such as ROA, Lister, Deb, Will Coles, Beastman, Ha Ha, Mini Graff, Numskull, and Vexta.

Some of the graffiti was painted on temporary structures such as large wooden panels, to protect the heritage walls and façades of the buildings, others were spraypainted straight onto the walls (and later removed or covered). Some of it still remains, little traces in the cracks here and there, or on abandoned objects and structures. Particularly the small concrete sculptures of Will Coles were resisting being eradicated, and we still find some remnants. Like everywhere else in Sydney, of course the Will Coles sculptures often get nicked, or removed (though the bigger pieces, such as a lounge chair and a washing machine, tend to resist this). Images and reflection on Coles’s work can be found in the XR Projections paper [7]. On the island, even when several motor cross events (by Red Bull) filled the eastern apron with a thick layer of mud and dirt, after this was removed some of the sculptures emerged, still stuck to features on the surface.

Other Will Coles sculptures were stuck to temporary structures that got entirely removed after the festival. This included some structures and buildings that were deemed redundant and moved away. Recently we rediscovered some of them on the south side of the island, large round concrete tanks over two metres high, the murals mostly painted over but some remnants of Coles’s sculptures still present. (See Section 7.3 for further elaboration.)

It seems that great efforts were made to remove all presence or even traces of the graffiti art that has been displayed on the island. This has to do not only with the heritage protection, but also the ongoing pressure of the Biennale—several people and other sources over the years have indicated that the Biennale organisers see the island as their territory, trying to monopolise it for their artistic presence. The successful Outpost festival was never repeated for various reasons, and other events struggled to create a presence. This is a shame, as at least some if not many of the works presented were of high artistic and conceptual standards, easily of the level of the Biennale itself (ironically, they dabbled in street art themselves, there are several examples in particularly 2008 and 2012 shows).

The main point is however, partially due to these pressures (competition, heritage protection) but mainly due to lack of vision about the role of the island, that nothing ever remained of all the fantastic works and events that took place on the island. It is like a drainpipe, the artistic water flows through but never sticks, after each event the site attempts to reset itself, endeavouring to erase all traces, and discouraging any resonance. This is common in the genre of course, often the ‘white cube’ gallery and every ‘black box’ theatre resets itself after each show, back to the neutral ‘blank canvas’ to be filled again with the next thing. Many galleries and museums however, build up collections. Yet, in the case of the island, nothing has been built up. This seems to be a missed opportunity, or rather a range of missed opportunities—while not all works would be suitable to remain (some would compete with other purposes of the site, or be too expensive to maintain, etc.). Especially the site-specific works of the Biennales, where I have often seen works in situ that were so well suited that it would have been most obvious if they remained. Also with the street art graffiti works at Outpost, it seems silly that so much effort was made to erase murals by painting over them or cleaning them off—some of those walls look worse than they did before, and certainly worse than when the murals were present.

By now, the island would have been one of the most exciting sculpture parks around, and home to many site-specific works with a worldwide reputation.

2.6. Current Situation on the Island

The island certainly isn’t desolate, there are always people around, and particularly on weekends and during events it gets very busy. I have visited Cockatoo Island many dozens of times (usually together with my son, a keen explorer who often initiates the visits), and we always find new things there, such as several geocaches recently.

There are not many permanent establishments on the island. There are two cafés, one at the public ferry stop as the main entrance, on the north-eastern point, and one on the south side (where the dedicated Biennale ferry often berths), another popular evening venue only lasted for a few years, but the sporadic (and somewhat heroic) attempts to establish shops (selling things such as design items, or kitchens (!)) never lasted long. The only lasting presence is that of a boat storage and maintenance facility on the south-east corner of the island, since a few years occupying the Sutherland Wharfs and Dock, perhaps somewhat fitting with its dockyard past but mainly concerned with recreational vessels (private moorings and berthing is in short supply in the bay).

Since 2014 an exhibition with photographs and stories from the dockyard past has been present in the Biloela House, a late 19th century building at the upper level on the east side of the island.

In the recent years developments seem to have stalled somewhat, of course also due to the COVID pandemic, when the island was closed to the public during Sydney lockdowns. But based on a public consultation in 2019 the Trust produced a Draft Concept Vision document [12] in which a very colourful and ‘vibrant’ future is envisioned, creating more green spaces and more activities on the island, although it remains to be seen if all of this is going to be an improvement, are more aimed at developing commercial interests.

At the end of 2022, Opera Australia staged Bizet’s Carmen on the Eastern Apron of the island (with fireworks).

3. Traces of Past Events

When engaging with an environment or particular location, I often focus on the traces, reminiscing the past events and presences [18]. Exploring traces and remnants of such past events can inform design approaches. An example is the ‘desire lines’, or ‘olifantenpaadjes’ in Dutch—users’ past behaviours which are revealed in the environment and can be used as design input.

Particularly due to its guerrilla nature, graffiti often leaves such traces. Artworks that are officially installed, are easier to un-install as part of the same organisational structures. This is one of the many reasons why graffiti artist Banksy is so important—he successfully attempted to reverse the process, by actually bringing his pieces into the gallery and (illegally) installing them there. Some lasted only a few hours, while others remained in place for days [19] (pp. 116–117).

Another interesting example of traces in graffiti practice, which I found at several prominent graffiti sites, where whole chunks that had come off the wall, pieces that showed the layers of paint accumulated over time, sometimes with dozens of ‘strata’ present.

After some of the events on Cockatoo Island it makes sense to remove the traces entirely, particular commercial ones such as the motor cross dirt mentioned, or the film sets. As discussed above, parts of the film set of Wolverine remained present, often not even noticed as such, so well they blended in with the environment. Until one touches the structure and realises that the brick wall is actually painted on cloth on a wooden frame, visually it was indistinguishable from the real walls. This posed a dilemma for the Trust, because by keeping these structures in place it meant that they also needed to maintain them, while they deteriorated over time because of their nature as temporary structures. Eventually signage was added to it in which the presence was explained (or excused, with the phrase “Parts of Stryker’s liar still remain in this area and have been absorbed back into Cockatoo Island’s rich patina of history and gritty remnant machinery”, accompanied by some images from the film, and a picture of Hugh Jackman posing in front of some of the machinery during the launch event of the movie).

More problematic and harder to erase are the traces of the Unbroken shoot, a movie directed by Angelina Jolie in late 2013, where the crew turned large areas of the island into a coal-stained winter island with artificial snow, a prison camp for British soldiers during WWII situated in Japan as accentuated with the many large scale signs in kanji. While a good source of income for the Trust, in personal conversations with the staff on the island at the time I heard how difficult it was to remove all the traces, particularly the coal stains and some buildings which I could see were hard to remove. Several of the large structures were retained as they provided much needed shade on the island, though they did not really fit in with the style of the island buildings. Eventually some of these structures were removed as they were inappropriate, but several are still there such as a fake ‘train station’ (though the fake clock tower on top has been removed, but only years later), and a wooden ship in some sort of oriental style on the western slipway, which looks completely anachronistic (wooden ships were never built or maintained on the Cockatoo Island dockyard).

Particularly for someone who knows the island well, the way it is used and depicted in these movies also contains an element of extended reality, partially due to the physical sets (the structures built for these movies, the addition of fake snow and black coal stains, etc.) and even more incongruous through CGI (Computer-Generated Imagery). Examples of such CGI ‘extended realities’ are the replacing of the suburb of Drummoyne on the nearby peninsula to the west of the island with snowy mountains in Wolverine (and the authentic steam-driven crane ‘moved’ to the other side of the dock to fit in the same frame), or superimposing an image of the nearby Woolwich Dock with its high cliff walls, on the slipway on the west side of the island in Unbroken for the scene about the end of WWII.

4. The UTS Design Camps

In 2011 the UTS (University of Technology Sydney) School of Design started with the Design Camps; students from all our design disciplines were working together on projects and events for three days on Cockatoo Island. It was part of the Design Studies stream in the School, which consists of several subjects (on design thinking, history, etc.) over the years in which students from all disciplines collaborate, such as product design, visual communication, interior design, and fashion and textiles. The Design Studies subjects (that I taught in, in addition to my teaching in product design and interaction design) were often a real eye opener for the students, working together with other disciplines. The Design Camp was meant to be even more of an eye opener, with the theme of “the designer as explorer”, taking the students away from the university campus, and for many of them through the overnight stays also away from home. (The situation in Sydney is that the majority of the students live at home with their parents, partially due to the cost of living, but also, unlike many countries in Europe where students move out in order to study in another nearby city, in Australia students tend to study in a university in their hometown. This is due to the distance between the major cities, such as Sydney, Brisbane and Melbourne which are all nearly 1000 kilometres apart from each other; the majority of the population in this continent lives in these urban centres.)

Because of its relative isolation, the island became a real hive of activities for the students and staff to work together in activities and explore the location. The students would stay on the camp site and cook together in the BBQ area, and for the participating staff accommodation was provided by renting all the available houses on the island. It really felt like we were able to ‘own’ the island, using it as a canvas for our purposes, while always treating it with respect and avoiding leaving traces or permanent damage. These events enabled me to experiment with a range of video projections and installations, different in each instance of the Design Camp, as a side project to the main teaching activities. In total there were ten Design Camps over seven years, from 2011–2017, usually in April or May (autumn); the last three years the number of first year students across the School (at that time with six degree courses) was too large to take everyone at the same time (or fit in one lecture theatre) so the cohort was split over two semesters. So we had two camps in a year: in April or May, and in September or October (spring). Logistically it was always a challenge to organise the camps, as the island can only be reached by boat. In addition to the regular public ferry services, the School of Design would charter a small ferry crossing to get a car with all the materials and equipment across for the whole camp.

The Design Camp workspace for the students and educational activities was usually the Mould Loft, a bright structure built in 1911, with views across the bay, of about 1500 m² across two floors. In the industrial past, these spaces were used to create the moulds for the ships being build on the island; tracings of these outlines can still be found on the first floor of the loft (sometimes on display, but usually covered with carpet tiles). In the instances where the Biennale was on at the same time as the Design Camp, sometimes they used the space and we had to relocate to other, less suitable locations. But being there are the time of the Biennale, had of course an added benefit particularly for the students.

UTS has a strong interest in Aboriginal culture, supporting research and indigenous scholars through two institutes, appointing indigenous staff and accommodating students, bestowing honorary doctorates on Elders to acknowledge their knowledge and wisdom, and endeavouring to incorporate Aboriginal knowledge, culture and worldviews in the teaching curricula. Often visitors are invited, for instance as guest lecturers, to present their work (and often their own background) in indigenous perspectives, such as Victor Steffensen (about his experience as indigenous bush fire practitioner) [20], Tyson Yunkaporta (about the relevance of Aboriginal knowledge systems and approaches) [21], Bill Gammage (an academic historical perspective on Aboriginal landscaping tradition through the use of fire) [22], and Bruce Pascoe (Adjunct Professor at UTS, who actively advocates the notion of Aboriginal farming techniques and settlement practices) [23]. In the first year of our courses, in the Design Thinking subject, the importance of these insights was given a prominent place [17]. Indigenous designers and scholars Michael Hromek and Danièle Hromek would tutor or lecture in the Design Thinking subject. Together with their sister Siân they had created an installation on Cockatoo Island at the Underbelly Arts Festival in 2015, Covered by Concrete, consisting of slabs of concrete which together formed the (current) outline of the island, reminiscing its past by the trace of the original outline, imprinted with the shapes of local plants (mostly gathered from surrounding headlands as they had disappeared from the island altogether) [24]. Danièle would deliver her lecture in the same session as my Traces lecture, on the island, and it was great to hear about her perspectives on the location and express (also in personal conversations) how disturbing it felt to her and people from the same cultural background to experience the island in its recent and current state—covered in concrete indeed (and the soil quite polluted). In that period, Danièle visited my Interactivation Studio, joining the discussion circle we would often have with MDes students, PhD researchers and colleagues, likening this mode of conversation to the traditional Aboriginal practice of the Yarning Circle. This is a free roaming mode of expression, without top-down direction, based on mutual respect and interest, allowing for new insights to emerge from the conversation and interactions in a serendipitous way. This is, of course, very similar to the notion of the Dérive of the Situationists in the 1960s, ‘drifting’ as an act of serendipitous explorations (as discussed in the XR Projections paper [7]. This another example of how traditional Aboriginal culture resonates with some of what we regard as our most advanced and innovative approaches in ‘Western’ society (there are some other examples discussed in my book Understanding Interaction, such as the value and power of decentralised networks which existed in the continent-wide, resonating network of the Songlines, encapsulating knowledge, culture, and law, which also resembles symbiotic relationships between trees and fungi in the mycelial networks in forests, and decentralised computer networks such as mesh networks and the Internet [25] (pp. 100–102)).

5. Projections at the Design Camps

At each Design Camp I would take the students on at least one VideoWalk of about one hour across the island, as a scheduled event. In some cases these were presented more like a walking lecture tour, other times more as performances. Furthermore I would experiment with several stationary projections, most of which were interactive in which case the audience could play with them and explore the composing of images and sounds. I would also deliver a plenary lecture, usually before the camp in the regular lecture series of the Design Thinking subject on campus, or in the later versions the lectures were delivered on the island. In my lecture, the island and some of its history was introduced, as well as discussing the notion of finding remnants and reminiscences in the environment that reflect past activities, behaviours, or events. This is part of my project Traces—reading the environment [18], that looks at how a deeper awareness of our surroundings and the cues in it can lead to insights and input for design approaches and responses. This notion is already reflected in some of the assignments that the students carry out on the island, which encourages them to engage with the different layers of history at the location. Their activities ranged from taking tracings from the patterns on sandstone building blocks (they vary according to which stonemason/convict worked on them), creating three-dimensional mobiles with found objects reflecting on possible futures of the island, empathising with past inhabitants, etc. Over the years, I have witnessed a vast wealth of explorations and installations created by students in the Design Camps, the majority of them really enjoyed engaging with the island and its layered history. It made it easier for my interactive projections, as a side project, to respond and build on these activities.

I would always stay on the island for the whole three-day duration of the Design Camp, to have time to develop and prepare the projections, often with the help of colleagues, friends or students (see below in the Acknowledgements section), having a room in one of the houses, to stay in and keep all the equipment. This enabled me to work on projections even during the night, when the island is very quiet—the last ferry leaves around midnight, and the rangers (people who work for the Trust and look after the island) leave at 4 p.m., with only a security guard or two on the island overnight. I would usually inform them of my intended activities, and have their support in switching of particular lights in order to avoid interference with the projections, while other activities were more improvised and stealthily executed.

On the occasions when the Design Camps coincided with the Biennale, access to some of the spaces I would usually address with projections were unavailable—although open to the public during the day, the buildings would close at 5 p.m. But it also opened up new opportunities to make connections through projections with the works around.

6. Interactive Video Projections on the Island

Over the years I used several building structures as projection surfaces for the audience to play with. They were the Facets interactive kaleidoscopic projections [5] in several locations, and several projections of fires and flames.

6.1. Facets Interactive Video Projections on Ceilings

I used the set-up developed for Facets of Expanded Architecture [26] interactive video installation (but with different footage, and projecting upwards to the ceiling instead of downwards onto the floor), which is a large, circular kaleidoscopic projection the parameters of which are interactively controlled by audience movements picked up with infrared analogue motion sensors.

Several of these interactive installations can be seen in Figure 5.

The first one was in 2012 on the ceiling of the Search Light Tower, a structure built in the WWII and later painted in striking colours by the Aboriginal people when they were present on the island in 2000. I projected from below with a strong NEC NP2150 3LCD XVGA projector with 4200 ANSI Lumens, with a wide angle lens, and attached to a computer running the image generation software. I hung the motion sensors off the ceiling, detecting movement of people below it so that by walking around under them the image patterns would respond, creating an immersive environment on the ceiling,

I had to switch off the lights on the structure in order to let the projection come out well. I had tested this out the night before with the portable projection set up (VideoWalker). In order to hang the sensors and place the USB interface, I had to climb on top of the tower, the ladder was only half covered so it was easy to get up there.

Not many people showed up in the night as it was far removed from the student camp, and even from the main staff building so I had to invite people to visit and interact with it. It was a good experiment, and I captured the interaction on camera.

At the end of the evening I took the computer and projector back inside, but made the mistake of leaving the sensors to be taken down in the morning. By then, the ranger had arrived back on the island again and just happened to pass by the tower when I was on top of it. I only just got away with it by being very apologetic, and realised that I needed to be more careful when carrying out these activities and act quicker.

In the 2015 Design Camp I set up a similar installation, but this time nearer the tent camp where the students were staying, on a large awning that acts as a shade cloth during the day, and an excellent projection surface at night (after switching off the lights). I couldn’t climb on top, but didn’t need to this time as I placed the sensors by hanging them off wires that were around it. The footage I used as base material for the patters were images from nature, including underwater footage which reflected the nearby bay. This was visible from the tent camp, and led to more attention to and interaction with the installation.

The next year (2016 Design Camp) I projected on the roof of the fake station that was left of the Unbroken film set, this time using footage of architectural structures which fitted better with this particular projection surface, to accentuate its incongruity.

6.2. Facets Interaction Video Projection on the Water Tower

The most extensive audience participation was achieved in the 2014 Design Camp with an interactive installation projecting onto the water tower, which is located centrally on the island, and visible from the mainland. Due to a bright light on the north side of the tower, I always had to project on the (dark) south side, which was less visible from the tent camp on the north. However, many students made it to the top, as well as the staff as the main building is right next to it, and people played for hours making patterns on the water tower. They could generate these patterns by manipulating small wooden blocks with accelerometers embedded, which were initially developed for one of the first Facets pieces as part of Smart Light Vivid in 2009 [5]. From the amount of engagement, and commentary as registered on video recordings it was clear that the participants really felt empowered and involved in the process of image creation (“I’m making the evil eyes”, one participant excitedly called out, referring to a pattern created that indeed looked a bit like a face, which I had not seen before). This projection was clearly visible from the mainland from the south, and my neighbour at the time in Birchgrove, a professional photographer, was kind enough to make photos of it and share them with me, as can be seen in Figure 1 in the left image.

6.3. Fire Projections

At almost every camp, starting with the second one in 2012, I made it a habit of projecting flames on the water tower, after I realised it was an excellent projection surface (as mentioned, on the south side due to the bright light on the other side, which got replaced by an even brighter one in 2014, and that I was not able to get switched off). It is a local landmark, the highest point on the island and visible from far away on the surrounding shores. I used the NEC projector, with the standard lens, propped up and aiming up to the top part of the tower, this worked really well. The video recordings of the flames are shot in show motion—fire has an immensely dynamic nature which is very hard to capture at a normal frame rate of 24 or 25 fps, when played back at that speed it gets jerky. So I have a habit of capturing flames in higher frame rate, 120 or 240 fps, and play it back slower. (Animators tell me that computer generated flames sometimes look better than real flames on video—this explains why; it is another example that ‘extending reality’ is not necessarily a linear and straightforward process.) The next year (2013) due to this practice run I was able to place the projector at daylight, have it turned on and play the flames footage from the attached computer with the video software, slowly appearing as the night falls (dusk is quite short in subtropic Sydney) which I captured on a video camera also set up in advance. This was done so that I was able to use the small dusk window of video projection opportunity optimally by doing a VideoWalk at the same time around the island (discussed below). In 2014 I projected on both water storages, the tower and the lower one. The garden of the main guest house where the design staff was staying was adjacent, and we enjoyed our outdoor dinner below the flames, a nice backdrop, see Figure 6. These projections on the tower had an extra meaning for me, because in the Dutch language a lighthouse is called a ‘vuurtoren’, fire tower, but even without that knowledge the contrast between the projected flames and the water creates an interesting tension.

While these water tower fire projections were not interactive, in the following years I presented an interactive fire installation in two different locations (a small space as a passage from the Mould Loft area to the Convict Precinct in 2015, and in 2016 inside the Mould Loft where the students were working). These fire projections consisted of a set of three videos of different fires projected on the wall, again in slow motion, but of each of the three videos the speed was controlled by one of three infrared analogue motion sensors which were hung in the space. A soundtrack was simultaneously influenced by the movements of the participants as picked up by the motion sensors (a very sparse beats track from the Jungle Brothers—uncredited, but with due respect, the effect was similar to ‘scratching’ records) displayed on two Genelec 8020A Studio Monitors. (The technology and video footage was adapted from another project I was involved in, learning about the sophisticated and effective use of fire as an indigenous traditional landscaping technique and bushfire prevention method [25] (pp. 102–104), as described in books by academic Bill Gammage [22], Bruce Pascoe [23], and our collaborator Victor Steffensen [20], mentioned above).

Realising I had a theme now, projecting flames on locations related to Cockatoo Island, I started looking for more opportunities. I had discovered that for some strange coincidence, one can draw a straight line from the Cockatoo Island water tower, to another water tower at a peninsula further east, closer to the sea, in Vaucluse, as shown in Figure 6. While this line does not show up on the nautical map of the area as a navigational aid, the line runs from Cockatoo Island in the west, crosses nearby Goat Island (lining up two marker poles for navigation), the exact middle of the Harbour Bridge, the tower of Fort Denison, a corner of Shark Island in Rose Bay, and then the Vaucluse water tower in the east. It also crosses through the base of Yurulbin Point in Birchgrove—where I lived at the time, on the site of the Birch Grove house from 1812 which was demolished in the 1967 to make room for two blocks of flats.

As an extension of the fire projections on the island, in 2013 I had an opportunity to project the flames on Fort Denison, a small island to the east of the Harbour Bridge and closer to the mouth of Port Jackson, strategically located. This was by invitation my colleague Mike Day, architect and lighting designer, who for an event with a group of lighting designers as part of Vivid organised an event on this location which is usually very hard to access. We had explored using my projections on his dynamic kinetic light sculpture (with the lights off) for the BEAMS festival in Chippendale some weeks prior, which we repeated here but it also gave me the opportunity to project my flames on the tower of Fort Denison (thereby turning it in another ‘vuurtoren’) and captured the whole thing on video (until I was told off for walking on the roof).

As part of the Design Camp in May 2015 I projected the flames on a screen on the window facing the bay, at my flat in Birchgrove which was on the ‘fire line’. I left it on, on a timer while I was on the island, and it was (just about) visible from the island, creating an experience of a remote location connecting to the island.

7. VideoWalks on the Island

At every Design Camp I would do at least one public VideoWalk around the island, using cliff faces, tunnels and buildings as ad hoc projection surfaces. The VideoWalker instrument is a portable projection set up developed and used by the author since 2000, consisting of a backpack with a large 12 V battery, mains converter (‘inverter’) to power the handheld 2000 ANSI Lumens projector, which has a range of custom built sensors attached which control the real-time image generation software (in Max/MSP-Jitter) running on a MacBook in the backpack. With this set-up, the operator or performer can select and manipulate the shape of video-images or stills, and place them in the environment.

With the controls, it is possible to shape the image to fit the outline and features of the projection surfaces (trees, buildings, cliffs), as a kind of live projection mapping. Earlier versions and performances are described in the paper The Projector as Instrument [6], and information and reflection about recent VideoWalker performances as well as the technology in more detail in the companion paper Exploring Extended Realities in this Special Issue [7].

The first VideoWalk I carried out on the island (in 2011) was mainly to explore the island and see how my regular bank of videos could be applied in this context, as can be seen in Figure 7. This VideoWalk was mainly improvised, a Dérive like exploration.

Over the years I was able to use specific video material from this location, re-situating video and audio of previous art works, installations, sculptures, graffiti, and movie sets, by projecting images of it where they used to be. It is important to me to use my own footage of these events, as a way of owning the re-situation; the material was often gathered with this purpose of projecting them back in the spaces, and shot in a certain way, with the aim to capture their spatial presence. This is important for a sense of agency with these projections, to make it more personal and appropriate, to resonate with the physical location of the island with all its layers of history, meaning and context. (The exceptions were the usage of some of the footage of the movies or documentaries shot on the island, using a few key scenes that reflected the location).

7.1. Taking Video for a Walk

In this section I am taking the reader along a typical island VideoWalk. In order to gather an audience, I would often start my VideoWalk by going through the student campsite, while on other occasions I would start from the students work space and take them and the staff members for a VideoWalk from there (this was often likened to the Pied Piper). There were some competing interests, understandably at this time many people were keen to enjoy dinner. As discussed above, my timing was determined by the sun setting so I had little leeway.

I would project on the floor, the concrete path in front of me, on the tents, etc. I would sometimes continue with the projection of more specifically situated material the old shipyard industrial buildings, as the spaces had less light to compete with the projections. I would be re-situating works from the past Biennales and the Outpost street art festival among others, choosing works that I thought were particularly successful in the way they integrated or responded to the space, and were suitable to be represented by projections. The intricate architectural installations of Philip Beesley, Hylozoic Series made out of laser cut acrylic huge feather like interactive structures, with lights, sound and movement, were particularly well suited to be projected back where they were in the Biennale of 2012 [27,28] in its somewhat intimate space. The largest space, the Turbine Shop, was harder to address, not only for my projections but also for the works presented there originally. Over the years I have seen that some artworks presented there struggled to deal with the vastness of the space, similar to the case of the Tate Modern Turbine Hall in London. One particularly appropriate work was by Cai Guo-Qiang, who often operates on the monumental scale with animal replicas in large quantities in several positions in space, fireworks and explosions [29]. This work, for the Biennale in 2010, consisted of several cars in different orientations hanging in the space forming a continuous suggested frozen movement, animated by a changing pattern of lighting poles, which were protruding from the cars. Although it was suitable material to project back, I could not always address the space at the full scale, instead projecting on smaller surfaces, and on the outside of the building which worked well.

My main activity in this area was projecting footage of the Wolverine movie back onto the locations where they were recorded, partially on the remnants of the film set. Particularly effective was a scene where the Wolverine and his brother bash through a window (seen above where I was projecting) and land on the floor to engage in a fight (some spatial dislocation going on there). We would then go through the ‘gates’ (which were still there) of the ‘prison’ where the ‘mutants’ (X-Men) were held. There I would move on to project footage of an art installation of the 2014 Biennale, in the space just past the Wolverine set gates, The Village by Randi Jørgensen and Katrine Malinovsky (Denmark), which was a collection of small houses with anthropomorphised features, charming on the surface but on closer inspection hermetically closed to resist visitors, a comment on attitudes towards asylum seekers and refugees in many countries [30] (pp. 226–227). This theme is echoed by Ai Weiwei, in his installation at the same location, The Law of the Journey, a sculptural installation of a grotesquely oversized dingy with refugees for the Biennale in 2018. I have footage of Ai Weiwei’s installation too, but I have not had a change to project it back; my intention is to bring both installations, and perhaps the prison scene from the movie, now that most of the actual Wolverine set has been removed, all together with projections in this location, to show their relationships.

7.2. Re-projecting the Mox Stencil

A particularly well suited image for re-projecting, because of its good size and placement, was an intricate mural, a stencilled image of a fish-eye lens view of the inside of the iconic Natural History Museum in London. It was stencil-painted there as part of the Outpost street art festival, and one of the first to be painted over after the show (again, due to its size, an easy target). It seemed utterly daft to me that this effort was made, as it looked perfectly in place in this location (the wall of one of the shipyard industrial buildings), so this was one of the first images I decided to project back, in May 2012, as shown in the image sequence below in Figure 8. At the time I knew little about the image, I even thought it was a train station (it reminded me of Liverpool Street station in London), but later recognised it as London’s Natural History Museum. When preparing this article, I realised I actually did not know who the author was of this image. As often with events like Outpost, the web site disappears after several years, and if there is no catalogue (such as the Biennale produces), all that is left are some random images on Instagram and Facebook. In this case, all I had was the brochure of the event that was produced at the time, a few copies of which I kept as I realised this might be the only lasting piece of documentation, however scant. Most of the other works I used for projecting back in situ I had managed to identify the creator(s) of, but this one was a mystery, as there was nothing in the brochure that I recognised, and several web search attempts did not yield any result. In 2021 I had discovered the same image (in a different tint) on a wall in a laneway in a western suburb in Sydney (Leichhardt), so in June 2022 when I passed by again I decided to ask inside the silkscreen print workshop (Publisher Textiles & Papers) on which wall the mural was tagged (among other paintings, and a bunch of Will Coles sculptures). When I asked the person who turned out to be the owner of the print workshop, Mark Cawood, about the origins of the mural, he revealed that it was actually his work! I recognised several other tags casually sprayed on the walls of the workshop, and more Will Coles sculptures. As it turned out, operating under the name of ‘Mox’, Cawood made the stencil for a commission on a wall in Enmore earlier in 2011 (painted over since…), also put one on his own wall (still in good condition), and then the one on Cockatoo Island at the end of 2011. Because it is such a large scale stencil, it is a lot of work to put the image on a wall. Cawood told me that the stencil got damaged while rushing to catch the ferry off the island, so it could never be used again. I am sharing all this, because it is a good example of extending the reality of a bland wall in an unused space on the island, with a lot of effort, only to be erased and forgotten about—until projected back (in May 2012), though only momentarily.

7.3. Re-situating Deb and Will Coles

Coming out of the industrial buildings towards the south-east corner of the island is another good location. On one spot where there used to be a large round concrete tank with a tag of Deb, which was since removed (the whole tank—there were several strewn around the island and tagged during Outpost, and as mentioned above I only recently rediscovered them gathered together elsewhere on the island but with the murals painted over). I would project the image back on the ground, or on the concrete WWII bomb shelter behind it. I would also project images of Will Coles cement sculptures that used to be there, in this location on the building.

This can be seen in the images in Figure 9, with the original structure shown on the left, with a painting by graffiti artist Deb and the Will Coles plaque, and another Will Coles sculpture (the LOL gun) in the background. Images of these were projected back during several of the VideoWalks (images in the middle or right).

Like other Will Coles sculptures, as a kind of concrete graffiti, his plaques can be found all over Sydney. They mimic the plaques commemorating famous people and buildings, in this case with the text “This plaque commemorates it’s (sic) illegal installation—Will Coles 2011”. Recently we also attempted to restore the left over pieces of the Will Coles plaque, and a balaclava, bits of which are still stuck on another one of the concrete tanks, as a kind of physical instead of virtual resituating (images on the right).

7.4. Large Scale Re-Projections

To re-project the larger outdoor murals on the east side of the island were a challenge, on some occasions I used a stronger projector (the NEC NP2150) on a mains power supply to address these surfaces. One was a very impressive mural of ROA (from Belgium) on a wall on the south end of the eastern apron, facing the entrance of the island—this mural was removed in a way that a faint outline is still visible—it was certainly not an improvement, it would have been better to just keep it there! It is one of the many examples where the surface actually looked worse for the shoddy removal job than when the mural was still present.

On another large wall on the other end of this row of industrial warehouses, I re-situated a work by Vexta [31] (pp. 241–243) which on one occasion was extra challenging due to the piles of dirt (ready for the motor cross event) blocking the access. On the same wall I would later also re-situate images of large Japanese billboards, mock ones that were put there for the film set of Unbroken.

7.5. Re-Projecting Mini Graff

I would then move on to a building in the middle of the south side of the island (actually a toilet block), where at the Outpost festival Mini Graff painted one of their trademark images of the Suburban Roadhouse series (see Figure 10) (one of these was also in the May Lane exhibition [7], [32] (pp. 51–52).

These are based on logos of well-known brands in advertising billboards, their semantics distorted so that it becomes a somewhat sarcastic and critical reflection on contemporary consumerism and the push of large companies to influence society (an Australian bank, in this case, the ‘grab’ logo reflects the already ominous real name of the bank ‘nab’). Again, it was obviously an enrichment to the building, but it got removed. In May 2013 I projected it back, for a brief moment restoring the original intent. Later that year, for the film set of Unbroken, the building got a Japanese POW camp makeover, blackened like several other buildings in that part of the island. Eventually it got cleaned off, and the building is back to ‘normal’. All this shows, of course, the difficulty of ‘preserving’ the state of the island while there are so many undeniable (and not always acceptable) forces which lead to a dynamic presence. These various stages are shown in Figure 10.

7.6. Tunnel Projections—Jonathan Jones and Daan Roosegaarde

From there we would go through the (straight, south–north) tunnel back to the camp site. On my first VideoWalk I projected one of my videos from a tunnel (captured somewhere in the Alps, in Switzerland), which gave a nice effect, animating the static tunnel. But more meaningful was to re-project footage of another Biennale artwork that I really liked and that worked extremely well in the tunnel, a dynamic light pattern by Jonathan Jones mounted to the ceiling in 2012 for the Sydney Biennale. Jones is well known for his fluorescent strip light patterns [33], and this one worked particularly well. Of course my projection of this footage was only a faint impression of the actual work, but as a statement and reference it was effective.

In the other tunnel, the ‘dog leg tunnel’ (it has a bend), I would re-project the light installation of Dutch artist Daan Roosegaarde [34] which consisted of a large assembly of interactive lights on bending poles in the Beinnale of 2012. This installation emulates a bunch of reeds moving in the wind, as we often see on the banks of rivers and lakes in The Netherlands, but then electronic and interactive, creating mesmerising dynamic light and motion patterns.

7.7. Re-Projecting Tori Wrånes Performance

In one case I had installed a fixed projection as part of the VideoWalk, as an interlude during the walk at a specific location. This was to re-situate a music theatre performance Stone and Singer by Tori Wrånes from Norway, I had witnessed on the opening weekend of the Biennale on 23 March 2014 [30] (pp. 270–271). The performance made a great impression, and it also made use of the vast space of the Turbine Shop very effectively. The audience stood around a circle drawn almost to the full width of the space, with the performer (Wrånes, in the role of a troll-like creature) in the middle of the circle. At some point in the performance, a large boulder-like structure which was mounted on a movable swinging arm attached above would be released, and in pendular motion sped just above the head of the Wrånes, initially at a high velocity, a very dramatic effect (and hence the circle, to protect the audience). Later, a group of actors would sing towards the rock, which itself had a speaker inside and produced sound, and it could also change shape dynamically.

A handful of musicians, playing wind instruments, were placed on several platforms high up on the walls (making use of the existing structures in this industrial building—the players were suspended with to the building with safety harnesses, and had to be placed there with a scissor lift). It was a most remarkable performance, utilising the characteristics of the space (the large scale video projection by Eva Koch of a waterfall at the far end of the building (which I re-projected in a VideoWalk in May 2015) was turned off during the performance). It was only performed twice… all this effort, and all that remained was the trace of the circle on the ground, and the boulder, suspended above the Biennale visitors, motionless, silent. Within days it was moved to another location at the other end of the building, near where the stencil of Mox was during Outpost. It seemed a great loss to me, there was hardly any indication that such a spectacular event had taken place nearby.

For the re-situating of the piece in May 2014 (the Biennale was still on) I used the hanging boulder and the wall behind it as a projection surface (the space was closed at night, but I could project through the fence, and the audience was able to see it from outside of the building), see Figure 11. The set up projected video footage of the performance, and a studio quality monitor (Genelec A8020) displayed the sound, the video projection briefly animated the boulder—although a very limited reminiscence of the actual event, it did more justice to the performance than just the hanging boulder. A small audience of students and staff, who followed me with the VideoWalker, enjoyed this presentation.

7.8. Recursive Projection

I also experimented with re-projecting actual re-projections, to explore a recursive process. Eventually, over four years, creating five iterations, each with a layer of virtuality added to the original real level. The ‘reality’ level was my son Rufus walking down the metal walkway which is suspended in the narrow space between the north side of the Turbine Shop on the outside, at roof level (about 12 m high), and the cliff face. In the Biennale of 2012, in the crevasse like space below the walkway, the artist Fujiko Nakaya had installed one of her ‘fog sculptures’, filling the space with a thick fog, ever changing shape and meandering out the front near the entrance of the Turbine Shop, and up past the metal gangway above. Rufus was about two years old then, and commented that it felt “wet”, and urging me to join him to explore the fog (“come on, pappa”). A year later, in May 2013, I asked him to ‘re-enact’ it while I projected the video (and audio) of the previous year back along the gangway. Being used to our video projections ‘in the wild’, this was not an unnatural thing for him to do, and it resulted in the three years old Rufus interacting with his two years old virtual presence. The next year we repeated this, now the four years old (real) Rufus interacting with the projection of his three years and two years old self. In May 2015 we did another layer, and another one in October of that year, so by then we had two five years old Rufusses, interacting with all the earlier ones. When he was six we did one more, in October 2016. By this stage it was getting hard to see, so this was our last one. All the successive layers are shown in Figure 12, with stills from the videos. Of course as it gets more layered, it becomes increasingly impossible to have all of the Rufi in one frame (the stills from the videos used here), but in the videos all layers are visible. It is a bit like the Droste effect, the recursive process of infinite regress by placing an image of the image inside the image (etc.).

Although these explorations were mostly improvised, lucky enough his clothes have a different colour every time which helps discerning the stages. But of course in retrospect, had we properly planned and designed this act, I would have taken more care of the ‘colour coding’, and also the framing and the light levels. Then, it would have been possible to create even more layers. But the interesting thing about this exploration is that it spans a period of over four years, creating the layers while capturing the growing up of the main actor.

8. Discussion and Conclusions

In this paper I have presented projects on the island that use audiovisual projections to overlay the environment with layers of meaning and reminiscence, particularly through the re-projection of previous events relevant for a certain location, creating echoes and resonances. This is shown through a number of examples, with reflective presentation of the events.

The implicit suggestion is that if more of the artworks and other activities that occurred on the island been preserved, a unique and impressive sculpture park would have emerged by now.

A key aspect is the (re-) framing of the image material, (re-) situating this in the context of the Island, in various locations. The intention was to create meaningful overlays, extending the environment with moving images and sound, bringing back lost views and in other cases altering the environment with the projections, commenting, relating, or paraphrasing.

The majority of the projections presented were interactive, and allowing for audiences to co-create the experiences. The interactive element in the VideoWalks was limited to the performer, a limitation that I have tried to overcome in the past by letting audience members use the set-up themselves, which was always much appreciated. To explore this further, on one occasion on the island in the Design Camp of October 2015 I created an audience version of the VideoWalker instrument, an assembly of projector and computer (Apple Mac Mini) running the real-time video generation programme, handheld and portable but on mains power (with a long cable). This was set up in the Mess Hall space on the upper level of the island, a sandstone building from 1851. Some of this can be seen in Figure 13, showing students (Oliver Bryant in the image on the left) and staff (Paul Sutton in the image on the right, who later started experimenting with placing photographs around the island to reflect on past events [35], including a photograph Paul took of my projection on underside roof of the fake train station in 2016 (see Section 6.1), and then placed this print back on the underside roof in September 2017).

The participants experimented with this set up and commented positively on the experience, and it was observed that they were quickly able to dynamically project back into the space. The imagery used was of a video of an installation that was in this location in the Biennale of 2014, and by moving the projection around in circles, participants we able to match the projection of the 360° pan unto the actual room. This is footage that I use myself in this location during the VideoWalk; it is a video of the mixed-media installation gradually_real by Swiss artist Christine Streuli, who created a copy of the walls of the room on panels in the room, overlayed with paint, print, and other materials. This installation was described as a “one-on-one replica” with “architectural strata” in the Biennale 2014 catalogue [30] (pp. 252–253). By re-projecting the footage of this installation, a double reflection or reminiscence was created, adding another layer to the “strata”.

Reflecting on these projects, design thinker Kees Dorst (who initiated the UTS Design Camps in 2011), emphasised that with these interactive, responsive projections a kind of public AR is created, that performer/designer and audience share the same extended reality space in the real world, blending the features of both. In this sense, it is “re-humanising the the de-humanised” as he puts it.

Due to the improvised, exploratory nature of the activities, it was not always possible to capture audience responses. Often audience members presented their appreciation, and particularly in the cases of the more well attended instances of the VideoWalks (dozens of people) responses were verbalised, and often captured on the video recordings so we were able to document this—these were always positive. However, these projection projects had less presence in the formal feedback, in the Student Feedback Surveys that we conduct at UTS these activities were barely mentioned. This may be because the Design Camps were only a part of a larger subject, Design Thinking (and as such did often get explicitly positive mentions), and the projection projects only a minor part of that.

The projections and interactive installations did show up in the #designerexplorer hashtag on Instagram, as part of the PhD research of design tutor Jenni Hagedorn carried out as part of the Design Thinking subject in 2012 and 2013, into social media, networked publics, and identity development of ‘emerging adults’ [36]

In later years, students in my other subjects (or graduates, even later), would often tell me that the projections and (inter)activities had made a great impression.

The main conclusion is that by using these audiovisual projections audiences and all participants can share an experience, of extended and mixed realities, blending meaningful virtual materials with the physical environments, linking the past to the present, creating extended layers of meaning. Through the explorations and experiments presented in this paper, many new insights, techniques and knowledge are gained, addressing the main research question of how to engage an audience. There are many advantages of sharing a real environment, unencumbered by individual headsets, or distracted by handheld screens.

It is always good to bear in mind though, that real reality has many ‘extended realities’ through impressive natural phenomena that occur, as can be seen some images I captured of such events as shown in Figure 14. This can be humbling, when compared to our artificial realities produced.