SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
The World Building Framework for Immersive
Storytelling Projects
Yan Breuleux1, Bruno de Coninck1, Simon Therrien1
1Université du Québec à Chicoutimi, NAD-UQAC, 2566 Rue de Bleury, 7e étage Montréal (Québec)
H3A 2J2, Canada
Abstract. This article explores issues associated with immersive
storytelling in order to examine how the field of World Building can
constitute a theoretical framework for practice in the context of VR-based
and Full Dome artistic projects. With respect to immersion, the intent will
be to interpret the concept of storytelling in relation with the recent
formulation of the concept of extended reality (XR). The very concept of
World Building is transauthor and transmedia by nature. The transauthor
dimension of World Building resides in the idea of subcreation, i.e.,
designing environments and interaction rules that help create a storytelling
basis for generating multiple stories. Once the universe has been conceived,
stories written by different authors take shape through transmedia processes
across multiple distribution media (film, video games, web, etc.). The
question then arises: How can the World Building approach shape the
construction of immersive experiences? The article sets out to answer this
question, and in doing so, to contribute to the research on environmental
storytelling.
1 Introduction
This article explores issues associated with immersive storytelling in order to examine how
World Building can constitute a theoretical framework for practice in the context of virtual
reality and Full Dome artistic projects. Our articulation of this theoretical framework is
inspired by the conceptions of researchers and practitioners from the film and video game
domains. This explains why the diverse sources for this article include books, articles,
specialized websites, presentation excerpts, and interviews available online. We respectively
present the conceptions of several figureheads of the narratological current of game studies
such as Mark J.P. Wolf, Henry Jenkins and Marie Laure Ryan. As for practitioners, we
discuss three designers: 1- Alex McDowell, film production designer and researcher;
2- Monika Bielskyte, futurist & future designer; and 3- Don Carson, game and theme park
designer. These three profiles will allow for describing a particular conception of World
© The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons
Attribution License 4.0 (http://creativecommons.org/licenses/by/4.0/).
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
Building and environmental storytelling that is focused on practice. After sketching a broad
picture of Digital Storytelling, we explore the World Building concept in relation with the
notion of environmental storytelling. This lens will then be used to examine research-creation
projects.
The aim of this article is thus to provide an outline of a World Building practice for the
field of Full Dome and virtual reality experience design. To do so, it is important to begin by
establishing a field of research for teaching and transmission in the field of the creative
industries (i.e., digital art, visual effects and video games). Next, we attempt to shed light on
how immersive installations shape the mise en scène of narrative dimensions. After proposing
a typology of immersive environments, we examine several research-creation projects carried
out at the École des Arts Numériques de l’Animation et du Design / School of Digital
Technology and Animation (NAD-UQAC) and at the Société des Arts Technologiques de
Montréal / Montreal Society for Arts and Technology (SAT).
2 Digital, Virtual, Interactive Storytelling
Digital Storytelling, as a field of study, emerged at the crossroads of common interests in the
research fields of digital art, literature, communication, and computer sciences in conjunction
with the development of new media and entertainment industries. These industries include
Massively Multiplayer Online Role-Playing Games (MMORPGs), interactive and immersive
experiences of all kinds, smart toys, etc. [1] Given the vast array of applications and practices
arising from this field, it should be pointed out that digital components and storytelling are
cross-cutting concepts that allow for much crossover between disciplines. In light of the
applications and installations produced by researchers in the field, Digital Storytelling
embodies the promises of the confluence of the new digital economy and technoculture.
As a brief historical reminder, storytelling dimensions have always been central to the
development of new apparatuses and installations. When patenting the Telesphere Mask on
October 4, 1960, Morton Helig referred to a “Stereoscopic-television appartus for individual
use” [2]. Hence, he was already establishing himself as articulating a new entertainment
apparatus. Subsequently, in 1966, Ivan Sutherland created the first real-time 3D animation
experience with his virtual reality headset, named the “Sword of Damocles”—a name that
aptly conveyed how heavy the device was. In his article “The Ultimate Display,” Sutherland
interestingly mentions the metaphor of Alice in Wonderland [3]. Virtual reality indeed
emerged as a technological embodiment of the famed journey through the looking-glass that
plunges Alice into her fantastic world where anything is possible. By making this parallel
with a literary classic, Sutherland concretely describes the fantastic nature of this new
medium, which, without this metaphor, may have struck some as rather abstract and therefore
harder to grasp. In doing so, he also successfully anchors this new medium in the timeline of
history by assigning it a specific and significant role from the standpoint of the public. Both
of these virtual reality apparatuses—mask and headset—anticipated, or rather concretely
realized (but without today’s powerful calculation capabilities) the fundamental
configuration of modern virtual reality devices.
If we were to look for a pivotal moment for digital storytelling in terms of a
diversification of practices and the active assertion of the term, it could be located from the
beginning to the mid-1990s [4]. With the development, democratization of the computer and
the multimedia web browser (the Mosaic browser in 1994), new digital media supports and
2
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
advances in 3D simulation, a new form of interactivity gained ground, based on real-time
connectivity to databases accessible locally and/or online. The utopian period of the 1990s
was characterized by the hypothesis of the advent of a new language borne by digital
technologies, particularly immersive ones. Janet Horowitz Murray, with his book Hamlet on
the Hollodeck initially published in 1997, became a leading figure on this topic, and continues
to be a key reference to this day [5]. The research in this field also includes the proceedings
of a series of computer science conferences that started in 2001 on the overarching theme of
Virtual Storytelling (ICVS). As mentioned in the introduction to the first conference in this
series, “The first International Conference on Virtual Storytelling gathers researchers from
the scientific, artistic, and industrial communities to demonstrate new methods and
techniques, show the latest results, and to exchange concepts and ideas for the use of Virtual
Reality technologies for creating, scripting, populating, rendering, and interacting with
stories, whatever their form, be it theatre, movie, cartoon, advertisement, puppet show,
multimedia work, video-games...” [6]. In 2008, the term “virtual” in the conference titles was
replaced with “interactive.” It is worth noting that Janet Horowitz Murray, whose works are
often cited as a theoretical framework, was the keynote speaker for the 2018 conference [7].
The utopia of creating a 360° literary remediation of a translation of Hamlet was realized in
2019 by the Commonwealth Shakespeare company in collaboration with Tech Giant Google
[8].
Currently, to explore recent technological innovations in interactive storytelling,
the community of practice associated with the Future of Storytelling (FoST) current has been
taking an interest in the promotion of artists, theoreticians, researchers, entrepreneurs,
designers, directors, etc., but also in research on new experiential practices [9]. Immersion
issues hold a prominent place in this community. To emphasize the pluridisciplinarity of
FoST contributions, it should be pointed out that this new field of creation also extends to
documentary practices. For example, the laboratory of researcher William Uricchio, of the
MIT Open Documentary Lab, explores, among other things, the experiential and informative
potential of the virtual reality documentary [10] According to Émilie Ropert-Dupont, the
practice of virtual journalism raises interesting questions about the role and purpose of
emotions in transmitting information via immersive media [11]. Quebec is home to an exhibit
series on the theme of Sensory Studies that partakes in the FoST current. This series is of
cultural significance, given that it is presented at Montreal’s Centre PHI every year and is
affiliated with the founder of the Future of Storytelling, Charles Melcher [12]. The most
recent conference in this series, entitled “The New Storytellers,” featured Douglas Trumbull
as its keynote speaker. The remarks made by Trumbull, who was responsible for the visual
effects in the film 2001: A Space Odyssey (produced by Stanley Kubrick), [13] addressed the
practice and language issues surrounding immersive cinema [14]. To clarify the subject of
this article in relation with Trumbull’s thinking, we are witnessing the advent of a new
medium shaped by the viewer’s special position in a situation of immersion. The first-person
point of view participates in co-constructing the story of the immersive experience. Indeed,
immersing the viewer in the story makes him or her a full-fledged player within it.
Michael Gödde et al. describe this practice as “VR film.” They also mention that producing
a 360° story requires guiding viewers’ attention to the important elements in the scene,
determining whether the “viewer” is an active participant or passive observer, choosing the
right point of view for the immersive camera in order to determine the anchoring point in the
environment, and thinking through the relationship between the temporality of the story,
on one hand, and its relation to space, on the other. Finally, the concepts of framing and
montage also need to be rethought in this context. The issue of the point of view and the
language of immersion is also addressed in reflections by authors such as Celine Tricart and
John Bucher [15].
3
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
These aspects show how the introduction of immersive cameras turns the
cinematographic narrative structure on its head. In sum, the fundamental question of digital
storytelling remains: How do digital technologies transform and shape the construction of
the storytelling dimensions of the experience?
2.1 Storytelling and technological environments
The recent rise and proliferation of virtual technologies has led to the need to update the
concept of virtual-reality continuum as proposed by Paul Milgram and his team in 1995 [16].
The concept of extended reality or cross reality (XR) has recently emerged in response to this
need [17]. The concept is interesting to consider in that it allows for merging mixed reality
technologies, including virtual reality, augmented reality and new forms of spatial
virtualization emerging from ubiquitous computing. In this context, and with the increase in
cell phones’ recording, analysis and calculation capabilities, a hybridization of virtual and
ubiquitous technologies may reasonably be expected in the (relative) long term. Without
getting into the statistical details, it is very likely that virtual reality will be democratized
primarily via the Internet of Things. This progression of technologies makes it possible to
capture and model spaces and bodies in real time. For example, Google’s previous
smartphone research program Tango was able to project point cloudes in order to capture
volumetric data in the user’s environment. The program’s spatial modelling allowed for
designing augmented reality applications and games adapted to the particularities of the
specific architectural space at hand. In addition, being able to digitally capture faces suggests
the future possibility of adapting data to the specific profile of each individual. This means
that our connected devices would transmit their actual real-world anchor point in space but
also the topological data captured in each location. In turn, this would mean that any real-life
environment could become the basis for a personalized immersive experience.
To better understand how such technologies shape storytelling dimensions, we have
divided immersive environments into five types. The goal behind this typology is to define
the conditions under which storytelling is developed and the extent to which it is possible to
adapt and/or create a narrative and sensory experience. More specifically, we drew up our
typology of immersive virtual environments (IVEs) following the logic of extended
environments (XR). This typology is based on five forms of technological environments:
1- virtual reality environments, 2- mixed reality (MR) environments, 3- ubiquitous
environments,
4- recorded environments, and 5- projected environments (outside the
computers). Given that virtual reality (VR) environments are calculated and projected within
the computer, we include video games, metaverses, simulators, etc. in this category. Mixed
reality (MR) refers to Augmented Reality environments created by the synchronization and
simulation of virtual objects that are adapted to the real world. Conversely, Augmented
Virtuality Environments (AV) synchronize the presence of real-world objects in the virtual
world. For example, this includes Location Based Entertainment (LBE), which leverages
different forms of synchronizing the real world within the virtual world. LBE incorporates
captured movement and modelling of physical space in synchronization with objects, people
and visual textures. The third category in our typology is Ubiquitous Environments, which
include physical computing, smart clothes and the Internet of Things. In this case, usage
becomes the prime focus, the interface becomes invisible, and the object integrates into the
real-world environment using various forms of geolocation in relation to the individual. For
their part, Recorded Environments are based on 360° live surround capturing and recording
of action in video format and point clouds. To avoid confusion, it is important to note that
4
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
Recorded Environments cover all forms of data capturing, from 360° video to Lidar
technologies and up to photogrammetry and videometry [18]. The final category is Projected
Environments (outside of computers), which include CAVE / FullDomes / Multi-screen and
video mapping. These projected environments are created by synchronizing multiple
projectors. This is referred to as Spatial Augmented Reality (SAR) [19]. In other words, a
building, dome, sculpture or any other type of medium is augmented using various
technologies and video projections. All of these environments help structure the narrative
dimensions of immersive storytelling. The principal hypothesis of this article is that the
World Building framework can contribute to research-creation in immersive storytelling.
2.2 World Building
The World Building approach upholds an industrial practice of transmedia storytelling that
makes it possible to provide franchises, television shows, films and video games with a
degree of unity with respect to the narrative, aesthetics and formal aspects, for distribution
across a wide array of platforms [20]. However, in light of an examination of the books
written on this topic, World Building can serve as a model for many diverse practices, from
literary practice and futuristic exploration of architectural scenarios to the production of
expanded reality (or cross reality) (XR) projects.
To properly circumscribe the World Building approach, it is worth relating a very
telling anecdote. During a conference entitled “World Building the Marvel Way,”
Bill Rosemann, Director of Marvel Games, told the story of a Captain America fan who,
during the 2003 Boston Marathon, decided to turn back to help out the rescuers; Roseman
then precisely detailed his vision of the term: “When Marvel does it job right, we all do our
job right, we create fictional world that can then impact the real world” [21]. Although there
are many ideological elements to take into account in the context of such an assertion, it does
highlight the foundations of the World Building conceptual framework, i.e., the complex
relationship between the primary world (real world) and the secondary world (imaginary
world). The imaginary world exists in parallel with the real world. Similarly, as Bielskyte
noted in a critique of traditional media, “We have to see the world, not something that we see
in a frame, but really as a world that would actually exist as parallel world to our physical
world” [22].
In a similar vein, according to Mark J. P. Wolf, World Building consists in designing
secondary worlds inspired by the real world and which establish the underlying conditions
that enable stories to emerge. Wolf uses the term “subcreation,” which he himself borrowed
from J. R. R. Tolkien [23], and notes that the “world” can be imagined in a geographical as
well as in an experiential form. A world can contain anything that can be experienced by
characters within its fictional realm, and the same goes for the events that can give direction
to their lives. Interestingly, Wolf makes reference to the etymological meaning of weorld, an
old German term meaning “all that concerns humans” [24].
Wolf also describes World Building as being organized by three broad rules of
design: invention, completeness, and consistency [25]. The first rule is invention. Invention
grows out of, but also distances itself from, the real world by creating new hybrid figures in
the design of technologies, cultures, artifacts, institutions and new forms of ecosystems.
In our view, Bielskyte gives a fairly good description of the relationship that the principle of
invention establishes between the secondary and primary worlds. To help bring this fictional
world into being, Bielskyte asks a series of questions: “[…] what are the landscapes we do
not know? What other plants, what are the animals. What is the contemporary art? What are
5
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
the sort of scientific breaktroughs or scientific locations? And sort of really looking at the
richness of diversity, plurality of our world and our culture and the environment that we live
in […] looking at it, at the shapes of theses colors, of theses stories and theses gestures and
transforming them. Taking the best of reality, remix in it in a new way, changing the scale of
things and adding a layer of magic and imagination” [26].
The second rule, completeness, has to do with the internal limits, or boundaries, that the
world sets up. In other words, it refers to the quantity of information available to the viewer,
visitor or user exploring the world. The idea is to enable this individual to find and verify the
information they need to guide their actions. In other words, the question to ask is whether
the explanations provided by the established systems inform the individual of the rules that
organize the world, while also keeping its boundaries concealed. How can the world be
contained, both in terms of territory and imagination? What strategies are employed to bound
the world’s geography while keeping its borders invisible to the world’s “guests”? These two
components must support the third rule, which is consistency. In other words, invention and
completeness obey fictional rules which must, in the context of the subcreation, give the
world a high degree of internal plausibility. Importantly, plausibility does not reside in the
world’s actual relationship to the real world, but rather in the world’s consistency via the
functioning of rules and fictional characters. Completeness refers to the degree of consistency
structuring the world’s ecosystem. A more complex world will make completeness
progressively more complex. As such, the details are just as important as the whole. Indeed,
the details must be in keeping with the general structure of the pre-established world. It is
therefore important to meticulously follow the general structure of the system of rules that
were put in place. Going along with imaginary systems, inventions and rules requires a
temporary suspension of disbelief. For example, in the recent TV series “West World”
(2016), the participants in a Far West simulation can break many actual laws of the real world.
We accept the actions of the characters because we know that this fictional world is populated
with androids. Consistency dictates even the smallest details, such as clothing texture,
weapon design, and the use of certain transportation modes. All these elements must, by their
presence, reinforce the logical and formal unity of the constructed world. It is thus no
coincidence if many of these worlds are geographically located on islands or spaceships. The
spatial confinement of the players in the story helps mark out the realm of action; indeed,
isolation justifies the world’s boundaries but also its social structures, architecture, costumes,
etc. The three rules of World Building thus allow for devising an imaginary world that abides
by its own internal rules and boundaries.
In 2008, Alex McDowell, Director of the World Building Institute (WBI) at the
University of South California Department of Cinematic Arts, summed up the approach as
follows: “World Building designates a narrative practice in which the design of a world
precedes the telling of a story; the richly detailed world becomes a container for narrative,
producing stories that emerge logically and organically from its well-designed core” [27].
Given this statement, it should come as no surprise that McDowell was Production Designer
for the film Minority Report (2002). When it came out, the film foreshadowed the future use
of dematerialized screens and multitouch technology. According to Wolf and McDowell, this
approach makes it possible to generate stories in accordance with a transauthorial and
transmedial logic [28]. Building fictional worlds also provides direction for urban planners
and architecture designers. In an article entitled “World Building and the Future of Media. A
case Study – Makoko 2036,” Laura Cechanowicz, Brian Cantrell and Alex McDowell relate
an experiment conducted by a hodgepodge group of students, architects, interactive media
designers, musicians, engineers, urban planners, animators, directors and artists. Drawing
inspiration from a specific location and culture, specifically the Lagos Lagoon in Makoko,
6
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
Nigeria, the group explored the articulation of different fictional scenarios, from both a story
and technology standpoint. In this interdisciplinary World Building context, the authors of
this last article wrote, “Any storyteller can weave a compelling narrative, but world builders
create storyworlds that support myriad stories by multiple storytellers across disparate
platforms — including those platforms that may not yet be named” [29]. It thus appears that
the world exists in itself, outside of any sort of media embodiment, and thus serves as a basis
for the infinite generation or proliferation of stories and substories. Whether
transdisciplinary, transmedia or transauthor, any form of subcreation constitutes a world that
guides the expressive use of digital technologies. Clearly, given its all-encompassing nature,
beyond its commercial and industrial applications, the World Building approach is
particularly fertile ground for designing immersive experiences. Without getting into details,
Wolf distinguishes between three perceptual states: immersion, absorption and saturation. It
can be postulated that any current or future world, whether existing or yet to be designed, can
provide the basis for building a world obeying specific perceptual states. The visitor will thus
be plunged into the world (immersion) and captivated by the elements that make up the
storyworld (absorption). The world’s complexity makes it impossible for the individual to
experience all of its elements (saturation). Outside of any particular narrative structure, such
as a story arc, being immersed in a fictional creation as a witness or participant brings into
play the prime characteristic of digital storytelling, which is interactivity. A world existing
and functioning according to its own specific organizational rules allows the person
immersed in it to experience the fictional world according to their own point of view. It is
worth reiterating that the world determines the preconditions for the emergence of multiple
stories. Like a virtual world persisting in the cloud of digital data and evolving according to
its own unique logic, by existing as a fiction, a world, an environment, a culture, etc., World
Building is an immersive construct. On this topic McDowell, who primarily expresses this
idea in talks and presentations, mentioned in a radio interview: “I think VR is just an
opportunity to think about a completely different medium. Once you take that step and you
get technology that allows you to immerse yourself completely in an unfamiliar environment,
my sense is that this environment will provoke the stories that it requires” [30]. Hence, the
environment engenders its own narrativity, by the mere fact of its existence. What is at play
is therefore no longer a narrative structure, but an invented world that exists within certain
boundaries and is organized according to its own specific rules.
From a practice perspective, there are many design rules and points of entry when
producing a (story)world. For example, one can use the language of the inhabitants or the
geography of the story as a starting point. Moreover, it is essential to determine the historical
period (be it real or fictional), the planet, the city, architecture, the building, the institutions,
etc., just as it is necessary to determine the characters’ emotional characteristics, interaction
rules, personal histories, powers, and so forth. Indeed, it is important for all these elements
to be determined since they lay the groundwork for viewers to experience a degree of
expectation within the context of the story. There is no specific methodology or recipe for
designing a “storyworld”; rather, an infinite number of points of entry can be considered. The
same is true of games, i.e., there are no specific rules in designing a storygame [31]. However,
among all the available choices (characters, languages, cultures, institutions, etc.), one design
variable is paramount: spatiality, which includes geography, architecture, location and space.
In other words, spatiality, whether imagined in a literary or virtual form, is at the core of the
practice of World Building. In fact, the practice of environmental storytelling is based
precisely upon this affirmation.
7
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
2.3 Environmental storytelling
Inspired by the storytelling strategies used in the architectural design of theme park
attractions, the field of environmental storytelling is premised on setting up the conditions
for stories to emerge by first constructing the space. In the words of Don Carson, former
theme park experience designer, “One of the trade secrets behind the design of entertaining
themed environments is that the story element is infused into the physical space a guest walks
or rides through” [32]. In this context, when it comes to both real-world and virtual
environments, the practice comes down to articulation strategies. Carson also discusses the
conditions for dialogue between real-world and virtual spaces. Based on an analysis of his
design experience, he defines how the specific narrative structure of theme parks’
architectural design helps better understand the field of environmental storytelling. Along
similar lines, Henry Jenkins mentions in his text “Game Design as a Narrative Architecture”
that spatiality is central to designing games, whether tabletop or virtual: “Game designers
don’t simply tell stories; they design worlds and sculpt spaces” [33]. Jenkins goes a step
further by maintaining that the field of “environmental storytelling creates the preconditions
for an immersive narrative experience in at least one of four ways: spatial stories can evoke
pre-existing narrative associations; they can provide a staging ground where narrative events
are enacted; they may embed narrative information within their mise-en-scene, or they
provide resources for emergent narratives” [34]. Hence, the imbrication of places, be they
open or closed, affords the interacting player or actor some agency enabling them to create
their own narrativity. On the website “Gamasutra,” Bart Stewart states that “Environmental
storytelling is the art of arranging a careful selection of the objects available in a game world
so that they suggest a story to the player who sees them,” adding that, “Environmental
storytelling is less direct. Instead of explicitly describing events, environmental storytelling
shows the final outcome of a sequence of events, then it invites players to make up their own
stories about what happened to cause that outcome” [35]. This goes to show that the player’s
agency conditions their narrative experience of the space. On this topic, Mary Laure Ryan
distinguishes between different relationships to space from a strategic and emotional point of
view. The space is emotional in that “the experience of that space is associated with emotional
values,” and Ryan goes on to say, “Whereas the emotional design of the space constructs the
subject by the relationship he or she has with their ambient environment, strategic design
constructs the subject as possessing agency” [36]. Hence, storytelling is also based on space
as perceived from an aesthetic as well as functional standpoint. This means it is important to
determine whether the subject will recognize a familiar (emotionally resonant) environment
or a strategic one that serves functional goals. More specifally, the strategic space can be in
the form of a sandbox, i.e., it can be viewed as pure indeterminacy, or in the form of level
design that responds to the strategic requirement of a particular affordance of space according
to the construction of the player’s experience.
3 World Building: Full Dome research-creation projects
The World Building hypothesis emerged progressively and spontaneously as an inescapable
solution in the context of creating a number of research-creation projects in the Full Dome
format. First, the Environmental Storytelling hypothesis structured the scripting of the
projects Re-Génération (2015) and Les Planètes (2018). The Illumination Frankenstein
(2018) project was guided by the World Building hypothesis throughout, from ideation all
the way up to performance in November of 2018 [37].
8
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
Fig. 1. Re-Génération. Société des Arts Technologiques. Cité des sciences et de l’industrie (2015)
To properly understand the evolution of the hypothesis of using the World Building
theoretical framework, it is worth providing an overview of key moments in this evolution.
In 2008, as part of its cultural activities on the heels of Cop21 in Paris, the Société des Arts
technologiques was commissioned with creating an immersive work on the theme of climate
change. David McConville, then Director of the Buckminster Fuller Institute, responded by
imagining a new form of transmission based on relationships between the human body and
the environment via embodiment approaches. His idea was to use environmental
visualizations from NASA, simulations of living organisms and synthetic environments to
convey a certain vision of climate change through the exploration of the sensory dimensions
of the immersive medium that was put in the service of the narrative. The intent was to
arrange multiple metaphors in order to create a sensory experience demonstrating that human
beings are not separate or isolated from the ecosystems they are changing and transforming.
The project, entitled Re-Génération, observed the natural world from the standpoint of the
time sequence of day and night, the tides, and land movements, all viewed as one giant breath
extending across the globe [38].
9
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
The narrative scenario, inspired by the Gaia hypothesis, consisted in identifying
different levels of knowledge emerging from studies on the broad theme of regeneration. The
second level involved writing a narrative presented in the form of a meditation on climate.
The story behind the project, written by David McConville, was scripted on the basis of a
succession of metaphors drawing links between the balance of ecosystems and the human
body. The narration, or storytelling, was thus built on a succession of continually shifting
environments. The structure was guided by the observation that narrativity, in a Full Dome
situation, no longer plays out through images but through the environment. Once
conceptualized, each environment in this scenario was produced by several teams composed
of artists and directors. It is important to point out that while it was performed in the
immersive format of a Full Dome projection, the project was created using a transauthor
approach.
Fig. 2. Les Planètes. Société des Arts Technologiques (2018). Photo: Nathanael Corre
The second experience where the environmental storytelling conceptual framework
was used was entitled Les Planètes (2018) [39]. Globally speaking, this project consisted in
a series of environments that responded to music and that could, in keeping with a
Transmedia aesthetic, be transposed to single-screen, multi-screen, panoramic, virtual-reality
and Full Dome formats. The first version of the project was presented as an immersive Full
Dome concert generated entirely in real time. The project involved creating a light-piano that
entertained a visual dialogue with pianist Louise Bessette regarding a new interpretation of
Les Planètes by composer Walter Boudreau. The light-piano enabled the projection of
dynamic constellations of small spheres scattered across the dome’s projection space. The
visual shapes were based on analysis of different sections of the musical score. Each section
was considered an abstract subworld, as it was divided into as many planets as exist in our
solar system. Using environmental storytelling directly resulted in generating the immersive
experience through a series of interactional rules between virtual objects. Thus, each
environment explored an array of music and sound visualization strategies, and the
succession of visualizations yielded a form of abstract storytelling or sensory storytelling. In
this context, environments of abstract shapes constituted spaces of mental projection (similar
to huge Rorsach ink blots) in which viewers drew their own meaning from the aesthetic
10
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
shapes as they allowed their attention to wander. Along the same lines, this approach
fundamentally relying on transmedia had already been explored in two previous projects
carried out in collaboration with the composer Alain Thibault, namely White Box (2011) and
Enigma (2017-2019) [40].
Fig. 3. Illumination Frankenstein. Société des Arts Technologiques (2018). Photo Yan Breuleux
The third Full Dome World Building experience that used environmental storytelling
was based on the fictional world of Frankenstein as imagined by Marie Shelley, whose novel
came to popularity in movie theatres. The project consisted in an immersive lecture by
sociologist Céline Lafontaine [41]. In this case, we adopted the opposite approach by
presenting the elements which, in the actual romantic era, shaped the world of
Dr. Frankenstein and his creature. Thus, Illumination Frankenstein takes the shape of a
spatialized story to present a succession of environments composed of archives from the
complex world that gave rise to Shelley’s scientific vision. This brings up the question: What
was the real world that inspired the novel? For example, a study of the novel reveals that
Frankenstein was put together from human and animal corpses, and was vegetarian.
An interpretation using a World Building approach could allow for a new modelling of the
creature, using details provided by the author. Indeed, studying the novel and comparing its
different versions would allow for composing a coherent virtual world in keeping with a
World Building logic, if we examine the story in its minute details. The immersive lecture
helped expose the actual world that give birth to the novel Frankenstein. Following a
narrative structure of environmental storytelling, composed as a journey within a myriad of
immersive environments, the project was, fundamentally, underpinned by an understanding
of different ways of applying the World Building model. The idea was also to show how,
“Two hundred years after its literary birth, the prescient light that this prophetic novel shed
11
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
on the beginnings of modernity still shines today and invites a reflection on the conquest,
manipulation and devastation of the living world” [42]. In other words, by its very structure,
the project attempted to draw links between the primary and secondary worlds of
Frankenstein, which are at once romantic and modern.
3.1 World building: virtual reality research-creation projects
Fig. 4. PeterPan360°, Bruno de Coninck, NAD - UQAC (2019).
The theoretical framework of World Building has helped guide the research and structure the
practice of a number of students at the NAD school pursuing research-creation in the course
of their master’s studies (Master 2). Bruno de Coninck has used different media to model a
virtual environment of the world of Peter Pan by blending a number of different
interpretations of this world imagined by James Matthew Barrie in the early 20th century
(1904, 1911). Once the prototype was completed, it gave rise to a series of hypotheses for
transcoding the world in different formats. The last version consisted in generating a virtualreality world. To achieve this, Coninck compared different descriptions of this world and
then modelled a 3D narrative environment that served as a medium for various game
mechanics. Semantic blending of countless interpretations of the tale were used to add to,
complement and compose a complete virtual environment.
12
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
Fig. 5. Hybrid environment. Simon Therrien (2019).
For his part, Simon Therrien has imagined a hybrid world based on a cross between the world
of Franz Kafka’s novel Métamorphose and Jim Henson’s Labyrinth. His environmental
storytelling project has taken the shape of designing a new video game depicting a number
of fantastic environments as settings for action. Therrien’s research has primarily involved
defining a series of conceptual tools to nourish reflection on practice.
Fig. 6. Cités-Mondes. Samuel David (2019).
Samuel David, with his project Cités-Mondes, has worked on environmental
storytelling by observing the creative dimensions of 3D modelling while adopting a reflexive
practitioner approach. Thus, David has created an animated universe determined by fictional
13
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
rules for creating animated ecosystems. In reality, he modelled a complete world that
functions according to its own internal rules. Objects conceived in this way could enable
writers to imagine stories, intrigues, tensions, etc. This is because the environment serves as
the medium for an imaginary narrative space. In a different vein, Eric Filion (also known by
the pseudonym Vj Nokami) has drawn inspiration from the study of the specific environment
of Iceland to make an immersive film on the unique sense of Solastagia described by
Glen Albrecht [43]. The goal of Filion’s project was to help the viewer visualize this unique
emotion, characterized as a sort of anguish provoked by the gradual shifting and
transformation of our natural environment.
Conclusion
The theoretical framework of World Building offers a springboard for analyzing the practice
of environmental storytelling in the field of Spatial Augmented Reality (SAR), and more
specifically in the Full Dome format with its succession of environments. In sum, our analysis
of projects upholds that the World Building approach structures the creation of storyworlds
and storygames that can be adapted to multiple apparatuses and installations. As such, and
viewed from an art and design standpoint, this approach makes it possible to structure
storytelling projects while taking into account a diversity of projection platforms. To
illustrate this assertion, one could mention for example the fact that the overarching theme
of a museum exhibit can constitute a storyworld. In this context, the different artifacts are
produced using a transmedia approach. After conducting our analysis, we have also noted
that the world’s organization is based on collaboration between players in the academic and
institutional milieus, and with professionals from multiple disciplinary backgrounds. In our
view, the transauthor dimension is especially relevant as it helps structure the contributions
of all the players involved. Another interesting fact is that environmental storytelling can be
embodied both in the exhibit’s scenography and in the media design of parallel worlds, virtual
reality experiences, etc. In this context, the experiential design approach adapts particularly
well to the storyworld formulated by all of the players, especially since all historical periods
obey rules pertaining to story structure and construction. Hence, the characteristics of World
Building, as formulated by Wolf, provide a theoretical framework for organizing the
relationship between art, design, technology and storytelling, by following a certain logic.
Whether in museology, documentaries, films or digital art projects, this relationship helps
define a new form of transmedia storytelling. In all cases, the process is based on a series of
critical steps. The first is to immerse oneself in the historical world, and the second, to
imagine a map, a place, an environment. However, the elements entailed by each step vary
depending on the project; depending on the case, one may need to include interactions with
the characters, the era, technologies, and common objects. From this world, this subcreation,
all media choices become possible.
In our case, in the context of our recent research and that of our students, the worlds
have, globally speaking, been based on the use of an existing story. The underlying research
makes it possible to imagine a certain historical environment and to convey knowledge
through a given storyworld, thus giving visibility to the paradigm on which the knowledge is
based. In this general context, the theoretical framework of World Building also transforms
the paradigm from which the project is created. The story plays out through the environment
in which the action occurs, but partakes in what Bielskyte describes as a storyworld
experience. The storyworld experience is developed in the field of VR films via close
collaboration between four key individuals: the director, the production manager,
the engineer (or technical artist) and the interaction designer. Each has a specific role to play:
14
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
1- the director is in charge of the story and the emotional design, 2- the production manager
carries out environment-related tasks, whether in terms of architecture or narrative, 3- the
engineer fleshes out how the story will be conveyed, and 4- the interaction designer
determines mechanics and logics of interacting with the world. This collaboration somewhat
diminishes the director’s status as a demiurge. As the designer of the experience, the director
adapts to and collaborates with an existing world and the other team members. The engineer
now participates in storytelling, and the interaction designer (as a specialist in relationships
with the world) determines the strategies the user will use to interact with the world. This
said, it is important to understand that this structure brings into play not just skills, but also a
space of horizontal communication. Indeed, each player in the process observes the issues
surrounding the story, the environments, the interaction and the technological means, all from
the characteristic angle of their particular profession. Thus, in a World Building approach,
each role involves exploring the conditions for bringing the story and environment into
existence (from both a modelling and technological standpoint) by connecting them with
different forms of interaction.
The project Dans la lune (2019), currently being developed under the direction of
Marc Caro in collaboration with a group of students and teachers from the École Supérieure
d’architecture de Nantes, consists in a VR film experience inspired by a hybridization of the
world of the film magician Georges Méliès and the retrofuturistic aesthetics of Jules Verne,
whose Machines de Nantes are an outstanding example of this aesthetic [44].
The construction of the film’s immersive décor is a central part of articulating the experience.
Consistent with the World Building framework, constructing this film involves the
combination of a world and the building of a storytelling environment where the action will
take place. In this context, the theoretical framework of World Building serves as a
foundation to analyze the project’s narrative dimensions as well as its creation process.
In light of current trends in immersive technologies, it appears evident that the
transmedia premise of storyworlds allows for adapting stories across different forms of
technological environments. One avenue to explore would be to look at how the model
applies to the four worlds of storytelling as set forth by Scott A. Lukas in 2013 [45],
which was linked to the theory of possible worlds as studied by Wolf and Ryan. [46], [47]
According to Lukas, the primary world is the realm of reality (a), the second is imaginary
(b), the third is experiential (c) and the fourth is the world of guests (d). These four
perspectives constitute as many trades and complementary endeavours and approaches. The
first world consists in the materiality of built structures, from an urban planning, architectural
and industrial standpoint. The second world, that of the imagination, stems from literature,
fiction and all forms of visualization (maps, models, patterns, language, etc.). The third
world, experiential, comprises technological environments such as those described in this
article, and the design of visitor interactions. The fourth world, or visitor world, corresponds
to the perspective of the receiver. Clearly, our field is located in the humanities and social
sciences. How can the unique perspective of the individual be understood in the context of
the interrelationships between all these worlds? Moreover, what knowledge would help to
include guests in the storytelling process? In our case, it is clear that we are in an
interdisciplinary context. The next step of the present study would be to establish a design
methodology making it possible to create a narrative continuum between these different
worlds, beyond their technologies. This brings into play the hypothesis of World Building as
a continuum of mixed reality or cross-reality—not technological, but rather narrative and
experiential, in nature.
15
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
Miller, Carolyn Handler. Digital storytelling: A creator’s guide to interactive
entertainment (Focal Press, 2014), p.14.
Patent of the Telesphere Mask invention:
[https://patents.google.com/patent/US2955156A/en], page accessed on February 6,
2019.
Sutherland, Ivan E. "The ultimate display." Multimedia: From Wagner to virtual
reality (1965): 506-508.
A detailed chronology is available in chapter 1 of the following book: Alexander,
Bryan. The New Digital Storytelling: Creating Narratives with New Media--Revised
and Updated Edition. (Abc-clio, 2017).
Murray, Janet Horowitz. Hamlet on the Holodeck : The Future of Narrative in
Cyberspace (New York: Free Press, 1997).
Balet, Olivier, and Gérard Subsol, Torguet, Patrice. Virtual Storytelling; Using Virtual
Reality Technologies for Storytelling: International Conference, ICVS 2001, Avignon,
France, September 27-28, 2001, Proceedings. Vol. 1. (Springer, 2001). P.2.
International Conference for Interactive Digital Storytelling :
[https://icids2018.scss.tcd.ie/], page accessed on February 6, 2019.
New York Times: [https://www.nytimes.com/2019/01/25/theater/hamlet-virtualreality-google.html], page accessed on February 6, 2019.
Future of Storytelling: [https://futureofstorytelling.org/], page accessed on February 6,
2019.
MIT Open Documentary Lab: [http://opendoclab.mit.edu/], page accessed on February
6, 2019.
Ropert-Dupont, Émilie. Journalisme et réalité virtuelle: Émotion ou information?.
(Éditions L’Harmattan, 2017).
Centre PHI: [https://phi-centre.com/en/event/fost-2015-en/], page accessed on
February 6, 2019.
The new storytellers: [https://phi-centre.com/en/event/the-new-storytellers-5-en/]
Douglas Trumbull : [https://www.youtube.com/watch?v=47wO-Az22v0], page
accessed on February 6, 2019.
Gödde, Michael, Frank Gabler, Dirk Siegmund, and Andreas Braun. "Cinematic
Narration in VR–Rethinking Film Conventions for 360 Degrees." In International
Conference on Virtual, Augmented and Mixed Reality, pp. 184-201. Springer, Cham,
2018. See: Tricart, Celine. Virtual Reality Filmmaking: Techniques & Best Practices
for VR Filmmakers. Taylor & Francis, 2017. Also worth mentioning is: Bucher, John.
Storytelling for virtual reality: methods and principles for crafting immersive
narratives. Focal Press, 2017.
Milgram, P., Takemura, H., Utsumi, A., & Kishino, F. (1995, December). Augmented
reality: A class of displays on the reality-virtuality continuum. In Telemanipulator and
telepresence technologies (Vol. 2351, pp. 282-293). International Society for Optics
and Photonics.).
The New Zealand VR/AR Association (NZVRARA). Virtual gets real. The Explosion
of Cross Reality in New Zealand, report one, September 2017.
Breuleux, Y. Narrativité immersive pour Vr et FullDome. Art & Technologie. Effects
Mtl, 2017. Palais des congrès de Montréal.
16
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
19. Bimber, Oliver, and Ramesh Raskar. Spatial augmented reality: merging real and
virtual worlds. AK Peters/CRC Press, 2005.
20. Boni, Marta, ed. World Building: Transmedia, Fans, Industries. Amsterdam University
Press, 2017.
21. World Building the Marvel Way, Bill Rosemann, Marvel Games (2017):[
https://youtu.be/p3J2ZuOmFWw?t=120 ], pages consulted on February 6, 2019. Also
worth mentioning is an article on the links between the Marvel empire and the World
Building approach: Jeffries, Dru. " The Worlds Align." World Building Transmedia,
Fans, Industries (2017): 287.
22. Monika Bielskyte, The New Storytellers II: World-building: Best Practices for
Creating Story Worlds: [ https://youtu.be/jtr0AZK64Do?t=1093], page accessed on
February 6, 2019.
23. Wolf, Mark JP. Building imaginary worlds: The theory and history of subcreation.
Routledge, 2014. P.19.
24. Wolf, Mark JP. Building imaginary worlds: The theory and history of subcreation.
Routledge, 2014. P.39.
25. Wolf, Mark JP. Building imaginary worlds: The theory and history of subcreation.
Routledge, 2014. P.44.
26. Monika Bielskyte, The New Storytellers II: World-building: Best Practices for
Creating Story Worlds: see (24:42): [https://youtu.be/jtr0AZK64Do?t=1093], page
accessed on February 6, 2019.
27. The World Building Institute: [http://worldbuilding.institute/about], page accessed on
February 6, 2019.
28. Wolf, Mark JP. Building imaginary worlds: The theory and history of subcreation.
Routledge, 2014. P.28.
29. Cechanowicz, Laura, Brian Cantrell, and Alex McDowell. "World Building and the
Future of Media: A Case Study-Makoko 2036." IEEE Technology and Society
Magazine 35, no. 4 (2016): 28-38.
30. Voices of VR: Alex McDowell on World Building in Storytelling, [voicesofvr.com,
podcast 309] (2016), page accessed on February 6, 2019.
31. Montfort, Nick. "Interactive Fiction as
‘Story,’’Game,’’Storygame,’’Novel,’’World,’’Literature,’’Puzzle,’’Problem,’’Riddle,’
and ‘Machine.’." First Person: New Media as Story, Game, and Performance (2004).
32. Don Carson:
[https://www.gamasutra.com/view/feature/131594/environmental_storytelling_.php],
page accessed on February 6, 2019.
33. Jenkins, Henry. "Game design as narrative." Computer 44, no. 53 (2004): 118-130.
34. Jenkins, Henry. "Game design as narrative." Computer 44, no. 53 (2004): 118-130.
35. Bart Stewart, Gamasutra:
[https://www.gamasutra.com/blogs/BartStewart/20151112/259159/Environmental_Stor
ytelling.php], page accessed on February 6, 2019.
36. Ryan, Marie-Laure. "L’expérience de l’espace dans les jeux vidéo et les récits
numériques." Cahiers de Narratologie. Analyse et théorie narratives 27 (2014). P.2.
37. Re-Génération: [http://www.uqac.ca/portfolio/yanbreuleux/recherche-creation/], page
accessed on February 6, 2019.
38. See: [http://sat.qc.ca/cop21], page accessed on February 20, 2019.
17
SHS Web of Conferences 64, 00003 (2019)
EAEA14 2019
https://doi.org/10.1051/shsconf/20196400003
39. See: [http://sat.qc.ca/fr/planetes], page accessed on February 20, 2019.
40. See: [https://vimeo.com/purform/enigma], page accessed on February 20, 2019.
41. See: [https://socio.umontreal.ca/repertoire-departement/vue/lafontaine-celine/], page
accessed on February 20, 2019.
42. Illumination Frankenstein: [https://sat.qc.ca/fr/evenements/illumination-frankenstein],
page accessed on February 20, 2019.
43. The theses of Bruno de Koninck, Simon Therrien, Samuel David and Eric Filion will
be made available during the summer of 2019 on the following website:
[https://constellation.uqac.ca/], page accessed on February 20, 2019.
44. The project is currently under production:
[https://www.instagram.com/architecture.representation/],
[https://www.instagram.com/architecture.representation/] and
[https://architecturerepres.wixsite.com/decor360], page accessed on February 20, 2019.
45. Lukas, Scott. The immersive worlds handbook: designing theme parks and consumer
spaces. Focal Press, 2012. P.230.
46. Bell, Alice, and Marie-Laure Ryan, eds. Possible Worlds Theory and Contemporary
Narratology. University of Nebraska Press, 2019.
47. Wolf, Mark JP. Building imaginary worlds: The theory and history of subcreation.
Routledge, 2014. P.17.
18