Disciplining the body? Reflections on the cross disciplinary
import of ‘embodied meaning’ into interaction design
MARKUSSEN, Thomas
Available from Sheffield Hallam University Research Archive (SHURA) at:
http://shura.shu.ac.uk/522/
This document is the author deposited version. You are advised to consult the
publisher's version if you wish to cite from it.
Published version
MARKUSSEN, Thomas (2009). Disciplining the body? Reflections on the cross
disciplinary import of ‘embodied meaning’ into interaction design. In: Undisciplined!
Design Research Society Conference 2008, Sheffield Hallam University, Sheffield,
UK, 16-19 July 2008.
Repository use policy
Copyright © and Moral Rights for the papers on this site are retained by the
individual authors and/or other copyright owners. Users may download and/or print
one copy of any article(s) in SHURA to facilitate their private study or for noncommercial research. You may not engage in further distribution of the material or
use it for any profit-making activities or any commercial gain.
Sheffield Hallam University Research Archive
http://shura.shu.ac.uk
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
Disciplining the body?
Reflections on the cross disciplinary import of ‘embodied meaning’ into
interaction designi
Thomas Markussen, Kolding School of Design, Denmark.
Abstract
The aim of this paper is above all critically to examine and clarify some of the
negative implications that the idea of ‘embodied meaning’ has for the
emergent field of interaction design research.
Originally, the term ‘embodied meaning’ has been brought into HCI research
from phenomenology and cognitive semantics in order to better understand
how user’s experience of new technological systems relies to an increasing
extent on full-body interaction. Embodied approaches to technology design
could thus be found in Winograd & Flores (1986), Dourish (2001), Lund (2003),
Klemmer, Hartman & Takayama (2006), Hornecker & Buur (2006), Hurtienne &
Israel (2007) among others.
However, fertile as this cross-disciplinary import may be, design research can
generally be criticised for being ‘undisciplined’, because of its tendency
merely to take over reductionist ideas of embodied meaning from those
neighbouring disciplines without questioning the inherent limitations it thereby
subscribe to.
In this paper I focus on this reductionism and what it means for interaction
design research. I start out by introducing the field of interaction design and
two central research questions that it raises. This will serve as a prerequisite for
understanding the overall intention of bringing the notion of ‘embodied
meaning’ from cognitive semantics into design research. Narrowing my
account down to the concepts of ‘image schemas’ and their ‘metaphorical
extension’, I then explain in more detail what is reductionistic about the notion
of embodied meaning. Having done so, I shed light on the consequences this
reductionism might have for design research by examining a recently
developed framework for intuitive user interaction along with two case
examples. In so doing I sketch an alternative view of embodied meaning for
interaction design research.
Keywords
Interaction Design, Embodied Meaning, Tangible User Interaction, Design
Theory, Cognitive Semiotics
Interaction design is commonly viewed as a dominating trend in HumanComputer Interaction (HCI) that focuses on designing user experiences with
technology. With the advent of tangible and physical computing, there have,
over the last decade or so, emerged new opportunities for interaction
designers to create technological systems or products that engage nearly
every aspect of human experience: emotional feelings, vision, movement,
gestures, and all sorts of interrelations thereof. As has been pointed out,
084/1
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
interaction is simply no longer unfolding by means of viewing and clicking on
graphical icons on a screen, but is embedded into an augmented reality inbetween image, object and the surrounding space (cf. Winograd, 1997; see
fig. 1).
In addition to this extended user interface (sometimes referred to as ‘tangible’,
‘graspable’, ‘intuitive’, and so on), interaction design also represents the
proliferation of information technology into new territories of our everyday life.
As a consequence, the attention of interaction designers is shifting from mere
usability, functionality, and efficiency to playful, explorative and emotional
interaction.
Following from this radical design evolution, current HCI research is struggling
to increase understanding of the emergent new forms of user interaction. In
particular, interaction design poses two challenges to HCI research, which
must be addressed. First, there is an urgent need to acquire more knowledge
of the new cross-modal experiences that characterize full-body interaction
with technological user interfaces. Related to this is the second challenge,
which concerns the question of how users are able mentally to interpret novel
technology use for practical purposes, play and social interaction. Since
interaction design often implies the use of technology in unfamiliar contexts,
users cannot always rely on their previous experiences and background
knowledge, but have to create meaning and understandings of their own or
collaboratively. Clearly, experience and meaning construction are mutually
intertwined in user interaction, so these two topics must be dealt with as coconstitutive aspects of the same problem.
By picking up recent theories of the ‘embodied meaning’ and ‘embodied
interaction’ from phenomenology and cognitive semantics, current HCI
research has in fact made considerable progress in understanding this
problem. Basically, these theories share the underlying assumption that
people’s subjective, felt experience of their bodies in action play a key role in
shaping human cognition and language (cf. Gibbs, 2005). However, to
understand the full heuristic potential of this cross-disciplinary import, it is
necessary first to pinpoint the knowledge gap in traditional HCI that the notion
of embodiment is generally believed to have filled in.
From Cognitive Models to Embodied Meaning in HCI
It is widely acknowledged that traditional HCI and product semantics simply
offer a too limited understanding for interaction design (see e.g. McCarthy &
Wright, 2004; Dunne, 2006). Generally, these frameworks tend to treat user
interaction as a disembodied abstract recognition process, where cognitive
models are matched more or less unambiguously against the visual form and
surfaces of an object (Norman, 1988; Krippendorff, 1989). A cognitive model is
usually conceived to be a product of people’s long-term memory, which has
been stored through repeated actions and experiences in the past. Thus, for
instance, when I see the sharp contours of an oblong item in my kitchen, I
automatically fetch the cognitive model of a ‘knife’ without even being
aware of it. I simply recognize the object by relating its visual form to my
previous experiences with knifes. The KNIFE MODEL provides me as it were with
tacit knowledge for the appropriate use of the object under the given
circumstances (e.g. cutting vegetables and meat, peeling skin of tomatoes,
084/2
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
etc.). In this way cognitive models are to a large extend responsible for the
smooth flow of everyday activities and at the same time for freeing cognitive
resources for other purposes such as social interaction (e.g. conversing your
partner while preparing a meal).
However, this conceptual framework is not equipped with sufficient
explanatory power for dealing with central aspects of interaction design (cf.
Markussen, 2008). For instance, principles of tangible and graspable user
interaction do not necessarily obey the same kind of rules as visual recognition
processes. Also, by embedding computational resources in physical
environments users are sometimes confronted with hybrid functionalities that
they have never experienced before. In such cases, either there simply is no
cognitive model available for immediate recognition, or existing cognitive
models must be reorganized or invented anew to fit the novelty and
innovative aspects of technology use. In fact, it is an acclaimed goal of many
interaction designers to create “aesthetic experiences comprised of both a
bodily sensation and an intellectual challenge” (Petersen et al., 2004). Or to
put it in other words: interaction design is no longer just used for fulfilling well
prescribed goal-oriented tasks, but often requires that user’s build new
concepts and learn through their bodily interaction with technology.
The related ideas of ‘embodied meaning’ and ‘embodied interaction’ have
been brought into HCI research from phenomenology and cognitive
semantics in order to account for this subtle interplay of bodily interaction and
meaning construction (see e.g. Winograd & Flores, 1986; Dourish, 2001; Fels,
2001; Klemmer, Hartman & Takayama, 2006; Hornecker, 2005; Hornecker &
Buur, 2006; Hurtienne & Israel, 2007). Contrary to cognitive models, embodied
meaning is not to be understood in terms of abstract mental representation,
but as primitive spatial patterns giving order and coherence to our experience.
As mentioned above, these patterns are held to be responsible for the internal
organization of meaning “at more abstract levels of cognition” (Johnson, 1987,
pp. xix-xx). In this transfer of lower-level structures from an experiential domain
onto higher-level domains of abstract meaning, imaginative and metaphoric
capabilities of the human mind are assumed to play a key role. This potentially
widens the notion of meaning construction in design from recognition to
include learning and the invention of novel interpretations.
In the following I look more closely on how this notion of embodied meaning in
one particular instance has been developed into a new theoretical
framework for interaction design.
Embodied Meaning and Metaphorical Extensions in Intuitive
User Interaction
Most recently, Hurtienne & Israel (2007) has made a plea for using cognitive
semantic theories of embodied meaning in the description of basic levels of
intuitive user interaction. More specifically, they ground their framework on
Lakoff & Johnson’s Conceptual Metaphor Theory (Lakoff & Johnson, 1980;
Johnson, 1987). In order to give an overview of these levels the authors start
out by presenting a model of the ‘continuum of knowledge’ that they believe
cover intuitive interaction (see fig. 1):
084/3
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
Fig 1. The Knowledge Continuum for Intuitive User interaction.
Adapted from Hurtienne & Israel (2007)
The first and most basic level concerns the body’s innate knowledge acquired
through reflexes, instinctive behaviour and other neurobiological skills. The next
‘sensor-motor level’ consists of general knowledge structures arising in infancy
from developmental learning and interaction with the physical world.
“Children learn for example to differentiate faces; they learn about
gravitation; they build up concepts for speed and animation” (Hurtienne &
Israel, 2007, p. 128). Then there is the ‘culture level’ of knowledge, which is
sensitive to the place and location where people live. For instance, it makes
good sense to build interfaces upon the desktop metaphor in Western cultures,
but this happens not to be the case worldwide. Finally, Hurtienne & Israel
distinguish a ‘specialist level’ of knowledge that consist of expertise acquired
through one’s education, profession or hobby.
Having done so, Hurtienne & Israel further suggest that the cognitive semantic
concepts of ‘image schema’ and ‘metaphorical projection’, laid out by
Lakoff & Johnson (1980, 1999), Lakoff (1987) and Johnson (1987), may help us
to better understand the interweaving of those levels in tangible user
interaction with technology. To see what is at stake here, let me just
summarize what is meant exactly by these two concepts.
The Relation between Sensor-motor and Socio-cultural levels
of Meaning in Conceptual Metaphor Theory
In The Body in the Mind (1987) Johnson defines an ‘image schema’ as “a
recurrent, dynamical pattern of our perceptual interaction and motor
programs that gives coherence and structure to our experience.” For instance,
there is the CONTAINER schema for in-out orientations in space, the PATH
schema for directions in space, and over 2 dozens other schemas that in a
similar vein structure how the world show up for us (cf. Gibbs & Colson, 2006).
What is particularly interesting is that Johnson (1987, pp. 30-31) in fact employs
everyday design-related activities to account for how image schematic
structures come into existence:
Consider, for example, only a few of the many in-out orientations that
might occur in the first few minutes of an ordinary day. You wake out of
084/4
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
a deep sleep and peer out from beneath the covers into your room. You
gradually emerge out of your stupor, pull yourself out from under the
covers, climb into your robe, stretch out your limbs, and walk in a daze
out of the bedroom and into the bathroom … You reach into the
medicine cabinet, take out the toothpaste, squeeze out some
toothpaste. (my obliteration)
The idea here is that through such simple manipulations of objects and bodily
actions, invariant spatial structures become gradually extracted and
entrenched in our long-term memory system in the form of image schemas. In
all of the above-mentioned experiences there are thus an underlying image
schematic structure of containment, which can be represented
diagrammatically as in fig. 2:
LM
TR
Fig. 2 CONTAINER schema
The encircled ‘landmark’ (LM) is illustrating how in all of the above examples
boundedness is perceived in relation to a container serving as a more or less
stable reference point. In the waking out of a deep sleep, it is the sleep; in
peering out from beneath the covers, it is the covers; in walking out the
bedroom, it is the room, and so forth. On the other hand, that which moves is
organised experientially as a trajector (TR) moving along a path, either out of
(the arrow pointing to the right) or into the container (the opposite arrow). This
indicates that most often the image schema involves some kind of force
dynamics as well. In the waking up sequence, our consciousness is
experienced as moving (as TR) out from a dazing to a waking state; in peering
out from the covers and in climbing into the robe, it is respectively our gaze
and whole body that operate as trajectors. Apart from such IN-OUT
orientations, there are schemas for UP-DOWN, LEFT-RIGHT, NEAR-FAR, and
numerous other experiential relations in space (Johnson, 1987, p. 126).
It is important to note that the image schemas are not found as such in
objective physical reality, but act rather as spatial structures that we are able
to project onto inanimate or non-physical entities. Just think about the
aforementioned action of stepping out of bed and climbing into your robe.
Beds and robes do not possess an inside and an outside by themselves. This
organization is utterly dependent on a subjective embodied experience of
moving from one location to another location in space or feeling oneself
being surrounding by a soft piece of textile. Moreover, these spatial structures
must somehow be projected onto the objects at hand.
Metaphorical Projection
It is this figurative extension of structures from the sensor-motor level onto
higher semantic levels in thought and language that is being referred to as
‘metaphorical projection’. By studying an impressive range of verbal
084/5
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
expressions and sentences, Lakoff & Johnson (1980) found out that image
schemas in fact act as ‘morphological skeletons’ ensuring the unity of
conceptual meaning. Consider again the CONTAINER-schema, which is
crucial for our ability to comprehend events, states, or our being among other
people meaningfully. Thus, we retrieve this schema for conceptualizing
economic transactions (“The salesperson closed the deal”), emotions (“I’m in
love!”), time (“you need to be here in five seconds!”), social relations (“He’s
an outsider”), and so on.
In Lakoff (1987, p. 283) the whole idea of how image schemas are
metaphorically elaborated into conceptual meaning structures is summarized
by what he refers to as the Spatialization of Form Hypothesis:
Strictly speaking, the Spatialization of Form hypothesis requires a
metaphorical mapping from physical space into a ‘conceptual space’.
Under this mapping, spatial structure is mapped into conceptual
structure. More specifically, images schemas (which structure space) are
mapped into the corresponding abstract configurations (which structure
concepts). The Spatialization of Form hypothesis thus maintains that
conceptual structure is understood in terms of image schemas plus a
metaphorical mapping.
From this quote, it appears that metaphorical projections of image schemas
rely on a unidirectional source-target construal principle as depicted in fig. 3:
Fig. 3
However, as Kimmel (2005) among others has convincingly argued this way of
thinking entails a certain reductionism that we need to be aware of. First, the
Spatialization of Form Hypothesis presupposes that embodied image schemas
are ascribed ontological and functional primacy, whereas the impact of
cultural and contextual factors in shaping conceptual structure is reduced to
a minimum.
Secondly, despite the fact that Johnson originally conceived image schemas
as being relatively malleable, so that they can be modified to fit many similar,
yet different, situations (Johnson 1987, p. 30), in many of his successors, there is
clearly a tendency to treat image schemas as pure idealizations, i.e. invariant
084/6
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
structures of long-term memory being uninfluenced by the local contextual
conditions. Yet I believe with Kimmel (2005) that there is a need for a concept
like ‘situated image schemas’ accounting for how image schemas are codetermined by their concrete embodied instantiation.
Thirdly, image schemas are most often conceived as simple and isolated
patterns in experience. However image schemas usually do not occur in
isolated fashion, but rather are experienced grouped as gestalts or wholes
involving the co-activation of several and even competing image schematic
structures. This observation was made more than ten years ago by Cienki
(1997), yet investigations of it seem only to be sparse. One of the things that
Cienki drew attention to was that the gestalt groupings of image schemas
cannot be understood merely as” a process of composition, given the nature
of a gestalt structure that it has properties not simply derivable from its parts”
(Cienki, 1997: 9). As we will see shortly, this phenomenon poses a challenge to
the source-target construal principle.
Mapping Image Schemas between the Physical and Digital
World
Interaction designers can no doubt benefit largely by turning towards
conceptual metaphor theory and cognitive semantics. After all, metaphors
have long been used in the design of digital media as a means for letting
users understand new forms of human-computer interaction in terms of what
they already know.
The recent interest of Hurtienne & Israel (2007), but also Lund (2003) in using
image schemas in the design of computer user interfaces should be seen
along this very same line. The overarching aim in designing with image
schemas is to enable users intuitively to comprehend abstract digital data in
terms of knowledge rooted in their direct perceptual and bodily interaction
with the physical world. However, if designers are not fully aware of the
inherent reductionism underlying the standard image schematic framework,
then it might cause some limited understandings of embodied meaning and
meaning construction at large. Consider, Hurtienne & Israel’s own fictitious
example – The Tangible Memories Box:
Fig. 4. The Tangible Memories Box (reproduced from Hurtienne & Israel, 2007)
084/7
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
The Tangible Memories Box consists of 6 trays each of which represent a time
period of 2 months. Each tray contains a memory item: an elephant for a visit
to a zoo in January, a shell for a holiday trip in March, a dice from a gaming
evening in May, and so forth. When connecting the box to a computer, folder
icons for each item group appear on a screen. Then by placing a special
cylinder in one of the smaller trays above, the display shows icons of each
digital file, which belong to the item in the tray (Hurtienne & Israel, 2007, pp.
132-33).
In their design of this box, Hurtienne & Israel describe how they made use of
essentially two organizing schemas – the CONTAINER and COLLECTION
schema respectively. At the largest scale, the box itself is a CONTAINER
mapped onto another (illusionary) CONTAINER, the screen. Inside the box, the
6 trays act as smaller CONTAINERS for a COLLECTION of objects, which
correspond to a COLLECTION of digital images stored in CONTAINERS on the
screen, viz. the file folders.
Fig. 5
I agree with Hurtienne & Israel that intuitive ease of use and comprehension is
likely to follow from this part of their artefact. Especially, it is worth noting how
the interface design cues users to extend their physical experience of
containment metaphorically into a coherent conceptualization of otherwise
scattered temporal events. More specifically, the computer interface plays on
the conceptual metaphor TIME IS A CONTAINER, which allows for making
different inferences such as MEMORIES ARE PHYSICAL OBJECTS IN A
CONTAINER. Recalling Bergson’s philosophical laments, one could say that this
spatialization process rectifies time, thereby making a fugitive and abstract
phenomenon intuitively graspable.
The Reductionism of Embodied Meaning in Interaction Design
However, there are a few uncertainties in Hurtienne & Israel’s argument
having to do notably with the intermediate object, i.e. the cylinder. Or more
precisely with their idea of how the image schematic structures involved in
manipulating the cylinder is mapped ‘physically’ onto the manipulation of
digital screen data.
084/8
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
There is two different image schemas involved in the cylinder interaction: (1)
rotating the cylinder is organized experientially according to the LEFT-RIGHT
schema; (2) placing the cylinder in an upright position is organized according
to the UP-DOWN schema. Now, Hurtienne & Israel propose a way of mapping
these structures onto two corresponding manipulations of digital data. Thus,
left-right rotation of the cylinder would correspond to left-right navigation of
the cursor from one image file to the next on the screen (fig. 6.1), whereas UPDOWN positioning of the cylinder should entail IN-OUT zooming on the image
(see fig. 6.2). Yet, this might not be as intuitive as it is presumed to be.
Source: Rotating
navigation
Schema: LEFT-RIGHT
Target: Linear cursor
Schema: PATH
Source: Upright Positioning
Schema: UP-DOWN
Target: ZOOMING
Schema: IN-OUT
Fig. 6.1, above: Rotation of cylinder mapped onto sequential navigation of
the screen cursor. Fig. 6.2, below: UP-DOWN positioning of cylinder mapped
onto IN-OUT zooming on an image
The Role of Cultural and Contextual Factors
Rotating a cylinder is something we typically associate strongly with IN-OUT
zooming (the dashed diagonal arrow) not sequential navigation along a
linear PATH. This associative link has become culturally entrenched as rotating
cylindrical lenses on cameras, telescopes and binoculars have been used for
this purpose for hundreds of years. And the current context, where users are
presented with a cylinder for image manipulation seems only to revive this link
in the users mind. So, unless users are able mentally to block this linkage to
their long-term memory and straightforwardly to couple rotation with
sequential navigation instead, this feature of the Tangible Memories Box is
likely, I would claim, to be experienced as counter-intuitive. Simply because it
runs counter to user’s contextually motivated expectations of use.
One of the main reasons why Hurtienne & Israel ignores the potential impact
from such cultural and contextual factors on the mapping of image schemas
084/9
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
is that they view innate and sensorimotor levels as functionally prior to these
factors. Admittedly, they do operate with both a cultural and a specialist level
of knowledge in their continuum model (see fig. 1 above). Yet, these levels
are not regarded as playing any constitutive role when designing with image
schemas. Thus, the authors write: “with our approach we would like to stick to
the original intention of using TUI [tangible user interfaces] and stay at the
sensorimotor level of our continuum of knowledge.” (Hurtienne & Israel, 2007,
p. 130). But it is an error to believe that designing with embodied schemas is
about ‘stripping off’ cultural layers in order to get down to the basics. Of
course, it is possible from a purely analytical point of view, but the Tangible
Memories Box itself more than indicates that it is reductionistic.
Situated Image Schemas
Another challenge for Hurtienne & Israel, as I see it, is to explain how user’s
pre-existing cognitive model of zooming becomes co-structured in
accordance with the UP-DOWN schema. To account for this, it is necessary to
see “how compound image schemas emerge when simpler ones hook up in
time or space to create more complex groupings” (Kimmel, 2005: 289).
Appropriating the cylinder as an instrument for zooming thus requires that the
user succeed in mentally fusing the UP-DOWN and IN-OUT schemas into a new
integrated model of zooming.
It is important to notice here that this meshing of schemas, or ‘blending’ as it is
dubbed in Fauconnier & Turner (1998, 2002), makes new structural properties
available that are simply not derivable from the act of manipulating a
cylinder or digital data in itself. Neither is it possible to understand the zooming
of image in terms of a unidirectional mapping of UP-DOWN onto the target
domain, since the IN-OUT is in fact playing just as active an organizing role as
do UP-DOWN. Instead, what this case reveals is that schemas from both the
source domain and the target domain are grouped into a third hybridized
structure serving inferential purposes for local understanding and meaningful
interaction: Positioning of the cylinder up or down is inferred to be the cause
of zooming effects on the screen.
I believe with Kimmel (2005) that cases like this points to the ‘situatedness’ of
image schemas. That is, how image schematic formation in user experience is
highly sensitive to its local embedded instantiation. In its present state
Hurtienne & Israel’s framework is unable to account for the situatedness of
image schemas, since image schemas are viewed as invariant and decontextualized units. Moreover, as situated image schemas might rely on the
contribution of structures from both source and target domains, it violates the
source-target construal principle, which is the only mapping procedure
recognized by Hurtienne & Israel.
The Compound Nature of Embodied Schemas
So far I have accounted for two reductionist notions of embodied meaning
inherent in, but not in any way limited to Hurtienne & Israel (2007): (1) the ruling
out of contextual and cultural factors in the mapping of image schemas onto
conceptual structure; and (2) the view of image schemas as pure idealizations
of long-term memory uninfluenced by the local contextual conditions. In this
084/10
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
final section, I argue that we also need to overcome the tendency to view
image schemas as isolated and simple patterns in experience.
In the existing research literature it is common to use the human body as a
way of describing the bodily experience, whereby the CONTAINER schema
comes into existence. Thus, Johnson (1987, p. 21) writes:
Our encounter with containment and boundedness is one of the most
pervasive features of our bodily experience. We are intimately aware of
our bodies as three-dimensional containers into which we out certain
things (food, water, air) and out of which other things emerge (food and
water wastes, air, blood, etc).
Johnson’s descriptions could perhaps give the impression that a primitive and
simple structure of CONTAINEMENT like that depicted in fig. 3 is repeatedly
activated in the experience of our body. However, Johnson is himself careful
to recognize that slightly differing activations of even the same image schema
might occur simultaneously in bodily experience. Let me try to illustrate this
with the following example.
RoBlood is a series of blood-taking robots that is meant to replace bio-analysts
in the Danish health care sector. Now we all know that blood-taking is a very
personal and intimate affair. So, in order to design a robot that patients would
feel secure and safe about, the design team decided to push the soft and
emotional aspects to the fore (Wetton, 2007). This design strategy is clearly
reflected in the way in which technology use, emotions and form are unified
in one of the design proposals named Sessio. Sessio integrates advanced
blood-taking robotic technology into the armrest of an organically shaped
interactive chair. When the patient places her arm on the armrest, it
automatically adjusts in height and follows the patient’s movements. Then a
comfortable vacuum surrounds the arm, so that it is enclosed and kept
tranquil. The blood-taking process starts, when after the position and depth of
veins have been identified, a needle from the armrest is inserted into the
patient’s arm (Nørhave, Madsen & Springborg, 2007).
Fig. 7 Sessio – a blood-taking robot
084/11
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
The Sessio experience interestingly reveals how stimulation of a fairly restricted
area of the body’s surface may give rise to two seemingly competing
activations of one and the same image schema. Sessio’s surrounding of one’s
arm is experientially organized according to a schema of ENCLOSING as
depicted in fig. 8.1. The open square illustrates how Sessio’s armrest acts as a
CONTAINER closing in on a part of the body (the circle). On the other hand,
the insertion of a needle is evoking the sense of an ENTRY into a CONTAINER
(fig. 8.2). Here, the needle is experienced to follow a penetrating PATH that
creates an opening of its own into the body. In this instance the body is itself
profiled as being the CONTAINER (the square) of a contained object (a
portion of blood; the small circle). Since all this is happening as an integrated
part of the same haptic experience, it is reasonable to conclude from this that
image schemas should not per definition by seen as simple and primitive
patterns in experience. On the contrary, it might be better to see the body as
a reservoir for complex and even ambiguous image schematic activations (fig.
8.3).
8.1 ENCLOSING
8.2 ENTRY
Fig. 8.3.
Compound Image schema
Note again, how the source-target principle is violated as both image
schemas contribute with structures to a third higher-level experiential gestalt.
To account for this gestalt, it may be useful to look at the emotional responses
involved. Having a foreign and pointed object inserted into our body is
typically not experienced is being particularly pleasant. On the other hand,
people tend to approve of experiences where larger parts of their body are
being surrounded in a smooth and comfortable way as in a hug, being
covered by a blanket, etc. As the two image schemas are co-activated in the
Sessio experience, an emotional tension is likely to emerge. The unpleasant
feeling from the sting of a needle is as it were downplayed or even counterbalanced by the pleasantness of comfortable vacuum embracing the
patient’s arm; something that might motivate the formation of a new
conceptual meaning – ‘an emotional robot’.
Conclusion
The findings of this paper point towards the need to revise some reductionist
notions of embodied meaning currently influencing interaction design
research. The reductionism found to be present in Huritenne & Israel (2007)
084/12
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
could, for instance, be traced in Lund (2003) and Hornecker & Buur (2006) as
well.
Needless to say, as I have focused primarily on theoretical issues and
concepts, this conclusion must be further investigated through empirical
studies. Thus, it would be of much interest to develop methods for retrieving
information on the effect that the deliberate designing with image schemas
might actually have on user experience. Do users really experience the
artefact as the designer think they would?
These missing empirical data notwithstanding, I suggest that the image
schematic approach do have some positive implications for design. In my
opinion the concept of image schemas is of valuable use to designers as a
powerful descriptive tool in analysing central aspects of embodied meaning
in technology design. Not only in users embodied interaction with, but also in
the creative production of digital artefacts. However, on the basis of my case
analysis, I find it relevant to single out 3 future developments that should be
devoted to the concept of image schemas.
(1) What the Sessio example was only able to indicate is that the
understanding of compound image schemas in lived experience could
benefit from richer phenomenological and qualitative descriptions of bodily
experience.
(2) There is a need to develop a dynamic description of how compound
image schemas are build up in the scenario-like sequence of user experience.
Thus, we have seen that image schemas are not necessarily instantiated as
static transcontextual structures as Hurtienne & Israel among many others
seem to maintain. Rather, image schemas can be transformed and tied up
with each other according to dynamic principles related to how users
experience the design context. The good question is of course what kind of
principles? In Imaz & Benyon (2007) and Markussen (2008), it is demonstrated
that the principle of ‘conceptual blending’ laid bare by Fauconnier & Turner
(1998, 2002) might be the most promising way of accounting for these
principles. But since this has only just begun to be studied in respect to
interaction design further evidence is required.
(3) More knowledge must be acquired as to how culturally motivated models
of use interweave with the mapping of image schemas. The example of the
Tangible Memories Box indicates that such cultural factors might play a
constitutive role, and that we therefore have to revise the source-target
construal principle. This issue is the object of a study by Quine (1991), who
posits that higher-level cultural models indeed govern the mapping and
grouping of image schemas. But Quine’s claim is still much discussed and
there is therefore a need to clarify this question further.
Obviously, design research must lean towards neighbouring disciplines such as
cognitive semantics and phenomenology in its attempt to solve these
problems. However, what I hope to have demonstrated is that if ideas from
these disciplines are uncritically imported into the design field, then there is a
risk of disciplining the body in unproductive ways as it might isolate the body
from its contextual situatedness. Disciplining design research, on the other
hand, could lead instead to a fruitful understanding of how the body and
084/13
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
contextual factors always act as two sides of a flipping coin.
References
Markussen, T. (2008). A Cognitive Semiotic Approach to the Aesthetic Interplay
between Form and Meaning in Responsive Environments. In S. Vihma & T-M.
Karjalainen (eds.) Design Semiotics in Use. University of Art and Design Helsinki.
(forthcoming)
Dourish, P. (2001). Where the action is: the foundation of embodied
interaction. Cambridge, Mass.: MIT Pr.
Gibbs, R. W. (2005). Embodiment and Cognitive Science. Cambridge U. Pr.
Gibbs, R. W. & Colston, H. L. (2006). The Cognitive Psychological Reality of
Image Schemas and their Transformations. In D. Geeraerts Cognitive
Linguistics: Basic Readings (pp. 239-268). Berlin: Mouton de Gruyter. (Original
work published 1995 in Cognitive Linguistics)
Hornecker, E. & Buur, J. (2006). Getting a grip on tangible interaction: a
framework on physical space and social interaction. CHI 2006 Proceedings.
Montréal, Quebec, Canada: ACM.
Hurtienne, J. & Israel, J. H. (2007). Image schemas and their metaphorical
extensions: intuitive patterns for tangible interaction. Proceedings of the 1st
International Conference on Tangible and Embedded Interaction. ACM Press.
Imaz, M. & Benyon, D. (2007). Designing with Blends: Conceptual Foundations
of Human-Computer Interaction and Software Engineering. MIT Press.
Johnson, M. (1987). The Body in the Mind. The Bodily Basis of Meaning,
Imagination and Reason. Chicago & London: The University of Chicago Pr.
Kimmel, M. (2005). Culture Regained – Situated and Compound Image
Schemas. B. Hampe (ed.) From Perception to Meaning. Image Schemas in
Cogntive Linguistics (pp. 285-312). Berlin: Mouton de Gruyter.
Klemmer, S., B. Hartmann & Takayama, L. (2006). How Bodies Matter: Five
Themes for Interaction Design. DIS 2006 (pp. 140-149). Pennsylvania: ACM.
Krippendorff, K. (1989). On the Essential Contexts of Artefacts. In V. Margolin &
R. Buchanan (eds.) The Idea of Design, 1995, 156-184.
Lakoff, G. & Johnson, M. (1980). Metaphors We Live By. The University of
Chicago Pr.
Lakoff, G. (1987). Women, Fire, and Dangerous Things. What Categories
Reveal About the Mind. Chicago & London: The University of Chicago Pr.
Lund, A. (2003). Massification of the Intangible: An investigation into
embodied meaning and information visualization. Ph.d.diss.http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-145
McCarthy, J. & Wright, P. (2004). Technology as Experience. Cambridge, Mass.:
MIT Pr.
Norman, D. (1988). The Psychology of Everyday Things. Basic Books.
084/14
Undisciplined! Proceedings of the Design Research Society Conference 2008.
Sheffield, UK. July 2008
Nørhave, K. L., Madsen, M. & Springborg, Louise (2007). Sessio. In B. Wetton
(ed.) Robots Blood – A Methodology. Kolding: Designskolen Kolding.
Petersen, M.G., Iversen, O., Krogh, P. & Ludvigsen, M. (2004). Aesthetic
Interaction - A pragmatic Aesthetics of Interactive Systems. Proceedings of
ACM DIS2004.
Quine, N. (1991). The Cultural Basis of Metaphor. In J. Fernandez (ed.) Beyond
Metaphor. The Theory of Tropes in Anthropology. Standford: Standford U. Pr.
Ulmer, B. Ishii, H. & Jacob, R. J. K. (2005). Token+Constraint Systems for Tangible
Interaction with Digital Information. In ACM Transactions on Computer-Human
Interaction, 12 (1), 81-118.
Wetton, B. (2007). Robots Blood – A Methodology. Kolding: Designskolen
Kolding.
Winograd, T. & Flores, F. (1986). Understanding Computers and Cognition: A
New Foundation for Design, Ablex Publishing Corporation, Norwood, NJ
Acknowledgments
iI
would like to thank one anonymous reviewer for valuable comments on an
earlier version of this paper.
Thomas Markussen
Markussen (b. 1973) holds a MA-degree in comparative literature and
semiotics from the University of Aarhus (DK). He is currently affiliated to
Designskolen Kolding (DK) where he is working on a PhD research project
dedicated to the developing of a cognitive semiotic framework for
understanding the interweaving of aesthetic experience and meaning
construction in interaction design.
In his earlier work, which includes lecturing, writings and media productions,
Markusen has been focusing on the impact of technology use within art,
architecture and design. He continually pursues these research interests as a
member of the editorial committees behind the international journals
Fibreculture and Inflextions.
084/15