I am Assistant Professor at University of Twente, Industrial Design, in the Human Centred Design group, and I teach both in Industrial Design and in Creative Technology (part of Human Media Interaction). My current research has three (in practice often overlapping) objectives: 1) Develop fast prototyping tools with which stakeholders in co-design sessions can quickly generate tangible artifacts for co-reflection on design proposals at the level of embodied experience and 2) Design Embodied Technologies that support the 'empowerment' of people with disabilities (in particular Autism and moderate intellectual disabilities) in their everyday lives 3) Show how tangible interaction design projects can form a research platform for investigating fundamentals of Embodied Cognition and how design may contribute to the relevant philosophical debates in this area.
In my Phd project (2013, Eindhoven University of Technology, Industrial Design) I applied principles from Embodied Cognition theory to the design of interactive artifacts (mixed physical-digital elements figuring in people's activities, with two case studies focusing on designing interactive technology supporting creative collaborative workshops/sessions in co-design settings, involving several commercial companies and many student projects).
see www.jellevandijk.org or download the thesis from academia Address: www.jellevandijk.org
Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society
The 'robot rights' debate, and its related question of 'robot responsibility', invokes some of th... more The 'robot rights' debate, and its related question of 'robot responsibility', invokes some of the most polarized positions in AI ethics. While some advocate for granting robots rights on a par with human beings, others, in a stark opposition argue that robots are not deserving of rights but are objects that should be our slaves. Grounded in post-Cartesian philosophical foundations, we argue not just to deny robots 'rights', but to deny that robots, as artifacts emerging out of and mediating human being, are the kinds of things that could be granted rights in the first place. Once we see robots as mediators of human being, we can understand how the 'robot rights' debate is focused on first world problems, at the expense of urgent ethical concerns, such as machine bias, machine elicited human labour exploitation, and erosion of privacy all impacting society's least privileged individuals. We conclude that, if human being is our starting point and human welfare is the primary concern, the negative impacts emerging from machinic systems, as well as the lack of taking responsibility by people designing, selling and deploying such machines, remains the most pressing ethical discussion in AI.
Proceedings of the 2017 ACM Conference Companion Publication on Designing Interactive Systems
Hand-drawn sketches can be an easy way for HCI researchers to communicate and express ideas, as w... more Hand-drawn sketches can be an easy way for HCI researchers to communicate and express ideas, as well as to document, explore and communicate concepts between the researcher and user, collaborator, manager or client. These sketches are fast, lightweight, easy to create, and-by varying their fidelity-they can be used in all stages of the HCI research and design process. Here, we aim to explore themes around sketching in HCI with the aim of producing tangible outputs in the form of visual records, articles and papers that review and promote this technique in HCI as a field: 'SketchingDIS: Hand-drawn sketching in HCI' SketchingDIS.wordpress.com, a one-day workshop will bring together researchers from various disciplines that have incorporated hand-drawn sketching into their everyday research practice, to share knowledge and methodologies, generate ideas, practice collaborative sketching, and to discuss the future of hand-drawn sketching in HCI and DIS itself.
This paper critically explores what it means to Design for Embodied Being-in-the-world (D4EB). It... more This paper critically explores what it means to Design for Embodied Being-in-the-world (D4EB). It aims to uncover what this perspective means for designing hybrids, the new interactive physical-digital artefacts developed in wearable, tangible and ubiquitous computing and augmented reality. D4EB is contrasted with the principle of embodied representation, applied for example in designing tangible interfaces between users and digital information. In contrast, D4EB starts from our phenomenological 'being-in-the-world'. Hybrids are conceived as participating in socially situated, sensorimotor couplings that govern the way the lived body operates in the lifeworld. D4EB rejects conceptual dualisms between the (representational) mind and the (physical) body and between (inner) mind and (outside) world. To illustrate its core principles, three design cases are presented. The cases are part of ongoing design-research that formed the basis for the framework. D4EB is further discussed in relation to personal identity, the role of external representations and the role of the designer. D4EB promises to open up a theoretically informed, largely unexplored design space, which can help designers utilize the full power of hybrid technologies. Hybrids may be designed to support people in their embodied being by sustaining, enriching and generating new ways of attuning to the lifeworld.
• Existing digital technologies exacerbate the biases of the human mind, inhibiting diversity ini... more • Existing digital technologies exacerbate the biases of the human mind, inhibiting diversity initiatives • Diversity Computing is a new framework incorporating innovation in theory, methodology, and technology that embraces diversity and avoids normative ordering. A neural network for classifying gang crime was presented recently at the "Artificial Intelligence, Ethics and Society" conference. When asked about the potential for such technology to have negative uses, one of the researchers told reporters, "I'm just an engineer," causing an outcry online and in the media. Meanwhile, in an article for the New York Times, Farhad Manjoo wrote that 2017 was the year in which it dawned on big tech companies that their digital systems come with real-world responsibility and that 2018 will need to be the year in which they, and we, figure out what that means. But more is required than responsible innovation. Taking responsibility, we argue, requires a radical reframing of the role of computing in human lifeworlds. We must envisage roles for technology in a desirable future that reflect and promote a better society. This is not just an argument about how people should put existing computational technologies (e.g., social media) to a different use-we claim that what is needed is the design of fundamentally new kinds of computing devices. Here we sketch such a vision, called diversity computing or DivComp, and possible ways to realize it. We outline a transformative theoretical framework, linked to current and emerging technologies, and we share speculative designs as examples of DivComp implementation. Our DivComp scenarios invite and facilitate shared meaning-making between individuals and groups, embracing differences rather than eliminating them, without recourse to normative frameworks. We further propose that a combination of
Proceedings of the Tenth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '17, 2017
DOI to the publisher's website. • The final author version and the galley proof are versions of t... more DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal. If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the "Taverne" license above, please follow below link for the End User Agreement:
Our approach builds on both the design traditions of participatory design and embodiment. We atte... more Our approach builds on both the design traditions of participatory design and embodiment. We attempt to connect these traditions to the existing body of knowledge on persuasion. First we describe some basic theoretical concepts and infer how they influence persuasive design. Then we present a basic framework with which we intend to address the different abstraction layers involved. Finally, we discuss the principal differences and meeting areas between the disciplines of design and communication, ending up with some considerations for a persuasion toolbox that is intended to help communication professionals and designers effectively design behavior change interventions that fit the messy lives of people in the real world.
Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction - TEI '13, 2013
ABSTRACT Driven by a critique of Ishii et al's recent vision of Radical Atoms we call for... more ABSTRACT Driven by a critique of Ishii et al's recent vision of Radical Atoms we call for a debate on the different conceptual paradigms underlying the TEI community and its activities. TEI was initiated to share and connect different perspectives, but we feel conceptual debate is lacking. To fuel this debate, we start with comparing two paradigms by examining the Radical Atoms proposal and balance it from our design-led perspective. Our aim with this paper is to revive the richness of TEI's multidisciplinary approach.
Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction - TEI '13
Driven by a critique of Ishii et al's recent vision of Radical Atoms we call for a debate on the ... more Driven by a critique of Ishii et al's recent vision of Radical Atoms we call for a debate on the different conceptual paradigms underlying the TEI community and its activities. TEI was initiated to share and connect different perspectives, but we feel conceptual debate is lacking. To fuel this debate, we start with comparing two paradigms by examining the Radical Atoms proposal and balance it from our design-led perspective. Our aim with this paper is to revive the richness of TEI's multidisciplinary approach.
We sketch out an alternative design space between traditional ‘digital technologies’ and ‘physica... more We sketch out an alternative design space between traditional ‘digital technologies’ and ‘physical product design’. We call this design space Co-Embodied Technology (CET) and position it against more representational approaches. Using skill, scaffolding, traces and social coordination, we discuss two TEI design examples we find promising for developing CET.
With Caroline Hummels (first author).
The TEI-community is based on various paradigms. We believ... more With Caroline Hummels (first author).
The TEI-community is based on various paradigms. We believe that the community matures by scrutinising these different paradigms and unravelling the consequences for designing for tangible, embedded and embodied interaction. In this paper we explore the consequences and possibilities of phenomenology-inspired embodied theory, and more specifically the concept of embodied sensemaking, i.e. human sensemaking using sensorimotor couplings to support social coordination between people. Based on our theoretical setting, we introduce seven design principles for developing face-to-face embodied sensemaking technology. We show in this paper how we used these principles to develop a mobile design and sensemaking studio for the encounter between two persons to sketch a future at the cross-section of their disciplines. By explaining these principles, we aim to show what embodied theory can bring the TEI-community, and invite others to do the same.
We sketch out an alternative design space between traditional 'digital technologies' and 'physica... more We sketch out an alternative design space between traditional 'digital technologies' and 'physical product design'. We call this design space Co-Embodied Technology (CET) and position it against more representational approaches. Using skill, scaffolding, traces and social coordination, we discuss two TEI design examples we find promising for developing CET.
Embodied Cognition has been proposed as a relevant theory for tangible and embedded interaction [... more Embodied Cognition has been proposed as a relevant theory for tangible and embedded interaction [14]. Based on two 2-year lasting Research-through-Design cases we identify three variations of the theory: 1) Distributed Representation and Computation, 2) Socially Situated Practices and 3) Sensorimotor Coupling & Enactment. Both social situatedness and sensorimotor coupling proved relevant for design and for understanding user behavior in context. We show how the ‘social’ and the ‘sensorimotor’ are part of one integrated sensemaking process we call ‘socio- sensorimotor coupling’. We argue that the, intuitively appealing, idea of using tangibles for external representation actually hinders designing for sensemaking as socio-sensorimotor coupling. We present a vision of Embodied Cognition Design, which goes beyond a representational interpretation, aiming to intervene more directly into the socio-sensorimotor loop.
Dijk, J. van and Lugt, R. van (2013). Scaffolds for design communication: Research through design... more Dijk, J. van and Lugt, R. van (2013). Scaffolds for design communication: Research through design of shared understanding in design meetings. Artificial Intelligence for Engineering Design, Analysis and Manufacturing, special issue 2, pp 121-131.
In this paper we explore the influence of the physical and social environment (the design space) son the formation of shared understanding in multidisciplinary design teams. We concentrate on the creative design meeting as a microenvironment for studying processes of design communication. Our applied research context entails the design of mixed physical–digital interactive systems supporting design meetings. Informed by theories of embodiment that have recently gained interest in cognitive science, we focus on the role of interactive “traces,” representational artifacts both created and used by participants as scaffolds for creating shared understanding. Our research through design approach resulted in two prototypes that form two concrete proposals of how the environment may scaffold shared understanding in design meetings. In several user studies we observed users working with our systems in natural contexts. Our analysis reveals how an ensemble of ongoing social as well as physical interactions, scaffolded by the interactive environment, grounds the formation of shared understanding in teams. We discuss implications for designing collaborative tools and for design communication theory in general.
Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society
The 'robot rights' debate, and its related question of 'robot responsibility', invokes some of th... more The 'robot rights' debate, and its related question of 'robot responsibility', invokes some of the most polarized positions in AI ethics. While some advocate for granting robots rights on a par with human beings, others, in a stark opposition argue that robots are not deserving of rights but are objects that should be our slaves. Grounded in post-Cartesian philosophical foundations, we argue not just to deny robots 'rights', but to deny that robots, as artifacts emerging out of and mediating human being, are the kinds of things that could be granted rights in the first place. Once we see robots as mediators of human being, we can understand how the 'robot rights' debate is focused on first world problems, at the expense of urgent ethical concerns, such as machine bias, machine elicited human labour exploitation, and erosion of privacy all impacting society's least privileged individuals. We conclude that, if human being is our starting point and human welfare is the primary concern, the negative impacts emerging from machinic systems, as well as the lack of taking responsibility by people designing, selling and deploying such machines, remains the most pressing ethical discussion in AI.
Proceedings of the 2017 ACM Conference Companion Publication on Designing Interactive Systems
Hand-drawn sketches can be an easy way for HCI researchers to communicate and express ideas, as w... more Hand-drawn sketches can be an easy way for HCI researchers to communicate and express ideas, as well as to document, explore and communicate concepts between the researcher and user, collaborator, manager or client. These sketches are fast, lightweight, easy to create, and-by varying their fidelity-they can be used in all stages of the HCI research and design process. Here, we aim to explore themes around sketching in HCI with the aim of producing tangible outputs in the form of visual records, articles and papers that review and promote this technique in HCI as a field: 'SketchingDIS: Hand-drawn sketching in HCI' SketchingDIS.wordpress.com, a one-day workshop will bring together researchers from various disciplines that have incorporated hand-drawn sketching into their everyday research practice, to share knowledge and methodologies, generate ideas, practice collaborative sketching, and to discuss the future of hand-drawn sketching in HCI and DIS itself.
This paper critically explores what it means to Design for Embodied Being-in-the-world (D4EB). It... more This paper critically explores what it means to Design for Embodied Being-in-the-world (D4EB). It aims to uncover what this perspective means for designing hybrids, the new interactive physical-digital artefacts developed in wearable, tangible and ubiquitous computing and augmented reality. D4EB is contrasted with the principle of embodied representation, applied for example in designing tangible interfaces between users and digital information. In contrast, D4EB starts from our phenomenological 'being-in-the-world'. Hybrids are conceived as participating in socially situated, sensorimotor couplings that govern the way the lived body operates in the lifeworld. D4EB rejects conceptual dualisms between the (representational) mind and the (physical) body and between (inner) mind and (outside) world. To illustrate its core principles, three design cases are presented. The cases are part of ongoing design-research that formed the basis for the framework. D4EB is further discussed in relation to personal identity, the role of external representations and the role of the designer. D4EB promises to open up a theoretically informed, largely unexplored design space, which can help designers utilize the full power of hybrid technologies. Hybrids may be designed to support people in their embodied being by sustaining, enriching and generating new ways of attuning to the lifeworld.
• Existing digital technologies exacerbate the biases of the human mind, inhibiting diversity ini... more • Existing digital technologies exacerbate the biases of the human mind, inhibiting diversity initiatives • Diversity Computing is a new framework incorporating innovation in theory, methodology, and technology that embraces diversity and avoids normative ordering. A neural network for classifying gang crime was presented recently at the "Artificial Intelligence, Ethics and Society" conference. When asked about the potential for such technology to have negative uses, one of the researchers told reporters, "I'm just an engineer," causing an outcry online and in the media. Meanwhile, in an article for the New York Times, Farhad Manjoo wrote that 2017 was the year in which it dawned on big tech companies that their digital systems come with real-world responsibility and that 2018 will need to be the year in which they, and we, figure out what that means. But more is required than responsible innovation. Taking responsibility, we argue, requires a radical reframing of the role of computing in human lifeworlds. We must envisage roles for technology in a desirable future that reflect and promote a better society. This is not just an argument about how people should put existing computational technologies (e.g., social media) to a different use-we claim that what is needed is the design of fundamentally new kinds of computing devices. Here we sketch such a vision, called diversity computing or DivComp, and possible ways to realize it. We outline a transformative theoretical framework, linked to current and emerging technologies, and we share speculative designs as examples of DivComp implementation. Our DivComp scenarios invite and facilitate shared meaning-making between individuals and groups, embracing differences rather than eliminating them, without recourse to normative frameworks. We further propose that a combination of
Proceedings of the Tenth International Conference on Tangible, Embedded, and Embodied Interaction - TEI '17, 2017
DOI to the publisher's website. • The final author version and the galley proof are versions of t... more DOI to the publisher's website. • The final author version and the galley proof are versions of the publication after peer review. • The final published version features the final layout of the paper including the volume, issue and page numbers. Link to publication General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal. If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the "Taverne" license above, please follow below link for the End User Agreement:
Our approach builds on both the design traditions of participatory design and embodiment. We atte... more Our approach builds on both the design traditions of participatory design and embodiment. We attempt to connect these traditions to the existing body of knowledge on persuasion. First we describe some basic theoretical concepts and infer how they influence persuasive design. Then we present a basic framework with which we intend to address the different abstraction layers involved. Finally, we discuss the principal differences and meeting areas between the disciplines of design and communication, ending up with some considerations for a persuasion toolbox that is intended to help communication professionals and designers effectively design behavior change interventions that fit the messy lives of people in the real world.
Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction - TEI '13, 2013
ABSTRACT Driven by a critique of Ishii et al's recent vision of Radical Atoms we call for... more ABSTRACT Driven by a critique of Ishii et al's recent vision of Radical Atoms we call for a debate on the different conceptual paradigms underlying the TEI community and its activities. TEI was initiated to share and connect different perspectives, but we feel conceptual debate is lacking. To fuel this debate, we start with comparing two paradigms by examining the Radical Atoms proposal and balance it from our design-led perspective. Our aim with this paper is to revive the richness of TEI's multidisciplinary approach.
Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction - TEI '13
Driven by a critique of Ishii et al's recent vision of Radical Atoms we call for a debate on the ... more Driven by a critique of Ishii et al's recent vision of Radical Atoms we call for a debate on the different conceptual paradigms underlying the TEI community and its activities. TEI was initiated to share and connect different perspectives, but we feel conceptual debate is lacking. To fuel this debate, we start with comparing two paradigms by examining the Radical Atoms proposal and balance it from our design-led perspective. Our aim with this paper is to revive the richness of TEI's multidisciplinary approach.
We sketch out an alternative design space between traditional ‘digital technologies’ and ‘physica... more We sketch out an alternative design space between traditional ‘digital technologies’ and ‘physical product design’. We call this design space Co-Embodied Technology (CET) and position it against more representational approaches. Using skill, scaffolding, traces and social coordination, we discuss two TEI design examples we find promising for developing CET.
With Caroline Hummels (first author).
The TEI-community is based on various paradigms. We believ... more With Caroline Hummels (first author).
The TEI-community is based on various paradigms. We believe that the community matures by scrutinising these different paradigms and unravelling the consequences for designing for tangible, embedded and embodied interaction. In this paper we explore the consequences and possibilities of phenomenology-inspired embodied theory, and more specifically the concept of embodied sensemaking, i.e. human sensemaking using sensorimotor couplings to support social coordination between people. Based on our theoretical setting, we introduce seven design principles for developing face-to-face embodied sensemaking technology. We show in this paper how we used these principles to develop a mobile design and sensemaking studio for the encounter between two persons to sketch a future at the cross-section of their disciplines. By explaining these principles, we aim to show what embodied theory can bring the TEI-community, and invite others to do the same.
We sketch out an alternative design space between traditional 'digital technologies' and 'physica... more We sketch out an alternative design space between traditional 'digital technologies' and 'physical product design'. We call this design space Co-Embodied Technology (CET) and position it against more representational approaches. Using skill, scaffolding, traces and social coordination, we discuss two TEI design examples we find promising for developing CET.
Embodied Cognition has been proposed as a relevant theory for tangible and embedded interaction [... more Embodied Cognition has been proposed as a relevant theory for tangible and embedded interaction [14]. Based on two 2-year lasting Research-through-Design cases we identify three variations of the theory: 1) Distributed Representation and Computation, 2) Socially Situated Practices and 3) Sensorimotor Coupling & Enactment. Both social situatedness and sensorimotor coupling proved relevant for design and for understanding user behavior in context. We show how the ‘social’ and the ‘sensorimotor’ are part of one integrated sensemaking process we call ‘socio- sensorimotor coupling’. We argue that the, intuitively appealing, idea of using tangibles for external representation actually hinders designing for sensemaking as socio-sensorimotor coupling. We present a vision of Embodied Cognition Design, which goes beyond a representational interpretation, aiming to intervene more directly into the socio-sensorimotor loop.
Dijk, J. van and Lugt, R. van (2013). Scaffolds for design communication: Research through design... more Dijk, J. van and Lugt, R. van (2013). Scaffolds for design communication: Research through design of shared understanding in design meetings. Artificial Intelligence for Engineering Design, Analysis and Manufacturing, special issue 2, pp 121-131.
In this paper we explore the influence of the physical and social environment (the design space) son the formation of shared understanding in multidisciplinary design teams. We concentrate on the creative design meeting as a microenvironment for studying processes of design communication. Our applied research context entails the design of mixed physical–digital interactive systems supporting design meetings. Informed by theories of embodiment that have recently gained interest in cognitive science, we focus on the role of interactive “traces,” representational artifacts both created and used by participants as scaffolds for creating shared understanding. Our research through design approach resulted in two prototypes that form two concrete proposals of how the environment may scaffold shared understanding in design meetings. In several user studies we observed users working with our systems in natural contexts. Our analysis reveals how an ensemble of ongoing social as well as physical interactions, scaffolded by the interactive environment, grounds the formation of shared understanding in teams. We discuss implications for designing collaborative tools and for design communication theory in general.
Introduction
This investigation explores relations between 1) a theory of human cognition, calle... more Introduction
This investigation explores relations between 1) a theory of human cognition, called Embodied Cognition, 2) the design of interactive systems and 3) the practice of ‘creative group meetings’ (of which the so-called ‘brainstorm’ is perhaps the best-known example). The investigation is one of Research-through-Design (Overbeeke et al., 2006). This means that, together with students and external stakeholders, I designed two interactive prototypes. Both systems contain a ‘mix’ of both physical and digital forms. Both are designed to be tools in creative meeting sessions, or brainstorms. The tools are meant to form a natural, element in the physical meeting space. The function of these devices is to support the formation of shared insight: that is, the tools should support the process by which participants together, during the activity, get a better grip on the design challenge that they are faced with. Over a series of iterations I reflected on the design process and outcome, and investigated how users interacted with the prototypes.
Embodied Cognition
A theory that might be helpful for designing technological tools, as part of the creative space, might be Embodied Cognition. The theory argues against the classic idea that thinking is something that happens purely ‘internal’ to us. Thinking, instead, emerges in action, out of continuous embodied interactions between brain, our body and the way our body is ‘situated’ in a physical- and social context. According to Embodied Cognition, cognition is best seen as a dynamic coupling (Clark, 1997; Dourish, 2001), or a process of coordination (Suchman, 2007; Clancey, 1997), or, as phenomenologists call it, as getting ‘grip’, through skilled action (Dreyfus, 2002; Merleau-Ponty, 1963).
One reason Embodied Cognition may be useful is because the field of interactive systems design shows a growing trend towards trying to integrate physical form and digital process. Looking at it from the perspective of Industrial Design, this entails adding interactive behaviors to physical products, using sensors, actuators, and the like. From the perspective of computer science, it means creating so-called ‘tangible’ interfaces, where various physical objects can be used to control digital information, a follow-up on the familiar ‘graphical’ interface. One may say it moves interaction with digital processes ‘back into the real world’, mixing it seamlessly with physical objects, environments, and social contexts.
Findings
The designs: NOOT and FLOOR-IT
NOOT, in its final form, consists of a system of tangible clips with which one can create time-markings in a continuous audio-recording of the creative session. In the final reflection I offered that NOOT couples individual moments of reflection-in-action to the overall group conversation, thereby supporting the formation of shared insight.
FLOOR-IT enables people to create digital photographs of any of the sketches or written texts (or other visual elements) created during the session. The traces form a conversational ‘scaffold’, to which people can point and refer during the talk. In a user study, comparing FLOOR-IT with a variation that projected the pictures on a shared wall, it was discovered that such traces function to help people position themselves socially in relation to others. Referring to ones personal trace during ongoing talk helps not so much to share factual information rather than that it serves to present yourself as a valuable partner in the activity, and to invite others to do so as well.
Based on my reflections on the design iterations, I was able to discern four variations of the theory that each have their own particular consequences for design. I call these the 1) distributed representation and computation perspective, 2) the socially situated practice perspective and 3) the sensorimotor & enactment perspective.
The distributed representation and computation perspective is perhaps most easily understood by those familiar with computational principles and it has proven to be a useful and relevant set of principles for interaction designers. Yet it actually hinders interaction designers in getting to the heart of the notion of embodiment. Instead, based on my design investigations I offer that the prime ingredients needed for understanding how my prototypes support shared insight are 1) the sensorimotor aspect of cognition (how insight emerges from real-time coupling of perception and action) and 2) the social situatedness of cognition (how cognition is socially coordinated between people). Moreover, sensorimotor coupling and social situatedness are strongly integrated in one unified embodied activity (Goodwin, 2000). In particular, the studies revealed how people would create expressive traces in the environment. Expressive traces, e.g. a physical sticky-note, a NOOT clip, or a trace in FLOOR-IT, are both the outcome of people’s earlier actions, as well guiding further action. That is, traces become part of people’s sensorimotor couplings. At the same time, expressive traces are also social artifacts, created in and for a social context, publicly available and socially accountable. They function to coordinate people’s social positioning in the physical space. Expressive traces form the linking pins between social interaction and sensorimotor coupling, thereby supporting the emergence of shared insight.
Conclusions for design
I offer a number of pitfalls and opportunities for designers that want to ground design in embodied cognition theory. I start with the claim that the classic interface concepts, which rely on information processing metaphors, are best explained with a ‘Distributed Representation and Computation’ version of embodied cognition. This goes also for many of the so-called ‘tangible media systems’, where tangible objects essentially ‘encode’ digital information in physical form. However useful, they stand in the way of designing for a more fundamental form of Embodied Cognition, as they too easily draw us back into ‘Cartesian thought’.
Based on Socially Situated Practices and Sensorimotor Coupling and Enactment, I propose a more fundamental form of Embodied Cognition Design. Embodied Cognition Design brings forth interactive systems that transform our ways of perceiving, our possibilities for acting, our ways of interacting socially with others, and it helps us to create endurable ‘expressive traces’ in the environment. In any concrete product proposal, all of these aspects will be part of the unified experience of the user. Through Embodied Cognition Design we may search for completely new roles for digital computing technology in human practices. This means going beyond the classical, Cartesian functions of storing, processing and presenting representational data. One consequence of this vision is that the ‘function’ of an artifact can no longer be predefined before one starts designing the ‘interface’: in Embodied Cognition Design, concrete interactions between the user and the system bring forth, or ‘enact’ the meaning that the system has for the user. This means one has to design the interactive behavior and ‘what the system is for’, both at the same time, with no a priori distinction between the digital- and the physical aspect, nor even the embedding context. One may for example design in iterative fashion, building series of functioning prototypes, which can be tried out such as to stay in close contact with the user and his context of practice throughout the entire project.
Uploads
Papers by jelle van dijk
The TEI-community is based on various paradigms. We believe that the community matures by scrutinising these different paradigms and unravelling the consequences for designing for tangible, embedded and embodied interaction. In this paper we explore the consequences and possibilities of phenomenology-inspired embodied theory, and more specifically the concept of embodied sensemaking, i.e. human sensemaking using sensorimotor couplings to support social coordination between people. Based on our theoretical setting, we introduce seven design principles for developing face-to-face embodied sensemaking technology. We show in this paper how we used these principles to develop a mobile design and sensemaking studio for the encounter between two persons to sketch a future at the cross-section of their disciplines. By explaining these principles, we aim to show what embodied theory can bring the TEI-community, and invite others to do the same.
In this paper we explore the influence of the physical and social environment (the design space) son the formation of shared understanding in multidisciplinary design teams. We concentrate on the creative design meeting as a microenvironment for studying processes of design communication. Our applied research context entails the design of mixed physical–digital interactive systems supporting design meetings. Informed by theories of embodiment that have recently gained interest in cognitive science, we focus on the role of interactive “traces,” representational artifacts both created and used by participants as scaffolds for creating shared understanding. Our research through design approach resulted in two prototypes that form two concrete proposals of how the environment may scaffold shared understanding in design meetings. In several user studies we observed users working with our systems in natural contexts. Our analysis reveals how an ensemble of ongoing social as well as physical interactions, scaffolded by the interactive environment, grounds the formation of shared understanding in teams. We discuss implications for designing collaborative tools and for design communication theory in general.
The TEI-community is based on various paradigms. We believe that the community matures by scrutinising these different paradigms and unravelling the consequences for designing for tangible, embedded and embodied interaction. In this paper we explore the consequences and possibilities of phenomenology-inspired embodied theory, and more specifically the concept of embodied sensemaking, i.e. human sensemaking using sensorimotor couplings to support social coordination between people. Based on our theoretical setting, we introduce seven design principles for developing face-to-face embodied sensemaking technology. We show in this paper how we used these principles to develop a mobile design and sensemaking studio for the encounter between two persons to sketch a future at the cross-section of their disciplines. By explaining these principles, we aim to show what embodied theory can bring the TEI-community, and invite others to do the same.
In this paper we explore the influence of the physical and social environment (the design space) son the formation of shared understanding in multidisciplinary design teams. We concentrate on the creative design meeting as a microenvironment for studying processes of design communication. Our applied research context entails the design of mixed physical–digital interactive systems supporting design meetings. Informed by theories of embodiment that have recently gained interest in cognitive science, we focus on the role of interactive “traces,” representational artifacts both created and used by participants as scaffolds for creating shared understanding. Our research through design approach resulted in two prototypes that form two concrete proposals of how the environment may scaffold shared understanding in design meetings. In several user studies we observed users working with our systems in natural contexts. Our analysis reveals how an ensemble of ongoing social as well as physical interactions, scaffolded by the interactive environment, grounds the formation of shared understanding in teams. We discuss implications for designing collaborative tools and for design communication theory in general.
This investigation explores relations between 1) a theory of human cognition, called Embodied Cognition, 2) the design of interactive systems and 3) the practice of ‘creative group meetings’ (of which the so-called ‘brainstorm’ is perhaps the best-known example). The investigation is one of Research-through-Design (Overbeeke et al., 2006). This means that, together with students and external stakeholders, I designed two interactive prototypes. Both systems contain a ‘mix’ of both physical and digital forms. Both are designed to be tools in creative meeting sessions, or brainstorms. The tools are meant to form a natural, element in the physical meeting space. The function of these devices is to support the formation of shared insight: that is, the tools should support the process by which participants together, during the activity, get a better grip on the design challenge that they are faced with. Over a series of iterations I reflected on the design process and outcome, and investigated how users interacted with the prototypes.
Embodied Cognition
A theory that might be helpful for designing technological tools, as part of the creative space, might be Embodied Cognition. The theory argues against the classic idea that thinking is something that happens purely ‘internal’ to us. Thinking, instead, emerges in action, out of continuous embodied interactions between brain, our body and the way our body is ‘situated’ in a physical- and social context. According to Embodied Cognition, cognition is best seen as a dynamic coupling (Clark, 1997; Dourish, 2001), or a process of coordination (Suchman, 2007; Clancey, 1997), or, as phenomenologists call it, as getting ‘grip’, through skilled action (Dreyfus, 2002; Merleau-Ponty, 1963).
One reason Embodied Cognition may be useful is because the field of interactive systems design shows a growing trend towards trying to integrate physical form and digital process. Looking at it from the perspective of Industrial Design, this entails adding interactive behaviors to physical products, using sensors, actuators, and the like. From the perspective of computer science, it means creating so-called ‘tangible’ interfaces, where various physical objects can be used to control digital information, a follow-up on the familiar ‘graphical’ interface. One may say it moves interaction with digital processes ‘back into the real world’, mixing it seamlessly with physical objects, environments, and social contexts.
Findings
The designs: NOOT and FLOOR-IT
NOOT, in its final form, consists of a system of tangible clips with which one can create time-markings in a continuous audio-recording of the creative session. In the final reflection I offered that NOOT couples individual moments of reflection-in-action to the overall group conversation, thereby supporting the formation of shared insight.
FLOOR-IT enables people to create digital photographs of any of the sketches or written texts (or other visual elements) created during the session. The traces form a conversational ‘scaffold’, to which people can point and refer during the talk. In a user study, comparing FLOOR-IT with a variation that projected the pictures on a shared wall, it was discovered that such traces function to help people position themselves socially in relation to others. Referring to ones personal trace during ongoing talk helps not so much to share factual information rather than that it serves to present yourself as a valuable partner in the activity, and to invite others to do so as well.
Based on my reflections on the design iterations, I was able to discern four variations of the theory that each have their own particular consequences for design. I call these the 1) distributed representation and computation perspective, 2) the socially situated practice perspective and 3) the sensorimotor & enactment perspective.
The distributed representation and computation perspective is perhaps most easily understood by those familiar with computational principles and it has proven to be a useful and relevant set of principles for interaction designers. Yet it actually hinders interaction designers in getting to the heart of the notion of embodiment. Instead, based on my design investigations I offer that the prime ingredients needed for understanding how my prototypes support shared insight are 1) the sensorimotor aspect of cognition (how insight emerges from real-time coupling of perception and action) and 2) the social situatedness of cognition (how cognition is socially coordinated between people). Moreover, sensorimotor coupling and social situatedness are strongly integrated in one unified embodied activity (Goodwin, 2000). In particular, the studies revealed how people would create expressive traces in the environment. Expressive traces, e.g. a physical sticky-note, a NOOT clip, or a trace in FLOOR-IT, are both the outcome of people’s earlier actions, as well guiding further action. That is, traces become part of people’s sensorimotor couplings. At the same time, expressive traces are also social artifacts, created in and for a social context, publicly available and socially accountable. They function to coordinate people’s social positioning in the physical space. Expressive traces form the linking pins between social interaction and sensorimotor coupling, thereby supporting the emergence of shared insight.
Conclusions for design
I offer a number of pitfalls and opportunities for designers that want to ground design in embodied cognition theory. I start with the claim that the classic interface concepts, which rely on information processing metaphors, are best explained with a ‘Distributed Representation and Computation’ version of embodied cognition. This goes also for many of the so-called ‘tangible media systems’, where tangible objects essentially ‘encode’ digital information in physical form. However useful, they stand in the way of designing for a more fundamental form of Embodied Cognition, as they too easily draw us back into ‘Cartesian thought’.
Based on Socially Situated Practices and Sensorimotor Coupling and Enactment, I propose a more fundamental form of Embodied Cognition Design. Embodied Cognition Design brings forth interactive systems that transform our ways of perceiving, our possibilities for acting, our ways of interacting socially with others, and it helps us to create endurable ‘expressive traces’ in the environment. In any concrete product proposal, all of these aspects will be part of the unified experience of the user. Through Embodied Cognition Design we may search for completely new roles for digital computing technology in human practices. This means going beyond the classical, Cartesian functions of storing, processing and presenting representational data. One consequence of this vision is that the ‘function’ of an artifact can no longer be predefined before one starts designing the ‘interface’: in Embodied Cognition Design, concrete interactions between the user and the system bring forth, or ‘enact’ the meaning that the system has for the user. This means one has to design the interactive behavior and ‘what the system is for’, both at the same time, with no a priori distinction between the digital- and the physical aspect, nor even the embedding context. One may for example design in iterative fashion, building series of functioning prototypes, which can be tried out such as to stay in close contact with the user and his context of practice throughout the entire project.