What Can a Robot’s Skin Be? Designing Texture-changing Skin for Human–Robot Social Interaction

Biological skin is an expressive medium that conveys a variety of information about an organism, e.g., age, sex, race, and health condition. In addition, an organism’s affective states can also be expressed through skin color [78], trembling, sweating, temperature [54], and muscle movements beneath the skin [35]. Some animals express their aroused states by changing skin textures or appendages’ shapes, a phenomenon called piloerection [20]. Such behaviors include humans displaying goosebumps, birds ruffling their feathers, cats raising their back fur, blowfish protruding spikes, and so on.

These biological phenomena have inspired a variety of expressive surfaces in the field of tangible human–computer interfaces. Examples include a fur interface that mimics bristling effects through vibration [30], an interface with luminescent tentacles following a user’s hand waves [60], and a carpetlike texturally rich interface for tactile and visual communication [14]. Similar ideas have been investigated in responsive clothing that expresses or exaggerates its wearer’s physical or emotional states. For example, bristling clothes developed by Ohkubo et al. [63] raise hairs in response to a wearer’s breathing rate. A face mask named “Aposema” [57] augments a person’s facial expressions with inflation and change of color.

Social robots can also express their emotions through haptic channels, with movements on or beneath the skin. Most of the movements simulate animals’ behaviors, such as breathing, heartbeat, purring, and warming. For example, the Haptic Creature robot [85, 86] displays affective states through its breathing rate, ear stiffness, and vibrotactile purr. The Cuddlebits robot [13] expresses emotions with 1-degree-of-freedom (DOF) breathinglike behaviors. Robots also express their emotions through the change of skin color [77], temperature [64], haptic forces [33], and even scents [70].

In each case, robots change physical properties globally across the skin, using vibrations, temperature, and overall color, or deform their skin on a full-body scale. Locally expressive texture change of a robot’s skin, as proposed in this work, remains a more unexplored expressive channel in social interaction. In addition, most of the existing works have heavily focused on emotional expression, with other expressive contexts under-explored.

Human skin is an active sensory organ with widely distributed tactile receptors that detect touch, pressure, pain, temperature, and so on [45]. Perception of touch affords transmission of social messages, as humans often use touch to share their feelings and enhance other forms of verbal or non-verbal communications [31].

Sensitive robotic skin has been widely explored [4, 74]. Enabling touch perception on robots’ skin allows them to be physically safe and interactive to be near humans [16, 76]. Most robots combine multiple sensor devices within the skin substrate, with arrays of sensors forming continuous coverage over the skin [39, 58]. For example, the Haptic Creature robot [86] embeds a network of 56 surface-mounted force-sensing resistors to recognize affective touch. The Huggable robot [75] utilizes over 1,000 Quantum Tunnelling Composite sensors, 400 temperature sensors, and 45 electric field sensing electrodes across the full body to detect the social contents of touch. Alternative sensing methods with lighter hardware but lower resolution include acoustic sensing [1], air-pressure sensing [2], conductive fur sensing [28], EIT-based sensing [73], and so on. Our recent work “ShadowSense” [36] uses computer vision to classify shadows created by touching the robot’s skin. It achieves high resolution and a large detection range with minimal hardware and is especially apt for sensing social touches on translucent and deformable skin.

Contrary to a conventional robot’s skin that serves as a solid boundary, biological skins are usually selective and permeable. Cellular membranes can actively regulate the exchange of molecules based on the cell’s requirements while maintaining a stable internal order irrespective of environmental changes [26]. Pores on human skin allow sweat and oil to escape from it while getting rid of toxins [19].

As for artificial skins, dynamic facades of buildings are capable of modulating permeability, for example by allowing in a controlled amount of air and light to regulate internal temperature and ventilation [15, 48]. Clothing materials can selectively let heat and sweat escape, while protecting the wearer from rain or snow. In the area of wearable HCI research, the “Second Skin” project [84] describes a device that can open texture elements to cool it off when the skin temperature rises during running and sweating.

Like biological and artificial skins, a robot’s skin may benefit from such selective and permeable features, enabling exchange regulation with surrounding environments. For example, a robot may open its “pores” to cool off internal mechanisms or close them to make itself heatproof, based on environmental conditions. A permeable skin may act as a social filter that regulates information exchange. For example, skin may change its translucence level to allow or prevent the robot from capturing the user’s visual data, thus adjusting to privacy requirements posed by interaction contexts. It may also change its transparency to metaphorically indicate an “openness to interact,” placing this function on the boundary between regulation and expression.

Finally, the skins of some animals serve as mechanical and functional effectors that actively change physical relations to environments, such as through locomotion, camouflaging, and adhesion. To list a few examples, hairlike cilia on the surface of cells can move the surrounding fluid and enable such cells to swim [51]. An octopus can change its skin color and textures to blend in with the seafloor [52]. Geckos’ feet are covered with hundreds of tiny hairs named setae, helping them to reversibly cling to any surfaces [10].

Roboticists have been inspired by such naturally occurring solutions to explore novel materials and mechanisms for functional robot’s skin. Examples include a snake-skin-inspired crawling robot [67], a climbing robot using gecko-inspired skin [56], an octopus-inspired three-dimensional (3D) morphing and camouflaging skin [65], and a musclelike skin for actuating inanimate object [12]. The above-mentioned bio-inspired robotic skins are primarily aimed at achieving functional purposes. Similar metaphors could be used to design functional skin effectors that convey social meanings.

A robot’s skin can display an explicit or iconic message through direct visual or haptic projection. Some social robots blend a screen within their skin, which enables displays of verbal messages or visual abstractions as information presentation. For example, the ENRICHME robot [17] displays written texts on its torso as instructions; an evacuation robot [68] sends emergency alerts using on-body dynamic signage. Robots may convey messages through a haptic channel as well, such as projecting braille with reconfigurable dots on the skin [82] (Figure 2(a)). Messages with explicit verbal translations directly communicate robot’s intent or state with no ambiguity but could require a high degree-of-freedom skin transformation, as well as more processing effort and a matched cultural background to comprehend abstract information.

Physiological changes of the skin can be caused by affective experiences on the psychological scale. In response to sudden and intense emotions such as fright and shock, animals can get piloerection, that is, erection of tiny textures or bristling of hairs due to involuntary muscle contraction, such as human goosebumps [7]. The color of skins may rapidly change out of psychological arousal (e.g., human faces “blush” in response to situations of personal embarrassment; the Panther Chameleon changes to red and yellow when angered as a warning to others to back off [27]).

Inspired by the biological equivalent, robots may rapidly alter skin textures, color, or shape of hair to express their affective states. For example, a robot can respond to alert with spiky, trembling textures; a furry robot being suddenly awakened may bristle its fur; getting angry causes tensed, inflated muscles to appear on the robot’s skin (Figure 2(b)).

As described in Section 2.4, biological skins actively exert forces onto environmental surfaces and have inspired functional applications in robotics, like a robot’s skin applying adhesive forces in climbing or gripping, shear forces in locomotion. Apart from functional purposes, a robot’s skin manipulating spatial relation can be a form of expression, reflecting the robot’s relationship to users and environments. For example, a robot’s skin may actively stick to a person’s body to display affection or to allow the individual to feel attractive or repulsive forces with its hand by communicating a willingness to be touched. Likewise, a robot may direct its attention with shear forces parallel to the skin, such as referring to a person in the space by moving its skin toward it (Figure 2(c)).

The visual appearance and material property of a robot’s skin can suggest its roles and modes of interaction, as mentioned in Section 2. Section 2.3 described a permeable robot’s skin as an active filter for functional and social regulation. We also conceptualize the exposure of skin as reflecting a robot’s dynamic modes in interaction. Users may visualize the level of disclosure of their personal data in front of a robot through permeability change. For instance, robots with sensors embedded underneath the skin may open pores to see and hear persons more clearly, thus suggesting the robot is more actively involved in interaction. However, a robot can selectively expose its internal states in front of a user, which may signal the robot’s identity and traits. For example, a robot may make itself more machinelike by revealing its internal gears and motors. Uncovering the hidden thinking process underneath the skin may increase its perceived honesty (Figure 2(d)).

Biological skins and their appendages gradually change their physical properties over time (e.g., hairs grow in length; aging human skin leads to wrinkles, loss of elasticity, laxity, and rough textures). Long-term growth and evolution across generations may result in skin changes in adaptation to environments, such as through natural selection [24].

Although robots do not suffer from natural growth or decay, growinglike mechanisms are of emerging interest in the field of biomimetic robots. A robot that grows and evolves its body allows for better adaptation to task constraints and complex, changing environments [53]. For example, Hawkes et al. [34] created a soft robot that navigates the environment through growth. Similarly, Dottore et al. [23] presented a plant-inspired growing robot that self-builds its structure and adapts the morphology to environments. In each case, growth is thought of as a functional behavior, such as allowing a robot to search, explore, and adapt to the environment.

We envision growthlike behaviors in a robot’s skin as a potential expressive language. Robots may integrate a short-term or long-term skin change to reflect relationships with environments, users, and time. To illustrate, in Figure 2(e), a vacuum robot grows its hairs to express the amount of dust collected; a companion robot gradually develops wrinkles on the skin, indicating the robot’s aging while an accompanied kid grows up; a plantlike robot gradually evolves its appendages reflecting living conditions.

The physical properties of skin can signal the health conditions of an organism and the state of being alive or dead. For instance, patches on the skin can indicate illness or injury; a warm and regularly breathing skin can be a signal of life; some species, such as sea anemones, have subtly moving tentacles communicating liveliness.

A robot’s skin may signal its health state (i.e., being alive or damaged) or similar representations, such as continuously moving skin appendages, changing color, and rigidity. For example, a robot may subtly wave its hairs or tentacles to express the sense of being alive and awake; a robot may highlight damages or errors with abnormal skin color displayed in malfunctioning areas (Figure 2(f)).

To summarize, we argue that there is a potentially rich design space for interactive robotic skin, beyond the state of the art. We note that for many of these expressive functions, we need robotic skin that can be actuated beyond globally controlled skin color and shape changes, and that skin texture would need to be finely controlled. Using texture thus as an expressive medium, we proposed six expressive codes, illustrated by speculative sketches. We now proceed to describe a design and engineering technique to achieve these textures, along with a design vocabulary afforded by these artificial skin textures.

A TU is the minimal element of a texture-changing skin, designed to be both visually and haptically expressive. Each TU has an application-specific shape, consisting of an elastomer body, an internal air cavity, and (optionally) a haptic-expressive element.

Figure 3 presents a design vocabulary for textures. Multiple properties of texture units can be manipulated in design to provide different haptic and visual experiences, including shape, materiality, motion, and force. Shape is defined by the geometry of external elastomer and the structure of internal air cavity and mainly affects a TU’s visual feedback (e.g., a cone versus a bump). The shape of the embedded air cavity affects the deformation of the TU. For example, when being deflated, a cylindrical cavity may result in a dent on the surface, while a valleylike cavity will generate a furrow. Materiality with varied stiffness and haptic elements can more strongly determine the haptic experience during the interaction. For example, a soft tip of a spike TU can be replaced with a sharp, rigid element (shaded area) to amplify the spiky and unpleasant feeling. In terms of the TU’s motion, each unit can be inflated and deflated. For each motion primitive, the speed of deformation can be converted by the frequency of internal airflow, while stiffness and range of deformation is determined by internal air pressure and the elasticity of elastomer. In addition, a texture unit can be structurally designed to transmit different types of forces with varied haptic feedback (e.g., repulsive force by a goosebump, attractive force by a suction cup, an oriented TU generating force perpendicular to the inclined surface, and a furrow TU applying contraction force along the skin surface).

To illustrate the variations, we present six examples of TU designs and their resulting deformation under inflation (green, solid boundary line) and deflation (yellow, dashed boundary line) of internal air cavities (Figure 4). We take a biomimetic approach to translate patterns of textures in nature to TU mechanisms that can transit between resting states and actuated states. Several design iterations were performed on the shape of cavities, thickness of walls, and placements of haptic materials, finally converging on the six visually and haptically expressive units. Each diagram illustrates the structure of a TU from a front sectional view (section plane perpendicular to the skin surface), except the stoma TU that shows a top view. A more detailed description of the textures is presented in Section 5.

Our design allows the combination of a large number of TUs into a substrate to form a TM. A substrate is made up of an elastomer body and embedded fluidic networks that connect to the air cavities of TUs. Each network can be separately pressure controlled. A substrate can be described by its shape and internal network connections. The shape of a substrate determines the skin’s macro-scale geometry, which depends on the shape of the robot. Embedded fluidic networks determine how TUs are connected and controlled.

Figure 3 provides a design vocabulary for combining TUs to form a Texture Module. The distribution of textures can provide distinct visual and haptic representations. Uniformly distributed textures may represent global patterns across the skin, while clustered textures may highlight local groups. The configuration of directional texture units (e.g., scales) is determined by the orientations of textures and their geometric relationship. For example, textures facing the same direction form a parallel pattern and may generate a directional force flow across the skin. The TM’s resolution, determined by the size of the TU and the density of texture placement, affects display quality as well as haptic finger, hand, and body interactions. The connectivity between texture units determines grouping and freedom of actuation. Having higher degrees of freedom may realize more diverse motions but results in more complicated actuation and control systems.

To fabricate a Texture Module, two elastomer layers are molded through a standard casting process with silicone rubber. The mold used for the upper layer models a desired TM structure, with one part casting the outer TM shape, and the other for fluidic cavities and chambers. The mold used for the lower layer may contain an inextensible film, such as a fiber or a article to constrain deformation downward. Then the two layers are separately cured and joined together by applying a thin layer of elastomer [38]. A detailed fabrication guide and 3D CAD files of molds are provided in supplemental files.

To integrate skins in the context of interactive robots, the size and noise level of the powering systems are important considerations, favoring a self-contained, low-noise profile. Thus, we designed a power screw actuated linear displacement pump to actuate textures. The core of this design is a re-purposed syringe, which we used as a cylindrical pump with a plunger displaced by a linear stepper motor. This system afforded low noise, high control accuracy, and high efficiency, compared to a commonly used rotary pump. A full description of the mechanism can be found in a supplemental file and in a previous publication [38].

Inspired by biological piloerection, such as humans growing goosebumps (Figure 5(a-1)) and porcupine fish protruding spikes (Figure 5(a-2)), we design a texture module with two 2D arrays of goosebumps and spikes, separately connected with two inner fluidic networks. Goosebumps transform from a resting flat surface to smooth bumps under positive air pressure (Figure 5(a-3)). Spikes units are structured as cones with rigid haptic elements embedded on the TU tips (Figure 5(a-4)). The spikes deform and retract the sharp tips under negative pressure.

We use the textures’ shapes and the speed of their movements to convey a variety of emotions. We map the shape as communicating an emotional valence, in that spikes naturally represent angry, defensive states, and goosebumps are related to pleasure and excitement. The frequency of textures’ movements is mapped to an arousal dimension, with higher texture change frequency communicating higher arousal. This is also inspired by biological analogies, such as the frequency of breath and heartbeats falls to a lower level when in a low arousal state and increases when aroused. The mappings were further validated with evidence in a controlled human experiment [37]. The results of the experiment indicated that participants consistently perceive the proposed texture behaviors as expressing specific emotions, with a similar distribution across interaction modes, including video viewing, in person observation, and touching the texture. The spike-goosebump combination is proved to be effective in conveying both emotional arousal and valence, with valance being more easily differentiated when touching the textures. This design improves upon most current haptic social robotic research with breathinglike or vibrotactile behaviors, which were shown to be somewhat ambiguous and ineffective in conveying emotional valence [85].

Muscle contraction can cause skin to bunch together, forming dynamic wrinkles between the bulk of muscles. For example, forehead wrinkles appear when a person frowns or furrows the eyebrows, expressing states like confused, concentrated, worried, or annoyed (Figure 5(b-1)). We simulate dynamic wrinkles on skin to display similar emotions. Each wrinkle unit is structured with a valleylike air cavity embedded in a flat elastomer body. The skin changes from a resting flat surface indicating a robot’s “relaxed” state (Figure 5(b-2)) to a furrowed surface with haptic tension under negative air pressure, indicating the robot being “concentrated” or “nervous” (Figure 5(b-3)).

Inspired by sea anemones slowly waving their tentacles (Figure 5(c-1)), we design a skin module with subtly moving tentacles for displaying liveliness in robots. Unlike the general method of designing each TU as an independent actuator, we use passive under-actuated tentacles driven by an actuation layer underneath. In the presented prototype, the module has a 4-DOF actuation layer with a resting flat surface attached by static tentacles (Figure 5(c-2)). Pressurizing the actuation layer deforms the surface, thus driving tentacles to move accordingly (Figure 5(c-3)).

An octopus’s suckers (Figure 5(d-1)) can reversibly adhere to different substrates by forming a seal at the rim and reducing pressure in the acetabular cavity [79]. This has inspired adhesive robotic skins in performing climbing, manipulation, and grasping tasks [5, 32]. Using a similar principle, we implement a skin module with arrays of suction cups: the skin can deflate to adhere to an environmental surface (Figure 5(d-3)) and pressurize to detach (Figure 5(d-2)). In human–robot interaction, a robot may display affection to a person by attaching to his body or stick to the ground and express resistance to be moved or picked up.

Humans use nonverbal behaviors to direct attention, such as pointing to or gazing at a specific object [6]. We use oriented skin textures to communicate a similar cue. The textures are designed by simulating biological folding patterns, like scales of pinecones [46] (Figure 5(e-1)) and using rhythmic beating movements to generate a directional flow. Each oriented texture unit has an inclined surface under positive air and is strengthened by an embedded rigid layer (Figure 5(e-3)). The unit can flatten and blend in the skin under air deflation (Figure 5(e-2)). With the beating of oriented textures, a skin can generate a continuous force toward the oriented direction. The force can be further visualized by passing an object along the skin or, by pointing the TUs toward the ground plane, support locomotion by the robot. In interaction, a robot may refer to a person or object by moving the skin or passing an object toward it.

Inspired by plants’ stomata that open and close to regulate exchange (Figure 5(f-1)), we make “pores” on a skin module to indicate the level of interaction exposure. Each texture unit opens the central pole in a resting state (Figure 5(f-2)). When pressurized, walls around a pore deform toward the center, thus closing the pore (Figure 5(f-3)). In this way, the skin can control the amount of air and light to pass through and indicate the willingness of exposure. Opening pores may increase a robot’s exposure to the environment, permitting it to acquire richer data; however, it is less protected by the reduced physical filtering.

The robotic skin modules described in the previous sections are small stand-alone patches of expressive texture behaviors. In practice, robotic skins will always be part of a robot’s larger morphological and functional design. The relationship between the textured skin and the robot’s overall morphology prompts several design considerations.

First, texture expression, as presented here, can be particularly useful for non-anthropomorphic and low-degree-of-freedom robots, as it is less constrained by the robot’s morphology compared to gestures and facial expressions. Many robots have functional shapes (e.g., vacuum cleaners, automatic doors, and cars), making it impossible for them to exploit anthropomorphic elements like faces or arms to express emotions and intentions. Existing non-anthropomorphic modalities, such as movements, sounds, and lights, have limited expressive effectiveness and may cause under-performance in their primary functions [11]. Thus, texture-changing skin could be a productive extension of the expressive vocabulary for such non-anthropomorphic robots.

An important design decision, when integrating an active skin in a robot design, is choosing the form and placement of the texture modules. This choice should be based on the function and structure of the robot, as well as the desired expression the skin should generate. For example, textures on a robot’s back may be less observable and communicate mainly haptically when a user is holding the robot; textures at the bottom of a robot may be neither visible nor reachable but express by moving the robot around or adhering to the ground.

The question of scalability of a texture-changing skin is not trivial and should be further explored with social robots of different shapes, sizes, and functions. The current design poses constraints on the surface property of robots, with a requirement of a relatively flat and small-scale profile. Future research will include improving fabrication techniques to allow for designing and manufacturing non-flat texture modules with potentially finer textures.

In addition, the current setup is powered by a system sized proportionally to the number of texture types and the scale of texture modules. More efforts need to be made to design a more compact actuation solution for large-scale and high-degree-of-freedom implementations.

While this article focuses on the expressive aspect of robotic skins, future work should explore other roles of interactive skins, such as adding perception channels for bi-directional communication. This could be achieved by embedding conductive sensors [28], measuring air pressure [2], and performing vision-based shadow sensing, as proposed in our previous work [36].

Dynamic textures on a robotic skin express mainly via two sensory channels: vision and touch. These two layers generate rich and distinct experiences: shapes and deformations above the skin draw visual impressions, whereas materiality and tangibility of the textures impact users through tactile sensations. Users interacting with a robot through these two differing modalities may have different interpretations of the expression or their intensities, as evidenced in our previous work [37]. This difference may also shape user behavior, for example getting closer to the robot to discover and experience different modes of expression.

The interplay between skin expressions and other communicative modalities of a robot (e.g., a face, gestures, or voice) must also be considered. Keeping expressions consistent across modalities may help avoid confusion and increase believability. Skin expressions may also be used to amplify the existing modalities, for instance, a “happy” face accompanied with “happy” textures may increase the perceived intensity of happiness compared to a face or a texture alone. Having conflicting expressions (e.g., a robot with a “happy” face and a “sad” skin) presents opportunities to design for more complex and layered internal states of the robot (e.g., hiding sadness with a fake smile).

Unlike many other nonverbal behaviors in human–human communication, skin is not normally regarded as a voluntary communication channel, nor does it have an explicit vocabulary for expression. Still, nature provides many examples of communicative skins that can be used as design metaphors, since humans have possibly developed an innate ability to interpret those signals during their co-evolution with other life forms [21]. We hope to make use of these metaphors in our design to evoke intuitive and affective meaning construction when users interact with the skin.

The involuntary nature of skin expressions provides interesting interaction possibilities, by tapping into a thus-far unused layer of meaning in human–robot interaction. Expressions may operate in a less noticeable and subconscious fashion, compared to purposeful facial and gesture movements. Ambiguity and subtleness of an expression may lead to more open interpretations [71] that vary for different people and scenarios. Interpreting such expressions could prove to be reliant on a subject’s prior knowledge and the interaction context. While the ambiguous nature of the skin expression can be challenging, it also provides room personalized designs, and opportunities for users to actively explore hidden meanings through interaction.