1 Introduction

Nowadays, Virtual Reality (VR) plays a large role in delivering immersive experiences for both business and entertainment. Researchers argue that to further enhance immersion in VR, physical interactions with digital contents should be conveyed within VR (Frederick Brooks 1999; Hoppe et al. 2018). Therefore, haptic feedback has a huge potential in enhancing VR experiences.

Many researches in the literatures presented devices that deliver a variety of haptic feedbacks in VR, such as to feel an object’s weight (Choi et al. 2018) or tangential or shear forces (Whitmire et al. 2018). These works mainly focused on the hands or arms, thus deemphasizing other regions of the human body. However, feedback on other regions, such as front and back torso, has high potential in increasing presence and immersion in VR (Tsetserukou 2010; García-Valle et al. 2016, 2018; Delazio et al. 2018). However, literature concerned with the front or back torso is scarce and limited.

Most haptic wearable devices explicitly provide a single type of feedback (Hardlight VR 2018; Konishi et al. 2016; Jones et al. 2004; Wu et al. 2010). Therefore, users have to wear many devices to receive multiple types of stimuli on various body locations. Multipurpose feedback devices are an emerging direction to address these challenges by triggering different stimuli, thus yielding higher immersion (Murakami et al. 2017; Choi et al. 2018; Dementyev et al. 2016; Ranasinghe et al. 2017, 2018; Harley et al. 2018).

To address the above challenges, we present HapticSnakes, which consists of two waist-worn serpentine-shaped robots capable of multiple types of feedbacks. Based on our literature review, HapticSnakes is the first to investigate a wearable snake morphology for providing novel feedbacks within immersive VR. Therefore, the aim of this paper is to expand our understanding of HapticSnakes feasibility and potential for enabling novel user experiences.

We implemented two prototypes (1) HapticSerpent (HS), which is lightweight and better suited for tapping and gestural feedback and (2) HapticHydra (HH), which is designed for delivering multiple feedback types through its multifunctional end effector that includes a gripper, a finger, a brush and a fan. We demonstrate how the versatility of the HapticSnakes enables it to reach different locations when worn in front or back torso, like the chest, abdomen, arms, neck, face, shoulder and back.

We extracted a design space based on our implemented prototypes, comprising different feedback types and control attributes to construct experiences. Accordingly, we conducted two user studies that evaluated our prototypes and the feedback types. Since taps are easy to deliver and useful for conveying a variety of cues, our first evaluation focused on distinguishability of tap locations and strengths on front and back torso. Participants had the highest accuracy in distinguishing feedback on the uppermost regions of the front and back torso and had superior overall accuracy in distinguishing tap strengths over tap locations.

To evaluate HapticSnakes within VR, our second user study investigated our robot’s capability of delivering multiple novel feedbacks in VR, as well as users’ impressions of using and wearing our robot. The results indicated that participants had distinct preferences for feedback types and were in favor of using our system throughout. The results also highlighted essential challenges in visuo-haptic mismatch while delivering feedback. Lastly, we extracted design considerations and discussed limitations and future research based on our work. This paper expands our brief report (Al-Sada et al. 2018) through extended robot design, analysis, implementations and evaluations.

In this paper, our contributions are the following: (1) the design and implementation of multi-haptic feedback wearable robots, (2) evaluation results that (a) gauge the users’ accuracy in distinguishing tap locations and strength on the front and back torso and (b) investigate our robot’s ability to deliver multiple novel feedbacks in VR as well as to investigate users’ impressions of receiving novel feedback in VR, including taps, gestures, shear forces, blowing air, brushing user’s hand and feeding the user.

2 Related works

Our work extends four strands of previous works: (1) vibrotactile feedback, (2) gestural feedback, (3) varied feedback in VR and (4) commercial products. We discuss each of these areas as follows:

Vibrotactile feedback has been thoroughly investigated in previous literatures. Numerous works presented vibrotactile systems for the torso using vests (Lindeman et al. 2004; Konishi et al. 2016; Jones et al. 2004; Wu et al. 2010). While vibrotactile feedback has a wide array of applications, it remains limited; as vibrations cannot stimulate sensations like shear forces or strong impact (Ion et al. 2015). Also, feedback is constrained to the area where the vibrotactile motors are fixed on.

Gestural feedback constitutes tangential and shear forces applied on the skin (Roudaut et al. 2013; Corley 2010), forming perceivable patterns like swipes or circles. Ion et al. (2015) presented arm-worn prototypes comprising tactors that can stretch and move along the skin, creating different patterns that can be used to convey information. Je et al. (2017a, b) introduced tactoRing, which used circular tractors embedded in a ring to drag the skin around fingers. Works in this domain mainly investigated haptic cues as potentially rich information mediums, where they could supplement interactions with wearable or mobile devices. HapticSnakes is different, as it tackles the domain of delivering multiple types of feedback in VR.

Varied feedback in VR Researchers have investigated a variety of feedback types, including haptic feedback, that aimed at attaining deeper immersion and presence in VR. Different works utilized fans to generate airflow, such as a fixed fan on a head-mounted display (Ranasinghe et al. 2018), multiple fans around user’s head (Rietzler et al. 2017) or wrist (Shim et al. 2018). Strasnick et al. (2017) presented a prototype that includes six small wrist-mounted brushes, where attributes like rotational speed, duration and direction can be utilized to convey different kinds of information. Force jacket (Delazio et al. 2018) uses inflatable modules that are distributed on the user’s torso, arms and back. Their prototype is able to convey different types of feedback by varying inflation speed, frequency or duration at each module. However, similar to vibrotactile motors, inflatable modules are prone to limitations in the diversity of expressible feedbacks and variety of feedback locations.

Eating in VR has also been investigated. Mae and Tuanquin (2017) explored behavioral changes in eating habits using VR, where they introduced redirected eating (RE), which controlled food desirability through olfactory and visual manipulations of real food within VR. Harley et al. (2018) explored involving various non-digital and passive sensory stimuli with eating in VR, such as touch, smell among others. Arnold et al. (2018) explored eating in VR as a part of a game mechanic, where users had to physically grab and eat food in order to win the game.

Compared to these works, HapticSnakes is proactive; it can manipulate food items (e.g. hand-in a cookie to the user) or directly feed users in VR. These capabilities enable a variety of intriguing VR experiences involving food, such as to tackle eating disorders (Ferrer-Garcia et al. 2013; de Carvalho et al. 2017).

Commercial products mainly use vests with embedded vibrotactile motors at varied locations to convey haptic feedback on the front and back torso or arms (Hardlight VR 2018; Eyeronman 2018; TactSuit 2019; Woojer 2018). ARAIG (2018) uses inflatable bladders to simulate impact or pressure applied to the torso. Therefore, we conclude that most commercial vests offer a single feedback type and exclude novel feedbacks.

Compared to previous works and products, the novelty of HapticSnakes is in its multi-feedback capability and flexibility. While previous works proposed wearables with a single-feedback method, HapticSnakes is able to deliver a variety of feedbacks using a single wearable robot. Such flexibility enables users to experience different feedback types without being overloaded with multiple wearables or accessories. Lastly, HapticSnakes’ flexible design enables it to reach various body locations, thereby making it applicable to a wider domain of experiences.

3 HapticSnakes

To embody the concept of HapticSnakes, we developed two robots that comprise wearable appendages with a flexible body. Our robots extend the design of snake robots with a fixed base, enabling them to reach a variety of locations and deliver different feedback types. The two robots are the following:

3.1 HapticSerpent (HS)

Design objective The objective of the HapticSerpent is to be able to deliver taps and gestures in different locations and force magnitudes with the lightest possible weight. Moreover, users should be able to wear the robot facing the front or back torso.

Robot design The robot’s body comprises six serially connected hobby servomotors as shown in Fig. 1 (EZ Robot (EZ Robot)), stall torque = 1.9 Nm). The servomotors are linked together using plastic brackets, which are light in weight, and their formation can be reconfigured to match different body dimensions.

Fig. 1
figure 1

Front, side and oblique views of HapticSerpent

Full size image

Dimensions and attachment The robot weighs 742 g and its total length is 51 cm, and both robots are mounted on a base with extended brackets and attached to a multitool vest (Fig. 1), weighing 300 g. The vest enables easy wearability, adjustment and fit for a variety of users.

End-effector We used a 3D printed bracket (W = 3 cm, H = 3 cm, L = 6.5 cm) and attached it to the last servo to be used as an end effector. We chose this end effector size as it is slightly bigger than a human finger, which provides a bigger contact surface when applying taps. Also, this end effector is long enough to enable adjusting its direction when delivering feedback. Other end effectors with varied dimensions or shapes can also be used, such as softer or sharper end effectors.

Control and power To control the robot, we utilized an EZ-Robot control board (EZ-Robot). We designed network-invokable controls on top of the EZ-Builder framework to enable easy creation and playback of movements. We powered the robot using 8 v 1800 mA Li-Po battery (approx. 25 min of continuous use).

3.2 HapticHydra (HH)

Design objective The objective of the HapticHydra (HH) is to be able to deliver multiple types of feedback to the user’s front and back torso, hands and face.

Robot design Similar to the HS, the HH is designed as a snake-like robot, yet the configuration of servomotors is slightly altered (Fig. 2). The three servomotors of the base are of type Robotis MX64AT (Robotis)(Stall torque = 5.5 Nm), which were selected for their high torque and PID control capabilities. The upper three servomotors are of type Robotis AX12A Stall torque = 1.5 Nm), which are used to position the end effectors for applying feedback.

Fig. 2
figure 2

Front and side views of HapticHydra. The multifunctional end effector is able to deliver multiple types of feedback as it is equipped with a gripper, a fan, a finger-shaped effector and a brush

Full size image

The lower brackets connecting the stronger servos are made from aluminum, while the upper ones are made out of plastic. This structure minimizes flexing and vibrations that may occur during rapid robot movements. We used servomotors as they provide a good trade-off between power and weight for our intended applications, enabling the robot to withstand the weight of the end effector and deliver the intended feedbacks.

The design of the multifunctional end-effector The HH’s end effector is optimized to offer multiple feedback types; it enables easy selection and delivery of varied feedbacks while being lightweight. Other end effectors, such as anthropomorphic robotic hands, are limited in their feedback variety (e.g. they are unable to deliver airflow or tickling stimuli). Moreover, robotic hands are complex in structure; they are relatively heavy (e.g. Allegro Hand weighs 1.8 kg) and require sophisticated controls of the hand pose and finger locations to deliver feedbacks. In comparison, the HH’s multifunctional end effector is superior in feedback variety, control over the type of delivered feedback and its associated attributes, while also being lightweight.

The rotary end effector (Fig. 2) is able to deliver a variety of stimulations and experiences. It is comprised of a plus-shaped 3D printed structure, with a different end effector type at each of its ends. This structure enables us to easily switch between four different types of feedback by simply rotating the whole structure.

We equipped each side with a different end effector type: a brush, a gripper (PhantomX), a fan and a finger, as shown in Fig. 2. By controlling the robot arm’s posture, we can control various feedback attributes, like the location and frequency of feedback, as well as the amount of exerted forces and angle of contact when delivering haptic feedback.

Dimensions and attachment The robot weighs 1.5 kg and is 42 cm in length (measured from the robot’s base). The rotary end effector weighs 150.8 g, 18 cm in width, 17.8 cm in length and 0.5 cm in height. The finger-shaped end effector has a radius of 1.6 cm. We used the same vest from the HS to attach the HH using a custom 3D printed bracket fastened using straps.

Control and power We extended the Robotis SDK (Robotis) to customize and integrate our controls. Our control software allows us to create and playback movements and motions through direct teaching (i.e. physically moving the robot to the desired pose). Motions can be played with varied attributes, such as varied playback speeds or joint angles. Moreover, we created a WebSocket network interface (WebSockets) with all robot commands, so that they can be easily invoked from VR environments. Our software was designed this way to enable rapid prototyping, experimentation and easy integration with different systems. The control software runs on a GPD win mini PC (GPD). To power our system, we used a 12v 1800 mA Li-Po battery that can provide approx. 20 min of continuous use of the robots.

4 HapticSnakes design space

In order to study different feedback types in HapticSnakes, we analyzed the robots form factor, workspace and end effectors. Then, we juxtaposed the extracted feedback types with robot control attributes, such as speed, applied torques and motion. Accordingly, we extracted a number of dimensions for designing experiences (as shown in Fig. 3), and they are presented in the next subsections.

Fig. 3
figure 3

Design space of HapticSnakes. Each design dimension corresponds to a specific feedback category and includes specific attributes

Full size image

4.1 Feedback and attributes

4.1.1 Taps

Taps resemble a force applied to a specific area for a brief amount of time. Taps are versatile and may be used to convey a variety of visual stimuli, events or notifications.

Design attributes Location constitutes where the feedback is delivered, which is applicable to all feedback types. When mounted on user’s front waist, HapticSnakes reachable workspace includes the user’s face, neck, torso, shoulders, arms and hands. When mounted on the back torso, reachable areas are the back torso, back of the neck, shoulders, arms, hands and head. Strength resembles the amount of applied force on impact, while direction represents the angle of applied force with respect to target area.

4.1.2 Gestures

Gestures allow for prolonged forces to be applied to the body at varied locations and directions, creating shapes like a circle, zigzag or a swipe. HapticSnakes is able to apply gestures on both users’ worn clothes and skin.

Design attributes In addition to location, strength determines the magnitude of applied force on the user’s body. Duration represents the time required to complete one gesture, while Trajectory corresponds to the sequence of movement path for creating the gesture.

4.1.3 Shear forces

Shear forces are carried out by continuously applying a force against a specific body region. They are useful in representing physical interactions like nudging, poking or objects that are stuck to the user’s body.

Design attributes Location attributes are applicable (as explained in taps). Strength resembles how much pressure is applied on the user’s body, while direction resembles the angle of applied force with respect to the application area. Duration determines how long the shear force is applied.

4.1.4 Airflow

The mounted fan is able to generate an airflow in different body regions, which is essential for increasing immersion (Ranasinghe et al. 2017, 2018; Rietzler et al. 2017). Unlike previous systems which focused on the face, HapticSnakes can deliver such feedback to other locations. For example, an airflow can target the neck or hands if a hand-tracking system is used.

Design attributes In addition to location, Intensity corresponds to amount of blown air on the user, which can be controlled by either the fan speed or distance between the fan and target area. Direction corresponds to the angle of the fan with respect to the target area. For example, a fan facing the user can blow air upward or downward, such that the airflow can be felt coming from under the chin or above the nose.

4.1.5 Brushing

Brushing on the user’s skin can generate a variety of tactile sensations, from tickling to scratching. Such feedback can be paired with a variety of visual stimuli, such as to resemble touching hair, fur or other textures.

Design attributes In addition to location, brushing has unique attributes. Intensity refers to how much force is applied on the users’ skin, less intensity resembles a gentle experience like tickling, and higher ones convey an immense sensation similar to scratching. Trajectory comprise the path taken during brushing, and Speed determines how fast the brush travels along a path.

4.1.6 Gripper-based experiences

This category covers several experiences that are delivered using the gripper: first, pinching the user’s skin at different locations; second, manipulating objects, such as handing in objects to users or feeding them; and lastly, pulling the user’s clothes.

Design attributes In addition to location, strength resembles the magnitude of the pulling force applied on user’s cloths or strength of pinching the skin. Speed resembles how fast the pinch is carried out or how fast the clothes are pulled. Direction determines the angle of pulling clothes.

4.2 Designing experiences

Designers can convey a variety of experiences by selecting different feedback types and attributes from the design space. Complex experiences can be created by sequencing feedback with different attributes. For instance, repeating a tap twice on the shoulder may resemble being patted by a virtual character.

Feedback can be designed for interacting with different applications, whether in VR or not. We demonstrate how we utilized the design space to construct experiences for delivering feedback in each of the user studies.

4.3 Discussion and limitations

The applicability and quality of experiences mainly depend on the robots’ design factors, specifications and controls. To optimize the robots’ design for specific feedback types, the design space dimensions and attributes of a desired experience should be expressed as robot design parameters, thereby determining adequate mechanical designs, end effectors types and level of control over the feedback attributes. Similarly, delivering feedbacks with specific attributes requires expressing them as robotic control parameters, such as servomotor speeds, angles, applied torques and robot poses. Therefore, transforming the design space’s feedback types and attributes to robotic parameters is essential for both optimizing the robot’s design and ensuring accurate feedback delivery.

There are further attributes that cannot be implemented without significant enhancements to our robots’ hardware or controls: for example, acceleration, like applying shear force or gestures with increasing speeds, or variant intensity, like applying a gesture with gradually increased strength. These factors require further optimizations to the robot’s mechanical design, control and accurate tracking of the user’s body.

Attributes like feedback strength and intensity are subjective dimensions: a tap that is considered painful by a user could be perceived as weak by another. These individual differences among users should be considered while designing each experience.

5 Evaluation 1: Distinguishing taps on the front and back torso

5.1 Study design

Taps present a versatile haptic feedback medium. They can be easily delivered with basic mechanical designs, varied attribute and can be used to resemble a wide variety of VR events (e.g. bumping into objects or being poked by virtual entities). However, related works are scarce for novel haptic cues in general, and especially taps applied on the front and back torso. Therefore, this study aims at broadening our understanding of applying taps as general haptic cues.

Objective The main objective of our study is to investigate user’s capability to distinguish the intensity and location of taps applied to the front and back torso. We accordingly utilized our design space attributes to determine a tap’s location and strength for our study.

5.1.1 Calibration of tap strengths and locations

Feedback regions Segmentation strategies of the feedback regions on the front and back torso are varied in previous works. For example, Jones et al. (2004) stimulated nine points on the lower abdomen and back, while Yang et al. (2002) used a total of 60 vibrotactile motors surrounding the torso (approx. 20 for each the front and back torso). Some previous works also chose different location arrangements based on their evaluation objectives: for example, to evaluate social acceptability of pinching and rolling to interact with smart garments (Karrer et al. 2011), which used six feedback points on the torso. Another study (Wagner et al. 2013) separated the front torso to six regions to evaluate on body-touch interactions.

Our evaluation objective is to explore the potential of delivering taps to the front and back torso. To the best of our knowledge, no previous work has investigated this type of feedback using this form of robots. Therefore, we build upon the model proposed by Karrer et al. (2011) that had the closest evaluation objective and feedback locations to our evaluation, by splitting larger regions to smaller ones and introducing two more extra regions on the chest (which were excluded from Karrer et al.’s work due to social unsuitability for their interaction method). Since our robots can accurately tap smaller regions, we decreased the sizes of the feedback regions in a similar way to what have been used in haptic vests (e.g. Jones et al. 2004). Therefore, we introduced a total of 16 regions (Cells) on each of the front and the back torso.

The 16 cells (Fig. 5) have a vertical spacing of 5–8 cm that depends on each participant’s chest and back size. These spacings are also used to compensate for potential operational errors, such as slight robot misalignments that may occur from continuous or rapid robot movements.

Location calibration Cells 1 through 4 are aligned horizontally to four points on the collarbone and shoulders on the front torso, and inner and outer edges of the shoulder bones on the back torso. As previously mentioned, the remaining 12 cells are aligned with 5–8 cm vertical spacing. On the back, the cells were slightly shifted to the edges to avoid hitting the neck or spinal cord. Our robot was calibrated to tap the user’s body in each point, taking into consideration dimensional differences between users’ bodies, as well as anatomical differences between males and females.

Strength calibration The robot was calibrated to tap the center of each cell from an approximate distance of 8 cm. This distance was chosen as the noise generated by servomotors from this distance at different speeds was undistinguishable, therefore, nullifying its potential effect.

Next, the strength was determined subjectively by participants; servomotor speeds were continuously adjusted, within a specific range of servomotor speeds for each of the strengths, until the difference between strong and weak taps was easily distinguishable by each participant. We have chosen subjective strong and weak feedback values to accommodate variations in users’ clothes that may affect sensed feedback (e.g. a thick shirt, or multiple layers of clothes).

The calibration process was repeated for each cell, covering both weak and strong feedbacks. To ensure applying sufficient force, strong and weak taps were tested and verified by users after calibrating each cell and before moving on to the next cell. For safety, we adjusted the servos speeds between 5 and 10 rpm for weak taps and 20–25 rpm for strong taps. The calibration process took approx. 50 min per user.

We tested the amount of exerted forces when applying taps with specified speeds using a force sensor (“Appendix 1”). Using the HS for the front torso, the exerted forces are approx. 2.93 N for weak taps and 5.31 N for strong taps. Using the HH for the back torso, the exerted forces are approx. 3.30 N for weak taps and 7.40 N for strong taps. Despite using the same servo speeds, the HH had a slightly higher amount of applied forces due to its weight and momentum when applying taps.

5.1.2 Participants and procedure

Participants We hired 20 college students (age m = 22.80, SD = 2.94, 11 females). They were distributed evenly in two groups (six females in front torso group). Participants came from different backgrounds, and all participants had a prior knowledge of VR through research or commercial platforms.

Procedure The user study was carried out in the same manner for both the front and back torso (Fig. 4). We started with an introduction to our work and the robots, followed by a profiling questionnaire. Next, we carried out the calibration processes as described. Next, participants took a familiarization tutorial, which included a single dry run of weak and strong taps for each of the 16 calibrated cells.

Fig. 4
figure 4

User study conditions. The HS and the HH were used, respectively, for taps in front and back torso

Full size image

The trials phase started by first blindfolding the participants and then subjecting them to feedback on the 16 cells. After each tap, participants had to verbally indicate the cell number and strength of the felt tap. Similar to the procedures of previous related studies (Luzhnica et al. 2016; Wilson et al. 2014; Gil et al. 2018; Cholewiak and McGrath 2006; Diener et al. 2017), we randomized and repeated the trials three times on each cell and strength level to minimize potential feedback errors and learning effects. Therefore, each participant was subjected to 96 taps (16 cells × 2 strengths × 3 repetitions). We have also monitored the robot’s position and feedback delivery for potential errors throughout the user study.

After the study, participants took a 5-point Likert scale usability questionnaire (1 is Disagree/Bad, 5 is Agree/Good), based on the Questionnaire for User Interaction Satisfaction (QUIS) (Chin et al. 1988). Lastly, each participant had a semi-structured interview covering aspects of usability and wearability. Each user study lasted approx. 2 h.

5.2 Results and analysis

In this section, we analyze the accuracies gathered from different conditions (Fig. 5). We start by highlighting our main analysis objectives, followed by the analysis results and discussion.

Fig. 5
figure 5

Calibration cells, average accuracy of distinguishing feedback strengths and locations at each cell location for all participants (standard deviation values in brackets). Results of the statistical significance tests on various regions are also illustrated (discussed in Results and analysis)

Full size image

5.2.1 Accuracy of distinguishing taps

We focused our investigation on exploring three main aspects of participants’ accuracies that we believe are essential for designing feedback for future experiences:

Q1 Whether or not the accuracies in upper regions are higher than lower regions. Since we used a within-subjects study design for each front and back torso, we used a repeated-measures ANOVA that compared rows 1, 2, 3 and 4 in each condition to validate our objective.

Q2 Whether or not the accuracies were higher in the chest than the abdomen on the front torso, and shoulder-blades than lower-back on the back torso. To form mentioned regions, we combined rows 1 and 2 to form the chest region and rows 3 and 4 to form the abdomen region on the front torso. Similarly, we combined rows 1 and 2 to form shoulder-blades region, and rows 3 and 4 to form the lower-back region on the back torso. We compared the results on the front and back torso separately, and we used paired-sample t tests due to our within-subjects user study design.

Q3 Whether or not the accuracies were higher in peripheral regions than in inner regions. We investigated this aspect as previous works suggested a potential difference in accuracies among peripheral and inner regions (Jones et al. 2004). We formed peripheral regions by combining columns 1 and 4 (as shown in feedback matrix of Fig. 5) and inner regions by combining columns 2 and 3. We used paired-sample t tests to compare accuracies across these regions within each front and back torso.

The following subsections examine each of the questions within the conditions of our user study.

5.2.1.1 Analysis of taps on the front torso

Distinguishing locations of strong and weak taps: Q1: We conducted a repeated-measures ANOVA between the four rows, which showed that the accuracies statistically differed for strong taps using Greenhouse–Geisser correction (F(1.10,3.29) = 16.81, p < 0.005) and weak taps (F(3,9) = 16.83, p < 0.001). In strong taps, post hoc tests using the Bonferroni correction showed significant difference between rows 1 and 2 (p < 0.05), rows 1 and 4 (p < 0.005), which indicate a generally higher accuracy in row 1 in comparison with other rows. In weak taps, a similar procedure was repeated, which only showed significant difference between rows 2 and 4 (p < 0.05).

Q2: We used a paired-sample t test to compare the accuracies of the chest and abdomen regions, which showed a significant difference in strong taps (t(7) = 3.37, p < 0.05) and in weak taps (t(7) = 4.66, p < 0.005). These results indicate that participants’ accuracy of distinguishing feedback on the chest was generally higher than the abdomen region. To address Q3, our tests did not show significant differences in accuracy between inner and peripheral regions.

Distinguishing tap strengths: Q1: Repeated-measures ANOVA results showed a significant difference in distinguishing accuracies of strong taps (F(3,9) = 11.70, p < 0.005). However, pair-wise comparisons did not show significant differences between regions, likely because of the limited sample size and used correction method (see discussion). Q2: Accuracies on the chest and abdomen were only statistically different for strong taps (t(7) = 5.58, p < 0.001). Significant differences were not observed in the evaluation of Q3.

Qualitative analysis Participants rated “I can easily distinguish the feedback location among different cells” with 2.5 (SD = 0.85) and “I can distinguish feedback location among contiguous cells” with 2.70 (SD = 0.95). Several participants also indicated that identifying feedback on the edges of the torso is easier than the center, yet statistical analysis did not reveal a significant difference. As shown in Fig. 5, both strong and weak taps were easily distinguishable by users, even when participants could not accurately distinguish the location of taps. Participants rated the ease of distinguishing tap strengths with 3.40 (SD = 1.07). Participants rated “Did the taps feel painful?” with 1.70 (0.82) and did not report any specifically painful feedback cells.

5.2.1.2 Analysis of taps on the back torso

Distinguishing locations of strong and weak taps: Q1: Repeated-measures ANOVA showed significant differences in distinguishing strong tap locations (F(3,9) = 15.53, p < 0.001) and weak tap locations (F(3,9) = 17.70, p < 0.001). Post hoc tests using the Bonferroni correction did not yield significant differences between accuracies of strong taps, while weak taps had a significant effect between rows 1 and 2 (p < 0.05). Similar to the front torso, we believe a larger sample could have yielded significant results in pair-wise comparisons.

Q2: Paired t test results indicate higher accuracies for shoulder-blades than lower-back in both strong taps (t(7) = 5.65, p < 0.001) and weak taps (t(7) = 3.696, p < 0.05). Q3: The accuracy of distinguishing feedback on peripheral regions was found to be higher than inner regions in strong taps (t(7) = 4.89, p < 0.01), while weak taps did not show any significant difference. Therefore, we conclude that distinguishing tap locations had higher accuracies in upper regions than in lower regions of the back torso.

Distinguishing tap strengths: Similar to our previous analysis, we carried out a repeated-measures ANOVA for Q1, followed by a paired-sample t test to compare the accuracies on the shoulder-blades and lower-back (Q2), and a paired t test to validate accuracies on inner and peripheral regions (Q3). As expected, no statistical difference could be observed in all tests as the accuracies were generally high across all cells (strong taps m = 82.08, SD = 11.98, weak taps m = 79.17, SD = 14.93).

Qualitative analysis: Overall, we believe participants thought it was difficult to distinguish the feedback locations. Participants rated “I can easily distinguish the taps applied on different cells” with 2.50 (SD = 1.08) and “I can distinguish feedback between contiguous cells” with 2.30 (SD = 1.06). In contrary, participants rated their ease of distinguishing feedback strengths with 2.90 (SD = 1.10), which further confirms the high accuracies and our analysis results of distinguishing feedback strengths. Although participants generally thought the feedback was not painful (m = 2.30, SD = 1.25), the interviews revealed that some participants thought that some feedback areas were painful. Three participants specifically indicated that cells 6 and 7, located right above the scapula of the shoulder bone, felt painful and should be avoided.

5.2.2 Overall analysis

5.2.2.1 Distinguishing taps on front and back torso

Distinguishing tap locations: On the front torso, the overall accuracy of distinguishing the locations of weak taps is (m = 50.21%, SD = 19.53) and strong taps is (m = 49.37%, SD = 20.90). On the back torso, the overall accuracy of distinguishing the locations of weak taps is (m = 58.13%, SD = 22.6) and strong taps is (m = 56.70%, SD = 26.58).These results indicate that the overall accuracy of distinguishing tap locations is not very high, yet participants had significantly high accuracy in upper regions of the front and back torso (as indicated in Sect. 5.2.1).

We compared strong and weak feedbacks separately on each front and back torso, with the goal of identifying which of condition had a significantly high accuracy. While there was not a difference on the front torso, paired-sample t tests on the back torso conditions showed that weak tap locations were more distinguishable than strong tap locations on the back torso (t(15) = 2.67, p < 0.05).

Moreover, to identify regions with significantly high accuracies, we compared the overall accuracies in distinguishing strong and weak tap locations between the front and back torso. We conducted a one-way ANOVA with Bonferroni correction that compared all the four conditions of distinguishing tap locations on front and back torso. Results did not show a significant difference in the accuracy within a specific condition; therefore, we conclude that users had a similar accuracy in distinguishing tap locations across different conditions.

Distinguishing tap strengths: Users had an overall high accuracy in distinguishing tap strengths on front and back torso. On the front torso, their overall accuracy in distinguishing strong taps is (m = 85.83%, SD = 13.42) and weak taps is (m = 96.04%, SD = 3.49). On the back torso, overall accuracy of distinguishing strong taps is (m = 82.1%, SD = 11.98) and weak taps is (m = 79.17%, SD = 14.93).

To compare the different conditions, we carried out a one-way ANOVA with Bonferroni correction that compared the four conditions of distinguishing tap strengths in front and back torso. Results showed significant difference (F(3,60) = 29.36, p < 0.001). Pair-wise comparisons showed significant differences between distinguishing strong taps on front torso and strong taps on the back torso (p < 0.05), and strong taps on front torso and weak taps on the back torso (p < 0.001), indicating a higher distinguishability of strong taps on the front torso than mentioned conditions. Weak taps were also found to be more distinguishable on the front torso than weak taps on the back torso (p < 0.001), but no significant difference was found between weak taps on the front torso and strong taps on the back torso. Therefore, we conclude that tap strengths were generally more distinguishable on the front torso than on the back torso.

We believe that these results justify the limited findings of our statistical analysis as there were not many significant differences among the cells because the accuracies were generally high.

Tap locations and strengths We compared the overall accuracies of distinguishing tap locations with tap strengths across both the front torso and back torso. We used paired-sample t tests to compare the overall accuracies of locations with strengths, separately on each front and back torso. Participants had superior accuracy in distinguishing tap strengths over locations, in both front torso (t(31) = 12.40, p < 0.0001) and back torso (t(31) = 5.07, p < 0.0001). We discuss the impact of these results in Design considerations section.

5.2.2.2 Male and female accuracy analysis

We analyzed the accuracies for both front and back torso conditions for males and females. Since the number of males and females is unbalanced in the front torso condition (six females and four males), we used Welch’s t test. Females (m = 56.94%, SD = 36.16) significantly outperformed males (m = 40.10%, SD = 33.69) in their overall accuracy of distinguishing the locations of weak taps on the front torso (t(142) = 3.007, p < 0.05, Cohen’s d = − 0.481). Similarly, a significant effect was found in overall females accuracy (m = 56.60%, SD = 34.58) compared to males (m = 38.54%, SD = 40.37) in distinguishing the locations of strong taps on the front torso (t(120) = 2.93, p < 0.05, Cohen’s d = 0.480).

We analyzed the results on the back torso under the same conditions and using the same statistical test, yet results did not show significant differences (“Appendix 2” contains further results). We further discuss these results in Discussion and limitations section.

5.2.3 User satisfaction, comfort and fit

Participants rated their overall satisfaction with the HS with 3.80 (SD = 0.92) and the HH with 3.00 (SD = 0.0). Generally, they really liked the idea of feeling the taps at different intensities and in different locations, and thought it could make a very novel VR experience.

Participants rated the comfort of the HS with 3.80 (SD = 0.79) and weight with 3.70 (SD = 0.95), while the HH was rated with 2.70 (SD = 1.34) for comfort and 3.10 (SD = 1.10) for weight. Although both the HS and the HH had close ratings for weight, we conclude that the overall perceived comfort was better for the HS, as its smaller size and lighter weight enabled it to be more comfortable throughout the study. Participants also thought that both robots should be lighter in weight, so that they can be worn for prolonged periods of time.

5.3 Discussion and limitations

The results of our analysis on distinguishability of taps on different regions indicate that the accuracies were higher in the upper regions of the front and back torso. We attribute the high accuracies on upper regions to closeness to the collar and shoulder bones on the front torso, and shoulder-blades on the back torso. Apart from strong taps applied to the back torso, results also showed that feedback was not significantly distinguishable in inner or peripheral regions. Therefore, we conclude that participants had a similar overall accuracy in distinguishing feedback in inner and peripheral regions.

The overall analysis also showed interesting aspects. The overall accuracies of distinguishing tap locations were not very high, and we conclude that participants had similar accuracies across all conditions. The results also point out that despite the overall high accuracy in distinguishing tap strengths, participants had significantly higher accuracies in the front torso than on the back torso. Lastly, the overall distinguishability of different tap strengths is significantly higher than tap locations, both on the front and back torso. Therefore, we believe these findings can be utilized to optimize tap delivery within different experiences: for example, delivering more precise taps on the upper regions than lower ones. We discuss essential implications for designing future experiences in Design considerations section.

The accuracy comparisons between males and females also revealed interesting insights. There was a statistical difference in distinguishing the locations of strong and weak taps on the front torso, where females had significantly higher accuracy. We attribute these results to anatomical differences between males and females on the front torso area, which contributed to higher female accuracy in distinguishing tap locations. We believe future systems should accommodate these anatomical differences during the calibration process, and also adapt user experiences to exploit such differences (e.g. an experience with a female participant could apply taps across many cell locations, and vice versa for males).

Another essential factor is the size of the users’ torso when using the HapticSnakes. We believe the width of the users’ front and back torso affected feedback accuracy. Users with larger bodies had sufficient distance between cells, which we believe have increased their distinguishability of feedback on different regions, and vice versa. Future systems should attempt to compensate for such variance in the users’ torso sizes.

In order to maintain safety and meet our evaluation objectives, we limited the servomotors speeds and the amount of exerted forces in this study. The robots are capable of exerting much higher and lower amount of forces, such as by increasing or decreasing the servomotor speeds. Therefore, future work should investigate the effect of exerting varied amount of forces, thoroughly investigating the full spectrum of possible applied forces on different body regions.

Lastly, we believe some of our statistical analysis results were slightly affected because of the used correction method (Type 2 errors). These tests may also indicate the need for a larger sample to show significant difference among users’ accuracies.

6 Evaluation 2: Investigating HH’s novel feedbacks in VR

6.1 Study design

The scarcity of studies about multi-haptic feedback devices motivated us to investigate the HH’s usability and potential for use in VR. The novelty of the HH is in its ability to deliver multiple types of novel feedback in VR. Previous works on novel haptic feedback wearables focused their evaluations on validating the capabilities of their designs in delivering novel feedbacks (Delazio et al. 2018; Al Maimani and Roudaut 2017). Therefore, we focused our study on investigating the unique capability of the HH in delivering multiple novel feedback types and therefore contributing with design insights, potential challenges and limitations of robots based on the HH.

Objectives The main objectives of this study are to (1) investigate the HH’s capability to deliver multiple feedbacks in VR and to (2) explore users’ impressions about using the HH and experiencing its novel feedbacks in VR. Accordingly, we evaluated taps, swipes, shear forces, brushing against skin, blowing air and feeding, where each was matched with visual and auditory stimuli.

6.1.1 Participants and apparatus

We hired ten college students, aged between 19 and 31 (m = 24, all males), who came from different disciplines and eight nationalities. Six participants indicated that they have experienced VR before and not had prior experiences of haptic feedback in VR.

Our experience was fully developed with the Unity3D (Unity3D). We ran the experience on a computer connected to an HTC Vive head-mounted display (HTC Vive) with headphones (as shown in Fig. 6).

Fig. 6
figure 6

(1) User study conditions and hardware. We utilized HTC Vive to deliver our VR experience. Two PCs were used, one for running Unity3D/VR experience, and the other to control the robot. (2) The VR environment involves several experiences with changing weather, ambient effects and day/night cycles

Full size image

Our Unity3D software communicates with the HH using WebSockets. As described in the HH implementation, robot control commands are sent from our software to trigger each robot movement and feedback delivery in synchronization with the visual and auditory stimuli running in our experience.

6.1.2 Experiences and story

We developed an immersive VR experience with a story, visual and auditory effects to match HH’s delivered stimuli. The story is about a person enduring a nightmare consisting of seven experiences that are shown and explained in Fig. 7. These experiences are not interactive, and their flow is predetermined. Each experience lasts for around 20–30 s. Varied environmental, atmospheric and sound effects were added in each experience, therefore contributing to the overall immersion and flow of experiences (Thomas Sheridan 1992; Ekman 2013; McMahan 2003).

Fig. 7
figure 7

After welcoming the user, the experiences start: (1) a cat comes and rubs her body against the users’ left forearm while meowing. Next, (2) a character approaches the user and pats his/her right shoulder twice, saying “you are tired, you should go to sleep.” Then, the screen fades to black and the user is taken to a scene where it is dark and raining, with matching thunder and rain sounds (The shown screenshots were taken without ambient effects for clarity). (3) An archer appears in front of the user and shoots an arrow to the user’s chest, then disappears into the darkness. (4) A knight appears and slashes user with his sword, then backs away and disappears. (5) A brawler appears and punches the user in the chest, (6) sending them flying into the sky. (7) the user is awoken from the dream, and he/she is fed a cookie

Full size image

6.1.3 Feedback calibration

We instructed users to stand straight and maintain their pose for the calibration. We used the design attributes of our design space to calibrate each experience. To ensure safety and to ease calibration, the direction was fixed for all feedbacks to be perpendicular to the user’s body, and the maximum strengths were fixed with a servo speed of 25 rpm, from a distance of approx. 8 cm (similar to the previous study). Calibration took 25 min per user and consisted of seven calibration processes for each of the seven experiences:

1. Cat Rub Brushing was carried out on the user’s left forearm, where we applied two swipes against the user’s skin with a length of 3 cm and a speed of 10 rpm. These attributes were chosen as they mimic the cat rubbing itself on user’s arm in VR, and brushing was repeated twice to enable users to experience brushing as a single swipe could be too short to feel. 2. Patting To resemble gentle pats on the user’s shoulders, the finger end effector was moved toward the user’s shoulder at a speed of 10 rpm from a distance of 5 cm. This movement was executed twice, where the robot briefly rested against the users’ shoulder in between actions to match the VR experience. 3. Knight slashing we calibrated the end effector on the surfaces of cells 2 and 3 centers. The angle of the base servomotor was increased by approx. 5° to generate a force for a length of 12 cm, a speed of 10 rpm and in a straight trajectory between the cells, therefore applying an approximate force of 3.63 N. We chose these cells as the upper chest has a semi-flat area and high sensitivity (as explained in Evaluation 1), enabling us to execute the swipe easier than in other areas. The feedback attributes were chosen to match the sword slash in the VR.

4. Arrow A shear force was applied to cell 4 on the chest. Upon position calibration, the angle of the base servomotor was increased by 10° to apply a force for a duration of approx. 4 s. The angle was increased by mentioned amount to create a shear force against the user’s torso that resembled the arrow in VR. 5. Punch was calibrated by moving the robot at the speed of 25 rpm and distance of 8 cm from the area between cell 2 and 3, matching where the character punches the user in VR. Similar to our previous evaluation, we chose 25 rpm so that we achieve the highest impact force while maintaining overall operational safety of the robot. 6. Flying a fan was positioned 15 cm away in front of the users face and was manually controlled. The fan is situated so that it blows air on the lower side of the user’s face, since the upper side is occluded by the head-mounted display. 7. Feeding was done by moving the gripper to be approx. 6 cm away from the mouth. Such length is chosen to both match the VR character’s hands and to enable users to easily lean forward and eat the cookie.

Stimuli synchronization To match visual stimuli in VR and HH’s delivered stimuli, we compensated for the robot movements by calling them ahead of each visual stimulus. This was achieved by observing the robot’s movements and compensating them manually on the VR system. We also used a dedicated network router to minimize any effect of network latency.

6.1.4 Procedure

First, users were briefed about the purpose of the user study and the system. Then, all participants had a 5-min simple trial of HTC Vive, after which we carried out the calibration as described. Next, the user study is started as explained in Experiences and story subsection, after which participants took a post-study questionnaire and were interviewed. We also adapted QUIS (Chin et al. 1988) to this study and extended it with questions about users’ impression of each of the experiences. Each trial lasted for approx. 90 min.

6.2 Results and analysis

6.2.1 Analysis of experiences preferences

We asked participants to rate how much they liked each experience (5-point Likert scale, 5 is best). Participants rated flying (m = 5, SD = 0.0), cat rubbing (m = 4.2, SD = 0.91), knight slashing (m = 4, SD = 0.94), feeding (m = 4.4, SD = 0.70), patting (m = 4.2, SD = 1.14), punch (m = 3.9, SD = 1.19) and arrow (m = 3.9, SD = 1.20). These results indicate that the experiences were generally enjoyable.

We also asked the participants to rank the experiences quality and enjoyment by comparing the experiences with each other. We used a 7-point Likert scale (7 is best), where each experience could be allocated one unique rank. Ranking the experiences would enable us to extract insights about users’ individual preferences. The ranks and their distributions are shown in Fig. 8.

Fig. 8
figure 8

Distribution of the experiences as well as the mean rank (error bars represent 95% CI). Ranks were significantly different for flying–Patting

Full size image

To analyze the ranks, we ran nonparametric Friedman test, which showed significant differences in the distribution of the experience’s ranks (χ2 (2) = 14.31, p < 0.05). We followed with Wilcoxon signed-ranks test with Bonferroni correction, which only revealed significant difference between flying and patting only (Z = − 3.400, p < 0.001). Apart from flying and patting, the results indicate that the distribution of the experiences–ranks was generally the same. Therefore, we conclude that participants generally had distinct preferences of the experiences and were not statistically biased toward a certain experience.

6.2.1.1 Qualitative analysis

We analyzed each experience by evaluating its overall score and qualitative user feedback. We believe such analysis would provide deeper insights about individual preferences about each experience. We examine each of the experiences and report its overall rank score, which is calculated by summing all the participants-allocated sub-scores as follows:

1. Flying (Overall Rank Score: 59) Eight participants ranked this experience in their top 3. One participant mentioned “The effect of air blowing was very appropriate; it wasn’t too much or too little”, another added “…it was very realistic, it felt like I was really flying away.” Also, they indicated that auditory, visual and haptic stimuli were consistent, hence calling it realistic. Therefore, we believe the experience was very enjoyable.

2. Knight Slashing (Overall Rank Score: 43) Five participants ranked this experience within their top 3. Some participants mentioned “…it was most realistic because the whole slash was carried out” and “it was intuitive, the timing was good and the motion on the chest was intense.” This feedback indicates that the visual stimuli and slashing gesture on the body were well received. However, some participants thought that the slashing gesture should be stronger so that it is more consistent with the visual stimulus of slashing.

3. Arrow (Overall Rank Score: 42) Four participants ranked this experience within their top 3. Participants mentioned: “I felt the arrow hit and stuck to my body,” “the whole arrow effect felt realistic, the animation, timing and hit was believable,” which indicate that the experience was both enjoyable and consistent. However, some participants discussed some shortcomings. They thought the arrow’s animation should be faster and proposed increasing the feedback strength both for impact and when pressing against their bodies.

4. Feeding (Overall Rank Score: 42): Although four participants ranked it among their top 3, participants expressed mixed views about this experience. Some participants thought it was very novel and enjoyable, a participant mentioned “feeding is the most realistic experience, because I get to taste the food in VR.” Another added “It was good, eating the cookie was easy.”

In contrary, other participants mentioned some challenges: “I had to bend a little for eating the cookie,” “The cookie hit my chin when I triet to eat it.” These comments highlight the issue of correctly aligning the cookie in both VR and real world, so it would be easier to eat. A number of issues affected this experience, especially robot shaking in accordance with user’s movements, and the harness loosening upon extended use. Lastly, one participant raised an important safety concern “…Machines close to the face are dangerous.” We further discuss aspects of safety in Sect. 8.2.

5. Punch (Overall Rank Score: 36) Three participants thought it was among their top 3 experiences. However, four participants criticized the impact force, mentioning it should be stronger to resemble a punch. Moreover, six participants thought the end effector was too small to convey a fist, indicating a mismatch in the visuo-haptic stimuli. We further discuss such challenge in Sect. 6.3.

6. Cat Rub (Overall Rank Score:33) had mixed views, and four participants ranked this experience in their top 3. One participant said, “Cat rubbing is my favorite, I felt the cat on my skin when it jumped at me” and “the cat was unexpected, it was scary but awesome.” Although mentioned comments indicate that the visuo-haptic stimuli were well synchronized, three participants complained about some discrepancy in stimuli: “The brush is rough so I didn’t like how it feels,” and another added “…it should have been softer a bit, like a cushion.” Such discrepancy made them dislike this experience.

7. Patting (Overall Rank Score: 25) was least favored by the participants, and only two participants ranked it within their top 3. They mentioned: “…it is the most basic action compared to all others,” “it is not memorable.” These comments indicate that the experience was not enjoyable. Moreover, since we used the finger end effector to apply the pats, participants mentioned the difference in sensed feedback in comparison with the character’s hands, “The physical feel of the hand is very different” and “a pat should be all over my shoulder.” We conclude that the experience was not intriguing to users, and we discuss its limitations in the discussion section.

6.2.2 Wearability, comfort and weight

On a 5-point Likert scale (5 is best), the HH was rated with 3.6 (SD = 1.07) for comfort and 3.1 (SD = 0.99) for weight. Several participants reported pain and pressure against the back and abdomen while wearing the HH. Aggregating the results from the previous evaluation, we conclude that the ergonomic design and weight of the HH should further be improved, especially for prolonged usage sessions.

6.3 Discussion and challenges

Experiences evaluation The results indicate that participants generally enjoyed the experiences and had individual preferences. They rated their overall satisfaction of the HH’s experiences with 4.5 (SD = 0.71). Since pair-wise comparisons did not reveal significant differences between experience preferences, we conclude that each user had individual favorite experiences. Therefore, prior knowledge of the user’s favorite stimuli could be used for designing customized experiences per user to increase their enjoyment.

In addition to challenges similar to those of the HS, our analysis indicates a number of particular challenges:

Visuo-haptic mismatch While participants did not raise notable comments about auditory feedbacks, visuo-haptic mismatch was raised by several participants, especially in the punch and the patting experiences. Participants thought that the end effector should have resembled sensations delivered by human hands, especially in terms of surface area when patting or punching. Other participants thought the arrow feedback should have been sharper and stroked harder to better match the arrow in VR. We discuss a possible solution to this shortcoming in Sect. 7.2.

Immersion and distractions Some participants though that robot weight, movements and the inertia of its movements broke the immersion. They especially indicated that it affected them when such movements occurred without visual stimuli (e.g. when robot moves to prepare for feedback delivery). Reducing robot’s movement speed and weight could contribute to addressing these issues.

Validating novel feedback in VR the HapticSnakes is capable of numerous novel haptic feedbacks, such as tugging, tapping and pinching as explained in the design space. To the best of our knowledge, the effect of these novel feedbacks on immersion and presence has not been previously investigated. Therefore, an essential future research direction is to investigate the effects of novel feedbacks on immersion and presence through comparative studies (Ranasinghe et al. 2018; Hoppe et al. 2018).

7 Design considerations

Based on HapticSnakes’ design process and evaluation results, we identified several considerations for both designing robotic wearables with diverse feedback capabilities and for designing user experiences based on the HapticSnakes:

7.1 Delivering taps on front torso and back

Tapping feedback model Our evaluation results indicated that users had superior accuracy in determining feedback strengths over locations. Therefore, we propose the feedback model shown in Fig. 9. The model provides more feedback cells on the upper regions, where users reported the highest accuracies in distinguishing tap locations. Similarly, lower cells were combined, as users were hardly able to accurately pinpoint feedback locations on these regions. On the back torso, users were found to have high accuracy in peripheral regions for distinguishing the locations of strong taps. Therefore, the back torso model includes several peripheral and inner regions to enable delivering feedback to these regions.

Fig. 9
figure 9

Based on the evaluation results, the revised feedback models have more feedback regions in the upper than in the lower regions. For the back torso, lower regions are segregated to reflect users’ ability to distinguish inner and peripheral cells within lower regions

Full size image

Emphasize feedback strengths As the accuracy in distinguishing tap strengths was found to be superior to distinguishing tap locations, future robot should focus on the capability to deliver feedback with a large spectrum of intensities. Our current robot may achieve larger spectrum of feedback intensities by altering the servomotor speeds, distance of impact and torque values. Therefore, we intend to expand our work to explore the effect of applying taps with higher and lower strengths.

7.2 Designing multi-feedback wearables

Limb-specific wearables An essential challenge of our robots is in delivering feedback to limbs within interactive VR experiences. The user’s physical movements impose several difficulties on feedback delivery, such as overshooting locations or failing to deliver feedback as users move their limbs away. Therefore, smaller wearable robots can be designed to deliver feedback to moving limbs. These robots can be worn on hands and legs where they can independently deliver feedback to those regions. Wearing multiple devices could also contribute to resolving the feedback singularity shortcoming of our current designs, yet it could be cumbersome for users as they have to wear multiple robots.

Shape-changing end-effectors The rotary end effector has several challenges. It only has four types of sub-end effectors, and its cross-design makes it prone to accidental or unintentional feedback due to the length of each end. Shape-changing is the capability of altering various physical attributes, such as shape or texture (Rasmussen et al. 2012). A shape-changing end effector, as in Fig. 10, can take different shapes to convey feedback with varied attributes, like surface areas or sharpness. Moreover, feedback types can be diversified by embedding different modules, like including heat modules (Peiris et al. 2017; Ranasinghe et al. 2017, 2018) or water sprinklers, which allow for different ranges of feedbacks. Therefore, future research should investigate novel shape-changing end effector designs that embed different feedback modules.

Fig. 10
figure 10

A tentacle shaped end effectors with seven DOFs. It can take different shapes, therefore providing different haptic feedbacks like shear forces with a small or large contact areas or multiple contact points

Full size image

8 Research challenges and future work

In this section, we identify and discuss a number of research opportunities and challenges that arise from the design and evaluations of the HapticSnakes.

8.1 Feedback design and delivery challenges

Interactive VR Delivering feedback to the user requires prior knowledge of the user’s current posture. Within interactive VR, users may constantly move and take different postures, such as when walking around a tracked space or holding and manipulating objects. Therefore, such scenarios are prone to accidental feedback (e.g. user’s body bumps unintentionally to the robot while moving) and feedback delivery failures (e.g. the user leans backward as the robot attempts to deliver feedback on front torso, causing the robot to fail in tapping the user). The accuracy of the delivered feedback is bound by the system’s ability to quickly detect and adapt the user’s body movements. Therefore, we intend to integrate external tracking systems or high-speed cameras (Optitrack 2018; Okumura et al. 2011) which are widely used for VR and robotics applications. These tracking systems would enable rapid adaption of the robot’s movements to users’ posture or limb movements, therefore allowing us to address mentioned challenges.

Calibration There are several dynamic factors that should be considered to improve and automate calibration: firstly, user’s body dimensions, worn clothes, robot placements with respect to the user’s body, and individual feedback preferences. Secondly, misalignments of the robot that may occur because of loosening straps or prolonged robot use. Thirdly, attributes like intensity and strength are user subjective, and they should be calibrated prior to each VR experience to avoid visuo-haptic mismatch. Our current calibration approach is time-consuming and prone to all above issues. Future work should bring advances in context awareness and machine learning to automate calibration and to cope with these changeable factors.

Singularity of feedback HapticSnakes is only capable of delivering feedback to one location at a time. Future work should explore other morphologies that are optimized for one or multiple types of feedbacks or body regions. Therefore, we intend to investigate extending our robot with multiple appendages and end effectors, so that it can concurrently deliver feedback in multiple body locations.

Feedback authoring tools Although our current robot control method allows recording and playing back motions with different attributes, it is cumbersome for rapid creation of different VR experiences. Therefore, matching HapticSnakes with a suitable feedback design tool is essential for exploiting its capabilities for future applications (Schneider et al. 2015). We believe a feedback authoring tool, with a graphical user interface, should fulfill two objectives. First, it should provide parametric user control over the feedback types and their attributes. For example, designers may create a tap with specific angular velocity and strength level. Second, the tool should allow designers to pair HapticSnakes’ feedback with various VR events, like linking a sword slash in VR to a swipe on the chest by the robot. Presenting these two capabilities in a user-friendly interface is critical for designing experiences based on the HapticSnakes, especially for designers and developers that do not have a background in robotics. Therefore, we intend to extend our robot control software to contain more attributes as well as a feedback designer interface to enable rapid development of intriguing user experiences.

8.2 Mechanical design and safety

Mechanical structure HapticSnakes form factor is far more capable of intriguing haptic feedback. Recent advances in tendon-based and pneumatically actuated robots are promising for this application domain, as they are generally light in weight, retain a soft structure and are back-drivable (Li et al. 2013; Nakata et al. 2015). Such properties make them ideal for designing wearable robots that are durable, comfortable to wear and safe for daily use.

Safety HapticSnakes biggest challenge is safety. Our robots are equipped with high-torque servomotors, and their rapid movements to situate or deliver needed feedback in close proximity to the user’s body pose numerous hazards. Therefore, we intend to investigate two main directions to improve the safety of our robots. First, as previously mentioned, soft robotic-designs based on pneumatic or tendon-driven structures offer various safety features. Second, equipping the robot with sensors, such as proximity sensors or tactile sensors (Schmitz et al. 2011), can contribute to increasing the robots safety during operation. Therefore, we intend to investigate these sensors for use in HapticSnakes.

8.3 Further experiences

HapticSnakes’ feedback can be utilized for a variety of purposes: for example, to indicate different game status, like low health or to handicap limb movements during gameplay. Additional application domains are also in delivering information-rich notifications (Al-Sada et al. 2019; Luzhnica et al. 2016) or to break immersion in case of emergency. Future work should investigate further application domains for these emerging feedback types.

Seated VR experiences are also an interesting context for the HapticSnakes, since the challenges associated with user movements can be overcome when users are seated. Calibration and feedback delivery could be robust, as the user’s body can accurately be tracked within a small workspace, such as by using depth cameras, yielding superior feedback accuracy and safety.

9 Conclusion

In this paper, we presented the HapticSnakes, which is a waist-worn snake-like multi-robot system that can deliver a variety of novel feedback types in multiple locations. We presented two implemented robots, followed by the design space to create feedback based on our prototypes. We proceeded with an evaluation of distinguishing tap accuracies and strengths on the front torso and back. Our second evaluation investigated users’ impressions of novel feedback within VR. Based on our evaluations, we extracted a number of design considerations and discussed challenges and future work.

Compared to existing works, HapticSnakes is superior in delivering multiple types of feedback in varied locations, which can have intriguing applications across different domains. However, realizing HapticSnakes for daily use has various challenges and requires significant development to ensure stability and safety. We believe that future robots based on the HapticSnakes can have a huge potential in enabling realistic VR experiences. Such robots would enable experiences that merge a variety of feedbacks that are delivered throughout the users’ body, therefore contributing to further immersion and enjoyment within future VR experiences.