Numerous studies have demonstrated that visuospatial attention is a requirement for successful wo... more Numerous studies have demonstrated that visuospatial attention is a requirement for successful working memory encoding. It is unknown, however, whether this established relationship manifests in consistent gaze dynamics as people orient their visuospatial attention toward an encoding target when searching for information in naturalistic environments. To test this hypothesis, participants' eye movements were recorded while they searched for and encoded objects in a virtual apartment (Experiment 1). We decomposed gaze into 61 features that capture gaze dynamics and a trained sliding window logistic regression model that has potential for use in real-time systems to predict when participants found target objects for working memory encoding. A model trained on group data successfully predicted when people oriented to a target for encoding for the trained task (Experiment 1) and for a novel task (Experiment 2), where a new set of participants found objects and encoded an associated nonword in a cluttered virtual kitchen. Six of these features were predictive of target orienting for encoding, even during the novel task, including decreased distances between subsequent fixation/saccade events, increased fixation probabilities, and slower saccade decelerations before encoding. This suggests that as people orient toward a target to encode new information at the end of search, they decrease task-irrelevant, exploratory sampling behaviors. This behavior was common across the two studies. Together, this research demonstrates how gaze dynamics can be used to capture target orienting for working memory encoding and has implications for real-world use in technology and special populations.
When enabling touch interactions on a deformable surface, the user is faced with a number of nove... more When enabling touch interactions on a deformable surface, the user is faced with a number of novel interaction challenges not present on firm surfaces. For example, the material used for the deformable surface can completely change the set of interactions that can be used. In addition, touching with one finger affects the overall shape of surface, and deformation for each touch individually can be hard. In this paper we discuss two prototype systems designed to enable us to study such challenges. We discuss how such challenges also present design opportunities to enhance interaction vocabulary of touch interactions on deformable surfaces and offer guidance to future designers of such interactions.
L'invention porte sur des techniques de motif de tenue de dispositif tactile habillable. Un h... more L'invention porte sur des techniques de motif de tenue de dispositif tactile habillable. Un habillage sensible au toucher peut etre configure pour couvrir sensiblement les surfaces exterieures d'un dispositif informatique. L'habillage sensible au toucher peut comprendre une pluralite de capteurs d'habillage configures pour detecter une interaction avec l'habillage a des emplacements definis. Le dispositif informatique peut comprendre un ou plusieurs modules utilisables pour obtenir une entree de la pluralite de capteurs d'habillage et pour decoder l'entree afin de determiner des motifs de tenue qui indiquent comment le dispositif informatique est tenu par un utilisateur. Diverses fonctions fournies par le dispositif informatique peuvent etre selectivement activees et/ou adaptees en fonction d'un motif de tenue determine de maniere a ce que la fonction fournie puisse changer pour correspondre au motif de tenue.
L'invention, selon des modes de realisation, porte sur des dispositifs de saisie avec des sur... more L'invention, selon des modes de realisation, porte sur des dispositifs de saisie avec des surfaces tactiles multiples courbees. Un mode de realisation decrit comprend un dispositif de saisie tactile ayant un element geometrique courbe comprenant un capteur tactile, le capteur tactile comprenant un groupement d'elements de capteur integres dans l'element geometrique courbe et etant configure de facon a detecter un emplacement d'un contact realise sur une surface de l'element geometrique courbe.
La presente invention concerne un module de gestion d’interaction (IMM) qui permet a des utilisat... more La presente invention concerne un module de gestion d’interaction (IMM) qui permet a des utilisateurs de s'engager sur une surface interactive dans un environnement collaboratif en utilisant differents dispositifs d'entree, tels que des dispositifs de clavier et des dispositifs de souris. L’IMM affiche des objets numeriques sur la surface interactive qui sont associes aux dispositifs de differentes facons. Les objets numeriques peuvent comprendre des interfaces d'affichage d'entree, des curseurs, des mecanismes d'entree de touche programmable, et ainsi de suite. En outre, l’IMM fournit un mecanisme servant a etablir une trame de reference pour gouverner le placement de chaque curseur sur la surface interactive. De plus, l’IMM fournit un mecanisme permettant a des utilisateurs de faire une copie numerique d'un article physique place sur la surface interactive. L’IMM fournit egalement un mecanisme qui duplique les actions prises sur la copie numerique par rappo...
Projector-camera (pro-cam) systems afford a wide range of interactive possibilities, combining bo... more Projector-camera (pro-cam) systems afford a wide range of interactive possibilities, combining both natural and mixed-reality 3D interaction. However, the latency inherent within these systems can cause the projection to ‘slip’ from its intended target, detracting from the overall experience. Because of this, pro-cam systems have typically shied away from truly dynamic scenarios. In turn, research has been exploring latency reduction techniques across a range of domains, but these techniques typically focus on custom hardware, limiting their widespread adoption. We explore software-only predictive approaches to minimize the effects of latency in pro-cam systems. In this paper, we focus our predictive approaches on realworld objects under fast motion and on-body projection, improving projection accuracy on fast moving targets. Alongside this we explore automatic latency measurement techniques, allowing our system to determine and account for its own latency. We detail predictive appr...
Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, 2018
We present Haptic Revolver, a handheld virtual reality controller that renders fingertip haptics ... more We present Haptic Revolver, a handheld virtual reality controller that renders fingertip haptics when interacting with virtual surfaces. Haptic Revolver's core haptic element is an actuated wheel that raises and lowers underneath the finger to render contact with a virtual surface. As the user's finger moves along the surface of an object, the controller spins the wheel to render shear forces and motion under the fingertip. The wheel is interchangeable and can contain physical textures, shapes, edges, or active elements to provide different sensations to the user. Because the controller is spatially tracked, these physical features can be spatially registered with the geometry of the virtual environment and rendered on-demand.
Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, 2018
CLAW is a handheld virtual reality controller that augments the typical controller functionality ... more CLAW is a handheld virtual reality controller that augments the typical controller functionality with force feedback and actuated movement to the index finger. Our controller enables three distinct interactions (grasping virtual object, touching virtual surfaces, and triggering) and changes its corresponding haptic rendering by sensing the differences in the user's grasp. A servo motor coupled with a force sensor renders controllable forces to the index finger during grasping and touching. Using position tracking, a voice coil actuator at the index fingertip generates vibrations for various textures synchronized with finger movement. CLAW also supports a haptic force feedback in the trigger mode when the user holds a gun.
2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2018
Head-Mounted Displays (HMDs) provide immersive experiences for virtual reality. However, their fi... more Head-Mounted Displays (HMDs) provide immersive experiences for virtual reality. However, their field of view (FOV) is still relatively small compared to the human eye, which adding sparse peripheral displays (SPDs) could address. We designed a new SPD, SparseLightVR2, which increases the HMD's FOV to 180° horizontally. We evaluated SparseLightVR2 with a study (N=29) by comparing three conditions: 1) no SPD, where the peripheral display (PD) was inactive; 2) extended SPD, where the PD provided visual cues consistent with and extending the HMD's main screen; and 3) counter-vection SPD, where the PD's visuals were flipped horizontally during VR travel to provide optic flow in the opposite direction of the travel. The participants experienced passive motion on a linear path and reported introspective measures such as sensation of self-motion. Results showed, compared to no SPD, both extended and counter-vection SPDs provided a more natural experience of motion, while extended SPD also enhanced vection intensity and believability of movement. Yet, visually induced motion sickness (VIMS) was not affected by display condition. To investigate the reason behind these non-significant results, we conducted a follow-up study and had users increase peripheral counter-vection visuals on the central HMD screen until they nulled out vection. Our results suggest extending HMDs through SPDs enhanced vection, naturalness, and believability of movement without enhancing VIMS, but reversed SPD motion cues might not be strong enough to reduce vection and VIMS.
Numerous studies have demonstrated that visuospatial attention is a requirement for successful wo... more Numerous studies have demonstrated that visuospatial attention is a requirement for successful working memory encoding. It is unknown, however, whether this established relationship manifests in consistent gaze dynamics as people orient their visuospatial attention toward an encoding target when searching for information in naturalistic environments. To test this hypothesis, participants' eye movements were recorded while they searched for and encoded objects in a virtual apartment (Experiment 1). We decomposed gaze into 61 features that capture gaze dynamics and a trained sliding window logistic regression model that has potential for use in real-time systems to predict when participants found target objects for working memory encoding. A model trained on group data successfully predicted when people oriented to a target for encoding for the trained task (Experiment 1) and for a novel task (Experiment 2), where a new set of participants found objects and encoded an associated nonword in a cluttered virtual kitchen. Six of these features were predictive of target orienting for encoding, even during the novel task, including decreased distances between subsequent fixation/saccade events, increased fixation probabilities, and slower saccade decelerations before encoding. This suggests that as people orient toward a target to encode new information at the end of search, they decrease task-irrelevant, exploratory sampling behaviors. This behavior was common across the two studies. Together, this research demonstrates how gaze dynamics can be used to capture target orienting for working memory encoding and has implications for real-world use in technology and special populations.
When enabling touch interactions on a deformable surface, the user is faced with a number of nove... more When enabling touch interactions on a deformable surface, the user is faced with a number of novel interaction challenges not present on firm surfaces. For example, the material used for the deformable surface can completely change the set of interactions that can be used. In addition, touching with one finger affects the overall shape of surface, and deformation for each touch individually can be hard. In this paper we discuss two prototype systems designed to enable us to study such challenges. We discuss how such challenges also present design opportunities to enhance interaction vocabulary of touch interactions on deformable surfaces and offer guidance to future designers of such interactions.
L'invention porte sur des techniques de motif de tenue de dispositif tactile habillable. Un h... more L'invention porte sur des techniques de motif de tenue de dispositif tactile habillable. Un habillage sensible au toucher peut etre configure pour couvrir sensiblement les surfaces exterieures d'un dispositif informatique. L'habillage sensible au toucher peut comprendre une pluralite de capteurs d'habillage configures pour detecter une interaction avec l'habillage a des emplacements definis. Le dispositif informatique peut comprendre un ou plusieurs modules utilisables pour obtenir une entree de la pluralite de capteurs d'habillage et pour decoder l'entree afin de determiner des motifs de tenue qui indiquent comment le dispositif informatique est tenu par un utilisateur. Diverses fonctions fournies par le dispositif informatique peuvent etre selectivement activees et/ou adaptees en fonction d'un motif de tenue determine de maniere a ce que la fonction fournie puisse changer pour correspondre au motif de tenue.
L'invention, selon des modes de realisation, porte sur des dispositifs de saisie avec des sur... more L'invention, selon des modes de realisation, porte sur des dispositifs de saisie avec des surfaces tactiles multiples courbees. Un mode de realisation decrit comprend un dispositif de saisie tactile ayant un element geometrique courbe comprenant un capteur tactile, le capteur tactile comprenant un groupement d'elements de capteur integres dans l'element geometrique courbe et etant configure de facon a detecter un emplacement d'un contact realise sur une surface de l'element geometrique courbe.
La presente invention concerne un module de gestion d’interaction (IMM) qui permet a des utilisat... more La presente invention concerne un module de gestion d’interaction (IMM) qui permet a des utilisateurs de s'engager sur une surface interactive dans un environnement collaboratif en utilisant differents dispositifs d'entree, tels que des dispositifs de clavier et des dispositifs de souris. L’IMM affiche des objets numeriques sur la surface interactive qui sont associes aux dispositifs de differentes facons. Les objets numeriques peuvent comprendre des interfaces d'affichage d'entree, des curseurs, des mecanismes d'entree de touche programmable, et ainsi de suite. En outre, l’IMM fournit un mecanisme servant a etablir une trame de reference pour gouverner le placement de chaque curseur sur la surface interactive. De plus, l’IMM fournit un mecanisme permettant a des utilisateurs de faire une copie numerique d'un article physique place sur la surface interactive. L’IMM fournit egalement un mecanisme qui duplique les actions prises sur la copie numerique par rappo...
Projector-camera (pro-cam) systems afford a wide range of interactive possibilities, combining bo... more Projector-camera (pro-cam) systems afford a wide range of interactive possibilities, combining both natural and mixed-reality 3D interaction. However, the latency inherent within these systems can cause the projection to ‘slip’ from its intended target, detracting from the overall experience. Because of this, pro-cam systems have typically shied away from truly dynamic scenarios. In turn, research has been exploring latency reduction techniques across a range of domains, but these techniques typically focus on custom hardware, limiting their widespread adoption. We explore software-only predictive approaches to minimize the effects of latency in pro-cam systems. In this paper, we focus our predictive approaches on realworld objects under fast motion and on-body projection, improving projection accuracy on fast moving targets. Alongside this we explore automatic latency measurement techniques, allowing our system to determine and account for its own latency. We detail predictive appr...
Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, 2018
We present Haptic Revolver, a handheld virtual reality controller that renders fingertip haptics ... more We present Haptic Revolver, a handheld virtual reality controller that renders fingertip haptics when interacting with virtual surfaces. Haptic Revolver's core haptic element is an actuated wheel that raises and lowers underneath the finger to render contact with a virtual surface. As the user's finger moves along the surface of an object, the controller spins the wheel to render shear forces and motion under the fingertip. The wheel is interchangeable and can contain physical textures, shapes, edges, or active elements to provide different sensations to the user. Because the controller is spatially tracked, these physical features can be spatially registered with the geometry of the virtual environment and rendered on-demand.
Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, 2018
CLAW is a handheld virtual reality controller that augments the typical controller functionality ... more CLAW is a handheld virtual reality controller that augments the typical controller functionality with force feedback and actuated movement to the index finger. Our controller enables three distinct interactions (grasping virtual object, touching virtual surfaces, and triggering) and changes its corresponding haptic rendering by sensing the differences in the user's grasp. A servo motor coupled with a force sensor renders controllable forces to the index finger during grasping and touching. Using position tracking, a voice coil actuator at the index fingertip generates vibrations for various textures synchronized with finger movement. CLAW also supports a haptic force feedback in the trigger mode when the user holds a gun.
2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2018
Head-Mounted Displays (HMDs) provide immersive experiences for virtual reality. However, their fi... more Head-Mounted Displays (HMDs) provide immersive experiences for virtual reality. However, their field of view (FOV) is still relatively small compared to the human eye, which adding sparse peripheral displays (SPDs) could address. We designed a new SPD, SparseLightVR2, which increases the HMD's FOV to 180° horizontally. We evaluated SparseLightVR2 with a study (N=29) by comparing three conditions: 1) no SPD, where the peripheral display (PD) was inactive; 2) extended SPD, where the PD provided visual cues consistent with and extending the HMD's main screen; and 3) counter-vection SPD, where the PD's visuals were flipped horizontally during VR travel to provide optic flow in the opposite direction of the travel. The participants experienced passive motion on a linear path and reported introspective measures such as sensation of self-motion. Results showed, compared to no SPD, both extended and counter-vection SPDs provided a more natural experience of motion, while extended SPD also enhanced vection intensity and believability of movement. Yet, visually induced motion sickness (VIMS) was not affected by display condition. To investigate the reason behind these non-significant results, we conducted a follow-up study and had users increase peripheral counter-vection visuals on the central HMD screen until they nulled out vection. Our results suggest extending HMDs through SPDs enhanced vection, naturalness, and believability of movement without enhancing VIMS, but reversed SPD motion cues might not be strong enough to reduce vection and VIMS.
Uploads