Showing 1–42 of 42 results for author: Ciocarlie, M

ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke

Authors: Jingxi Xu, Runsheng Wang, Siqi Shang, Ava Chen, Lauren Winterbottom, To-Liang Hsu, Wenxi Chen, Khondoker Ahmed, Pedro Leandro La Rotta, Xinyue Zhu, Dawn M. Nilsen, Joel Stein, Matei Ciocarlie

Abstract: Intent inferral on a hand orthosis for stroke patients is challenging due to the difficulty of data collection from impaired subjects. Additionally, EMG signals exhibit significant variations across different conditions, sessions, and subjects, making it hard for classifiers to generalize. Traditional approaches require a large labeled dataset from the new condition, session, or subject to train i… ▽ More Intent inferral on a hand orthosis for stroke patients is challenging due to the difficulty of data collection from impaired subjects. Additionally, EMG signals exhibit significant variations across different conditions, sessions, and subjects, making it hard for classifiers to generalize. Traditional approaches require a large labeled dataset from the new condition, session, or subject to train intent classifiers; however, this data collection process is burdensome and time-consuming. In this paper, we propose ChatEMG, an autoregressive generative model that can generate synthetic EMG signals conditioned on prompts (i.e., a given sequence of EMG signals). ChatEMG enables us to collect only a small dataset from the new condition, session, or subject and expand it with synthetic samples conditioned on prompts from this new context. ChatEMG leverages a vast repository of previous data via generative training while still remaining context-specific via prompting. Our experiments show that these synthetic samples are classifier-agnostic and can improve intent inferral accuracy for different types of classifiers. We demonstrate that our complete approach can be integrated into a single patient session, including the use of the classifier for functional orthosis-assisted tasks. To the best of our knowledge, this is the first time an intent classifier trained partially on synthetic data has been deployed for functional control of an orthosis by a stroke survivor. Videos and additional information can be found at https://jxu.ai/chatemg. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: 8 pages

Task-Based Design and Policy Co-Optimization for Tendon-driven Underactuated Kinematic Chains

Authors: Sharfin Islam, Zhanpeng He, Matei Ciocarlie

Abstract: Underactuated manipulators reduce the number of bulky motors, thereby enabling compact and mechanically robust designs. However, fewer actuators than joints means that the manipulator can only access a specific manifold within the joint space, which is particular to a given hardware configuration and can be low-dimensional and/or discontinuous. Determining an appropriate set of hardware parameters… ▽ More Underactuated manipulators reduce the number of bulky motors, thereby enabling compact and mechanically robust designs. However, fewer actuators than joints means that the manipulator can only access a specific manifold within the joint space, which is particular to a given hardware configuration and can be low-dimensional and/or discontinuous. Determining an appropriate set of hardware parameters for this class of mechanisms, therefore, is difficult - even for traditional task-based co-optimization methods. In this paper, our goal is to implement a task-based design and policy co-optimization method for underactuated, tendon-driven manipulators. We first formulate a general model for an underactuated, tendon-driven transmission. We then use this model to co-optimize a three-link, two-actuator kinematic chain using reinforcement learning. We demonstrate that our optimized tendon transmission and control policy can be transferred reliably to physical hardware with real-world reaching experiments. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Meta-Learning for Fast Adaptation in Intent Inferral on a Robotic Hand Orthosis for Stroke

Authors: Pedro Leandro La Rotta, Jingxi Xu, Ava Chen, Lauren Winterbottom, Wenxi Chen, Dawn Nilsen, Joel Stein, Matei Ciocarlie

Abstract: We propose MetaEMG, a meta-learning approach for fast adaptation in intent inferral on a robotic hand orthosis for stroke. One key challenge in machine learning for assistive and rehabilitative robotics with disabled-bodied subjects is the difficulty of collecting labeled training data. Muscle tone and spasticity often vary significantly among stroke subjects, and hand function can even change acr… ▽ More We propose MetaEMG, a meta-learning approach for fast adaptation in intent inferral on a robotic hand orthosis for stroke. One key challenge in machine learning for assistive and rehabilitative robotics with disabled-bodied subjects is the difficulty of collecting labeled training data. Muscle tone and spasticity often vary significantly among stroke subjects, and hand function can even change across different use sessions of the device for the same subject. We investigate the use of meta-learning to mitigate the burden of data collection needed to adapt high-capacity neural networks to a new session or subject. Our experiments on real clinical data collected from five stroke subjects show that MetaEMG can improve the intent inferral accuracy with a small session- or subject-specific dataset and very few fine-tuning epochs. To the best of our knowledge, we are the first to formulate intent inferral on stroke subjects as a meta-learning problem and demonstrate fast adaptation to a new session or subject for controlling a robotic hand orthosis with EMG signals. △ Less

Submitted 19 March, 2024; originally announced March 2024.

Grasp Force Assistance via Throttle-based Wrist Angle Control on a Robotic Hand Orthosis for C6-C7 Spinal Cord Injury

Authors: Joaquin Palacios, Alexandra Deli-Ivanov, Ava Chen, Lauren Winterbottom, Dawn M. Nilsen, Joel Stein, Matei Ciocarlie

Abstract: Individuals with hand paralysis resulting from C6-C7 spinal cord injuries frequently rely on tenodesis for grasping. However, tenodesis generates limited grasping force and demands constant exertion to maintain a grasp, leading to fatigue and sometimes pain. We introduce the MyHand-SCI, a wearable robot that provides grasping assistance through motorized exotendons. Our user-driven device enables… ▽ More Individuals with hand paralysis resulting from C6-C7 spinal cord injuries frequently rely on tenodesis for grasping. However, tenodesis generates limited grasping force and demands constant exertion to maintain a grasp, leading to fatigue and sometimes pain. We introduce the MyHand-SCI, a wearable robot that provides grasping assistance through motorized exotendons. Our user-driven device enables independent, ipsilateral operation via a novel Throttle-based Wrist Angle control method, which allows users to maintain grasps without continued wrist extension. A pilot case study with a person with C6 spinal cord injury shows an improvement in functional grasping and grasping force, as well as a preserved ability to modulate grasping force while using our device, thus improving their ability to manipulate everyday objects. This research is a step towards developing effective and intuitive wearable assistive devices for individuals with spinal cord injury. △ Less

Submitted 12 February, 2024; originally announced February 2024.

Comments: 6 pages, 8 figures, 1 table

Volitional Control of the Paretic Hand Post-Stroke Increases Finger Stiffness and Resistance to Robot-Assisted Movement

Authors: Ava Chen, Katelyn Lee, Lauren Winterbottom, Jingxi Xu, Connor Lee, Grace Munger, Alexandra Deli-Ivanov, Dawn M. Nilsen, Joel Stein, Matei Ciocarlie

Abstract: Increased effort during use of the paretic arm and hand can provoke involuntary abnormal synergy patterns and amplify stiffness effects of muscle tone for individuals after stroke, which can add difficulty for user-controlled devices to assist hand movement during functional tasks. We study how volitional effort, exerted in an attempt to open or close the hand, affects resistance to robot-assisted… ▽ More Increased effort during use of the paretic arm and hand can provoke involuntary abnormal synergy patterns and amplify stiffness effects of muscle tone for individuals after stroke, which can add difficulty for user-controlled devices to assist hand movement during functional tasks. We study how volitional effort, exerted in an attempt to open or close the hand, affects resistance to robot-assisted movement at the finger level. We perform experiments with three chronic stroke survivors to measure changes in stiffness when the user is actively exerting effort to activate ipsilateral EMG-controlled robot-assisted hand movements, compared with when the fingers are passively stretched, as well as overall effects from sustained active engagement and use. Our results suggest that active engagement of the upper extremity increases muscle tone in the finger to a much greater degree than through passive-stretch or sustained exertion over time. Potential design implications of this work suggest that developers should anticipate higher levels of finger stiffness when relying on user-driven ipsilateral control methods for assistive or rehabilitative devices for stroke. △ Less

Submitted 21 June, 2024; v1 submitted 12 February, 2024; originally announced February 2024.

Comments: 6 pages, 6 figures, 2 tables. Accepted to BioRob 2024

Tactile-based Object Retrieval From Granular Media

Authors: Jingxi Xu, Yinsen Jia, Dongxiao Yang, Patrick Meng, Xinyue Zhu, Zihan Guo, Shuran Song, Matei Ciocarlie

Abstract: We introduce GEOTACT, a robotic manipulation method capable of retrieving objects buried in granular media. This is a challenging task due to the need to interact with granular media, and doing so based exclusively on tactile feedback, since a buried object can be completely hidden from vision. Tactile feedback is in itself challenging in this context, due to ubiquitous contact with the surroundin… ▽ More We introduce GEOTACT, a robotic manipulation method capable of retrieving objects buried in granular media. This is a challenging task due to the need to interact with granular media, and doing so based exclusively on tactile feedback, since a buried object can be completely hidden from vision. Tactile feedback is in itself challenging in this context, due to ubiquitous contact with the surrounding media, and the inherent noise level induced by the tactile readings. To address these challenges, we use a learning method trained end-to-end with simulated sensor noise. We show that our problem formulation leads to the natural emergence of learned pushing behaviors that the manipulator uses to reduce uncertainty and funnel the object to a stable grasp despite spurious and noisy tactile readings. We also introduce a training curriculum that enables learning these behaviors in simulation, followed by zero-shot transfer to real hardware. To the best of our knowledge, GEOTACT is the first method to reliably retrieve a number of different objects from a granular environment, doing so on real hardware and with integrated tactile sensing. Videos and additional information can be found at https://jxu.ai/geotact. △ Less

Submitted 21 February, 2024; v1 submitted 6 February, 2024; originally announced February 2024.

R$\times$R: Rapid eXploration for Reinforcement Learning via Sampling-based Reset Distributions and Imitation Pre-training

Authors: Gagan Khandate, Tristan L. Saidi, Siqi Shang, Eric T. Chang, Yang Liu, Seth Dennis, Johnson Adams, Matei Ciocarlie

Abstract: We present a method for enabling Reinforcement Learning of motor control policies for complex skills such as dexterous manipulation. We posit that a key difficulty for training such policies is the difficulty of exploring the problem state space, as the accessible and useful regions of this space form a complex structure along manifolds of the original high-dimensional state space. This work prese… ▽ More We present a method for enabling Reinforcement Learning of motor control policies for complex skills such as dexterous manipulation. We posit that a key difficulty for training such policies is the difficulty of exploring the problem state space, as the accessible and useful regions of this space form a complex structure along manifolds of the original high-dimensional state space. This work presents a method to enable and support exploration with Sampling-based Planning. We use a generally applicable non-holonomic Rapidly-exploring Random Trees algorithm and present multiple methods to use the resulting structure to bootstrap model-free Reinforcement Learning. Our method is effective at learning various challenging dexterous motor control skills of higher difficulty than previously shown. In particular, we achieve dexterous in-hand manipulation of complex objects while simultaneously securing the object without the use of passive support surfaces. These policies also transfer effectively to real robots. A number of example videos can also be found on the project website: https://sbrl.cs.columbia.edu △ Less

Submitted 27 January, 2024; originally announced January 2024.

Comments: 20 pages, 14 figures, submitted to Autonomous Robots, RSS 2023 Special Issue. arXiv admin note: substantial text overlap with arXiv:2303.03486

Towards Tenodesis-Modulated Control of an Assistive Hand Exoskeleton for SCI

Authors: Joaquin Palacios, Alexandra Deli-Ivanov, Ava Chen, Lauren Winterbottom, Dawn M. Nilsen, Joel Stein, Matei Ciocarlie

Abstract: Restoration of hand function is one of the highest priorities for SCI populations. In this work, we present a prototype of a robotic assistive orthosis capable of implementing tenodesis user control. The underactuated device provides active grasping assistance while preserving free wrist mobility through the use of Bowden cables. This device enables force modulation during grasping, which was effe… ▽ More Restoration of hand function is one of the highest priorities for SCI populations. In this work, we present a prototype of a robotic assistive orthosis capable of implementing tenodesis user control. The underactuated device provides active grasping assistance while preserving free wrist mobility through the use of Bowden cables. This device enables force modulation during grasping, which was effectively leveraged by a participant with C6 SCI to demonstrate improved grasping abilities using the orthosis, scoring 11 on the Grasp and Release Test using the device compared to 1 without it. △ Less

Submitted 28 November, 2023; originally announced November 2023.

Comments: 2 pages, 4 figures. Presented at the IROS 2023 Assistive Robotics for Citizens Workshop

An Investigation of Multi-feature Extraction and Super-resolution with Fast Microphone Arrays

Authors: Eric T. Chang, Runsheng Wang, Peter Ballentine, Jingxi Xu, Trey Smith, Brian Coltin, Ioannis Kymissis, Matei Ciocarlie

Abstract: In this work, we use MEMS microphones as vibration sensors to simultaneously classify texture and estimate contact position and velocity. Vibration sensors are an important facet of both human and robotic tactile sensing, providing fast detection of contact and onset of slip. Microphones are an attractive option for implementing vibration sensing as they offer a fast response and can be sampled qu… ▽ More In this work, we use MEMS microphones as vibration sensors to simultaneously classify texture and estimate contact position and velocity. Vibration sensors are an important facet of both human and robotic tactile sensing, providing fast detection of contact and onset of slip. Microphones are an attractive option for implementing vibration sensing as they offer a fast response and can be sampled quickly, are affordable, and occupy a very small footprint. Our prototype sensor uses only a sparse array (8-9 mm spacing) of distributed MEMS microphones (<$1, 3.76 x 2.95 x 1.10 mm) embedded under an elastomer. We use transformer-based architectures for data analysis, taking advantage of the microphones' high sampling rate to run our models on time-series data as opposed to individual snapshots. This approach allows us to obtain 77.3% average accuracy on 4-class texture classification (84.2% when excluding the slowest drag velocity), 1.8 mm mean error on contact localization, and 5.6 mm/s mean error on contact velocity. We show that the learned texture and localization models are robust to varying velocity and generalize to unseen velocities. We also report that our sensor provides fast contact detection, an important advantage of fast transducers. This investigation illustrates the capabilities one can achieve with a MEMS microphone array alone, leaving valuable sensor real estate available for integration with complementary tactile sensing modalities. △ Less

Submitted 7 March, 2024; v1 submitted 29 September, 2023; originally announced October 2023.

Comments: 6 pages, 4 figures, accepted to 2024 IEEE International Conference on Robotics and Automation (ICRA)

MORPH: Design Co-optimization with Reinforcement Learning via a Differentiable Hardware Model Proxy

Authors: Zhanpeng He, Matei Ciocarlie

Abstract: We introduce MORPH, a method for co-optimization of hardware design parameters and control policies in simulation using reinforcement learning. Like most co-optimization methods, MORPH relies on a model of the hardware being optimized, usually simulated based on the laws of physics. However, such a model is often difficult to integrate into an effective optimization routine. To address this, we in… ▽ More We introduce MORPH, a method for co-optimization of hardware design parameters and control policies in simulation using reinforcement learning. Like most co-optimization methods, MORPH relies on a model of the hardware being optimized, usually simulated based on the laws of physics. However, such a model is often difficult to integrate into an effective optimization routine. To address this, we introduce a proxy hardware model, which is always differentiable and enables efficient co-optimization alongside a long-horizon control policy using RL. MORPH is designed to ensure that the optimized hardware proxy remains as close as possible to its realistic counterpart, while still enabling task completion. We demonstrate our approach on simulated 2D reaching and 3D multi-fingered manipulation tasks. △ Less

Submitted 29 September, 2023; originally announced September 2023.

Decision Making for Human-in-the-loop Robotic Agents via Uncertainty-Aware Reinforcement Learning

Authors: Siddharth Singi, Zhanpeng He, Alvin Pan, Sandip Patel, Gunnar A. Sigurdsson, Robinson Piramuthu, Shuran Song, Matei Ciocarlie

Abstract: In a Human-in-the-Loop paradigm, a robotic agent is able to act mostly autonomously in solving a task, but can request help from an external expert when needed. However, knowing when to request such assistance is critical: too few requests can lead to the robot making mistakes, but too many requests can overload the expert. In this paper, we present a Reinforcement Learning based approach to this… ▽ More In a Human-in-the-Loop paradigm, a robotic agent is able to act mostly autonomously in solving a task, but can request help from an external expert when needed. However, knowing when to request such assistance is critical: too few requests can lead to the robot making mistakes, but too many requests can overload the expert. In this paper, we present a Reinforcement Learning based approach to this problem, where a semi-autonomous agent asks for external assistance when it has low confidence in the eventual success of the task. The confidence level is computed by estimating the variance of the return from the current state. We show that this estimate can be iteratively improved during training using a Bellman-like recursion. On discrete navigation problems with both fully- and partially-observable state information, we show that our method makes effective use of a limited budget of expert calls at run-time, despite having no access to the expert at training time. △ Less

Submitted 14 March, 2023; v1 submitted 12 March, 2023; originally announced March 2023.

Dexterous In-hand Manipulation by Guiding Exploration with Simple Sub-skill Controllers

Authors: Gagan Khandate, Cameron Mehlman, Xingsheng Wei, Matei Ciocarlie

Abstract: Recently, reinforcement learning has led to dexterous manipulation skills of increasing complexity. Nonetheless, learning these skills in simulation still exhibits poor sample-efficiency which stems from the fact these skills are learned from scratch without the benefit of any domain expertise. In this work, we aim to improve the sample efficiency of learning dexterous in-hand manipulation skills… ▽ More Recently, reinforcement learning has led to dexterous manipulation skills of increasing complexity. Nonetheless, learning these skills in simulation still exhibits poor sample-efficiency which stems from the fact these skills are learned from scratch without the benefit of any domain expertise. In this work, we aim to improve the sample efficiency of learning dexterous in-hand manipulation skills using controllers available via domain knowledge. To this end, we design simple sub-skill controllers and demonstrate improved sample efficiency using a framework that guides exploration toward relevant state space by following actions from these controllers. We are the first to demonstrate learning hard-to-explore finger-gaiting in-hand manipulation skills without the use of an exploratory reset distribution. Video results can be found at https://roamlab.github.io/vge △ Less

Submitted 26 September, 2023; v1 submitted 6 March, 2023; originally announced March 2023.

Comments: 6 pages, 7 figures, submitted to International Conference on Robotics and Automation 2024

Sampling-based Exploration for Reinforcement Learning of Dexterous Manipulation

Authors: Gagan Khandate, Siqi Shang, Eric T. Chang, Tristan Luca Saidi, Yang Liu, Seth Matthew Dennis, Johnson Adams, Matei Ciocarlie

Abstract: In this paper, we present a novel method for achieving dexterous manipulation of complex objects, while simultaneously securing the object without the use of passive support surfaces. We posit that a key difficulty for training such policies in a Reinforcement Learning framework is the difficulty of exploring the problem state space, as the accessible regions of this space form a complex structure… ▽ More In this paper, we present a novel method for achieving dexterous manipulation of complex objects, while simultaneously securing the object without the use of passive support surfaces. We posit that a key difficulty for training such policies in a Reinforcement Learning framework is the difficulty of exploring the problem state space, as the accessible regions of this space form a complex structure along manifolds of a high-dimensional space. To address this challenge, we use two versions of the non-holonomic Rapidly-Exploring Random Trees algorithm; one version is more general, but requires explicit use of the environment's transition function, while the second version uses manipulation-specific kinematic constraints to attain better sample efficiency. In both cases, we use states found via sampling-based exploration to generate reset distributions that enable training control policies under full dynamic constraints via model-free Reinforcement Learning. We show that these policies are effective at manipulation problems of higher difficulty than previously shown, and also transfer effectively to real robots. Videos of the real-hand demonstrations can be found on the project website: https://sbrl.cs.columbia.edu/ △ Less

Submitted 23 May, 2023; v1 submitted 6 March, 2023; originally announced March 2023.

Comments: 10 pages, 7 figures, accepted at Robotics Science & Systems 2023

TANDEM3D: Active Tactile Exploration for 3D Object Recognition

Authors: Jingxi Xu, Han Lin, Shuran Song, Matei Ciocarlie

Abstract: Tactile recognition of 3D objects remains a challenging task. Compared to 2D shapes, the complex geometry of 3D surfaces requires richer tactile signals, more dexterous actions, and more advanced encoding techniques. In this work, we propose TANDEM3D, a method that applies a co-training framework for exploration and decision making to 3D object recognition with tactile signals. Starting with our p… ▽ More Tactile recognition of 3D objects remains a challenging task. Compared to 2D shapes, the complex geometry of 3D surfaces requires richer tactile signals, more dexterous actions, and more advanced encoding techniques. In this work, we propose TANDEM3D, a method that applies a co-training framework for exploration and decision making to 3D object recognition with tactile signals. Starting with our previous work, which introduced a co-training paradigm for 2D recognition problems, we introduce a number of advances that enable us to scale up to 3D. TANDEM3D is based on a novel encoder that builds 3D object representation from contact positions and normals using PointNet++. Furthermore, by enabling 6DOF movement, TANDEM3D explores and collects discriminative touch information with high efficiency. Our method is trained entirely in simulation and validated with real-world experiments. Compared to state-of-the-art baselines, TANDEM3D achieves higher accuracy and a lower number of actions in recognizing 3D objects and is also shown to be more robust to different types and amounts of sensor noise. Video is available at https://jxu.ai/tandem3d. △ Less

Submitted 28 February, 2023; v1 submitted 19 September, 2022; originally announced September 2022.

Comments: 7 pages. Accepted to International Conference on Robotics and Automation (ICRA) 2023

TANDEM: Learning Joint Exploration and Decision Making with Tactile Sensors

Authors: Jingxi Xu, Shuran Song, Matei Ciocarlie

Abstract: Inspired by the human ability to perform complex manipulation in the complete absence of vision (like retrieving an object from a pocket), the robotic manipulation field is motivated to develop new methods for tactile-based object interaction. However, tactile sensing presents the challenge of being an active sensing modality: a touch sensor provides sparse, local data, and must be used in conjunc… ▽ More Inspired by the human ability to perform complex manipulation in the complete absence of vision (like retrieving an object from a pocket), the robotic manipulation field is motivated to develop new methods for tactile-based object interaction. However, tactile sensing presents the challenge of being an active sensing modality: a touch sensor provides sparse, local data, and must be used in conjunction with effective exploration strategies in order to collect information. In this work, we focus on the process of guiding tactile exploration, and its interplay with task-related decision making. We propose TANDEM (TActile exploration aNd DEcision Making), an architecture to learn efficient exploration strategies in conjunction with decision making. Our approach is based on separate but co-trained modules for exploration and discrimination. We demonstrate this method on a tactile object recognition task, where a robot equipped with a touch sensor must explore and identify an object from a known set based on binary contact signals alone. TANDEM achieves higher accuracy with fewer actions than alternative methods and is also shown to be more robust to sensor noise. △ Less

Submitted 21 July, 2022; v1 submitted 1 March, 2022; originally announced March 2022.

Comments: Accepted to Robotics and Automation Letters (RA-L) and International Conference on Intelligent Robots and Systems (IROS) 2022

Design of Spiral-Cable Forearm Exoskeleton to Assist Supination for Hemiparetic Stroke Subjects

Authors: Ava Chen, Lauren Winterbottom, Katherine O'Reilly, Sangwoo Park, Dawn Nilsen, Joel Stein, Matei Ciocarlie

Abstract: We present the development of a cable-based passive forearm exoskeleton that is designed to assist supination for hemiparetic stroke survivors. Our device uniquely provides torque sufficient for counteracting spasticity within a below-elbow apparatus. The mechanism consists of a spiral single-tendon routing embedded in a rigid forearm brace and terminated at the hand and upper-forearm. A spool wit… ▽ More We present the development of a cable-based passive forearm exoskeleton that is designed to assist supination for hemiparetic stroke survivors. Our device uniquely provides torque sufficient for counteracting spasticity within a below-elbow apparatus. The mechanism consists of a spiral single-tendon routing embedded in a rigid forearm brace and terminated at the hand and upper-forearm. A spool with an internal releasable-ratchet mechanism allows the user to manually retract the tendon and rotate the hand to counteract involuntary pronation synergies due to stroke. We characterize the mechanism with benchtop testing and five healthy subjects, and perform a preliminary assessment of the exoskeleton with a single chronic stroke subject having minimal supination ability. The mechanism can be integrated into an existing active hand-opening orthosis to enable supination support during grasping tasks, and also allows for a future actuated supination strategy. △ Less

Submitted 16 May, 2022; v1 submitted 5 October, 2021; originally announced October 2021.

Comments: 6 pages; Accepted to International Conference on Rehabilitation Robotics (ICORR) 2022

Thumb Stabilization and Assistance in a Robotic Hand Orthosis for Post-Stroke Hemiparesis

Authors: Ava Chen, Lauren Winterbottom, Sangwoo Park, Jingxi Xu, Dawn Nilsen, Joel Stein, Matei Ciocarlie

Abstract: We propose a dual-cable method of stabilizing the thumb in the context of a hand orthosis designed for individuals with upper extremity hemiparesis after stroke. This cable network adds opposition/reposition capabilities to the thumb, and increases the likelihood of forming a hand pose that can successfully manipulate objects. In addition to a passive-thumb version (where both cables are of fixed… ▽ More We propose a dual-cable method of stabilizing the thumb in the context of a hand orthosis designed for individuals with upper extremity hemiparesis after stroke. This cable network adds opposition/reposition capabilities to the thumb, and increases the likelihood of forming a hand pose that can successfully manipulate objects. In addition to a passive-thumb version (where both cables are of fixed length), our approach also allows for a single-actuator active-thumb version (where the extension cable is actuated while the abductor remains passive), which allows a range of motion intended to facilitate creating and maintaining grasps. We performed experiments with five chronic stroke survivors consisting of unimanual resistive-pull tasks and bimanual twisting tasks with simulated real-world objects; these explored the effects of thumb assistance on grasp stability and functional range of motion. Our results show that both active- and passive-thumb versions achieved similar performance in terms of improving grasp force generation over a no-device baseline, but active thumb stabilization enabled users to maintain grasps for longer durations. △ Less

Submitted 10 May, 2022; v1 submitted 5 October, 2021; originally announced October 2021.

Comments: 7 pages, 6 figures

Discovering Synergies for Robot Manipulation with Multi-Task Reinforcement Learning

Authors: Zhanpeng He, Matei Ciocarlie

Abstract: Controlling robotic manipulators with high-dimensional action spaces for dexterous tasks is a challenging problem. Inspired by human manipulation, researchers have studied generating and using postural synergies for robot hands to accomplish manipulation tasks, leveraging the lower dimensional nature of synergistic action spaces. However, many of these works require pre-collected data from an exis… ▽ More Controlling robotic manipulators with high-dimensional action spaces for dexterous tasks is a challenging problem. Inspired by human manipulation, researchers have studied generating and using postural synergies for robot hands to accomplish manipulation tasks, leveraging the lower dimensional nature of synergistic action spaces. However, many of these works require pre-collected data from an existing controller in order to derive such a subspace by means of dimensionality reduction. In this paper, we present a framework that simultaneously discovers a synergy space and a multi-task policy that operates on this low-dimensional action space to accomplish diverse manipulation tasks. We demonstrate that our end-to-end method is able to perform multiple tasks using few synergies, and outperforms sequential methods that apply dimensionality reduction to independently collected data. We also show that deriving synergies using multiple tasks can lead to a subspace that enables robots to efficiently learn new manipulation tasks and interactions with new objects. △ Less

Submitted 29 June, 2022; v1 submitted 4 October, 2021; originally announced October 2021.

Journal ref: IEEE International Conference on Robotics and Automation 2022

On the Feasibility of Learning Finger-gaiting In-hand Manipulation with Intrinsic Sensing

Authors: Gagan Khandate, Maxmillian Haas-Heger, Matei Ciocarlie

Abstract: Finger-gaiting manipulation is an important skill to achieve large-angle in-hand re-orientation of objects. However, achieving these gaits with arbitrary orientations of the hand is challenging due to the unstable nature of the task. In this work, we use model-free reinforcement learning (RL) to learn finger-gaiting only via precision grasps and demonstrate finger-gaiting for rotation about an axi… ▽ More Finger-gaiting manipulation is an important skill to achieve large-angle in-hand re-orientation of objects. However, achieving these gaits with arbitrary orientations of the hand is challenging due to the unstable nature of the task. In this work, we use model-free reinforcement learning (RL) to learn finger-gaiting only via precision grasps and demonstrate finger-gaiting for rotation about an axis purely using on-board proprioceptive and tactile feedback. To tackle the inherent instability of precision grasping, we propose the use of initial state distributions that enable effective exploration of the state space. Our method can learn finger-gaiting with significantly improved sample complexity than the state-of-the-art. The policies we obtain are robust and also transfer to novel objects. Videos can be found at https://roamlab.github.io/learnfg/ △ Less

Submitted 15 March, 2022; v1 submitted 26 September, 2021; originally announced September 2021.

Comments: 7 pages, 9 figures, accepted at ICRA 2022

Design Paradigms Based on Spring Agonists for Underactuated Robot Hands: Concepts and Application

Authors: Tianjian Chen, Tianyi Zhang, Matei Ciocarlie

Abstract: In this paper, we focus on a rarely used paradigm in the design of underactuated robot hands: the use of springs as agonists and tendons as antagonists. We formalize this approach in a design matrix also considering its interplay with the underactuation method used (one tendon for multiple joints vs. multiple tendons on one motor shaft). We then show how different cells in this design matrix can b… ▽ More In this paper, we focus on a rarely used paradigm in the design of underactuated robot hands: the use of springs as agonists and tendons as antagonists. We formalize this approach in a design matrix also considering its interplay with the underactuation method used (one tendon for multiple joints vs. multiple tendons on one motor shaft). We then show how different cells in this design matrix can be combined in order to facilitate the implementation of desired postural synergies with a single motor. Furthermore, we show that when agonist and antagonist tendons are combined on the same motor shaft, the resulting spring force cancellation can be leveraged to produce multiple desirable behaviors, which we demonstrate in a physical prototype. △ Less

Submitted 25 March, 2021; v1 submitted 3 November, 2020; originally announced November 2020.

Comments: Published in the International Conference on Robotics and Automation (ICRA) 2021

Adaptive Semi-Supervised Intent Inferral to Control a Powered Hand Orthosis for Stroke

Authors: Jingxi Xu, Cassie Meeker, Ava Chen, Lauren Winterbottom, Michaela Fraser, Sangwoo Park, Lynne M. Weber, Mitchell Miya, Dawn Nilsen, Joel Stein, Matei Ciocarlie

Abstract: In order to provide therapy in a functional context, controls for wearable robotic orthoses need to be robust and intuitive. We have previously introduced an intuitive, user-driven, EMG-based method to operate a robotic hand orthosis, but the process of training a control that is robust to concept drift (changes in the input signal) places a substantial burden on the user. In this paper, we explor… ▽ More In order to provide therapy in a functional context, controls for wearable robotic orthoses need to be robust and intuitive. We have previously introduced an intuitive, user-driven, EMG-based method to operate a robotic hand orthosis, but the process of training a control that is robust to concept drift (changes in the input signal) places a substantial burden on the user. In this paper, we explore semi-supervised learning as a paradigm for controlling a powered hand orthosis for stroke subjects. To the best of our knowledge, this is the first use of semi-supervised learning for an orthotic application. Specifically, we propose a disagreement-based semi-supervision algorithm for handling intrasession concept drift based on multimodal ipsilateral sensing. We evaluate the performance of our algorithm on data collected from five stroke subjects. Our results show that the proposed algorithm helps the device adapt to intrasession drift using unlabeled data and reduces the training burden placed on the user. We also validate the feasibility of our proposed algorithm with a functional task; in these experiments, two subjects successfully completed multiple instances of a pick-and-handover task. △ Less

Submitted 1 March, 2022; v1 submitted 30 October, 2020; originally announced November 2020.

Comments: 7 pages; Accepted to International Conference on Robotics and Automation (ICRA) 2022

Hardware as Policy: Mechanical and Computational Co-Optimization using Deep Reinforcement Learning

Authors: Tianjian Chen, Zhanpeng He, Matei Ciocarlie

Abstract: Deep Reinforcement Learning (RL) has shown great success in learning complex control policies for a variety of applications in robotics. However, in most such cases, the hardware of the robot has been considered immutable, modeled as part of the environment. In this study, we explore the problem of learning hardware and control parameters together in a unified RL framework. To achieve this, we pro… ▽ More Deep Reinforcement Learning (RL) has shown great success in learning complex control policies for a variety of applications in robotics. However, in most such cases, the hardware of the robot has been considered immutable, modeled as part of the environment. In this study, we explore the problem of learning hardware and control parameters together in a unified RL framework. To achieve this, we propose to model the robot body as a "hardware policy", analogous to and optimized jointly with its computational counterpart. We show that, by modeling such hardware policies as auto-differentiable computational graphs, the ensuing optimization problem can be solved efficiently by gradient-based algorithms from the Policy Optimization family. We present two such design examples: a toy mass-spring problem, and a real-world problem of designing an underactuated hand. We compare our method against traditional co-optimization approaches, and also demonstrate its effectiveness by building a physical prototype based on the learned hardware parameters. Videos and more details are available at https://roamlab.github.io/hwasp/ . △ Less

Submitted 9 November, 2020; v1 submitted 10 August, 2020; originally announced August 2020.

Comments: Conference on Robot Learning (CoRL) 2020

A Sensorized Multicurved Robot Finger with Data-driven Touch Sensing via Overlapping Light Signals

Authors: Pedro Piacenza, Keith Behrman, Benedikt Schifferer, Ioannis Kymissis, Matei Ciocarlie

Abstract: Despite significant advances in touch and force transduction, tactile sensing is still far from ubiquitous in robotic manipulation. Existing methods for building touch sensors have proven difficult to integrate into robot fingers due to multiple challenges, including difficulty in covering multicurved surfaces, high wire count, or packaging constrains preventing their use in dexterous hands. In th… ▽ More Despite significant advances in touch and force transduction, tactile sensing is still far from ubiquitous in robotic manipulation. Existing methods for building touch sensors have proven difficult to integrate into robot fingers due to multiple challenges, including difficulty in covering multicurved surfaces, high wire count, or packaging constrains preventing their use in dexterous hands. In this paper, we present a multicurved robotic finger with accurate touch localization and normal force detection over complex, three-dimensional surfaces. The key to our approach is the novel use of overlapping signals from light emitters and receivers embedded in a transparent waveguide layer that covers the functional areas of the finger. By measuring light transport between every emitter and receiver, we show that we can obtain a very rich signal set that changes in response to deformation of the finger due to touch. We then show that purely data-driven deep learning methods are able to extract useful information from such data, such as contact location and applied normal force, without the need for analytical models. The final result is a fully integrated, sensorized robot finger, with a low wire count and using easily accessible manufacturing methods, designed for easy integration into dexterous manipulators. △ Less

Submitted 1 April, 2020; originally announced April 2020.

Comments: To be published in IEEE/ASME Transactions on Mechatronics. https://ieeexplore.ieee.org/document/9006916

A Continuous Teleoperation Subspace with Empirical and Algorithmic Mapping Algorithms for Non-Anthropomorphic Hands

Authors: Cassie Meeker, Maximilian Haas-Heger, Matei Ciocarlie

Abstract: Teleoperation is a valuable tool for robotic manipulators in highly unstructured environments. However, finding an intuitive mapping between a human hand and a non-anthropomorphic robot hand can be difficult, due to the hands' dissimilar kinematics. In this paper, we seek to create a mapping between the human hand and a fully actuated, non-anthropomorphic robot hand that is intuitive enough to ena… ▽ More Teleoperation is a valuable tool for robotic manipulators in highly unstructured environments. However, finding an intuitive mapping between a human hand and a non-anthropomorphic robot hand can be difficult, due to the hands' dissimilar kinematics. In this paper, we seek to create a mapping between the human hand and a fully actuated, non-anthropomorphic robot hand that is intuitive enough to enable effective real-time teleoperation, even for novice users. To accomplish this, we propose a low-dimensional teleoperation subspace which can be used as an intermediary for mapping between hand pose spaces. We present two different methods to define the teleoperation subspace: an empirical definition, which requires a person to define hand motions in an intuitive, hand-specific way, and an algorithmic definition, which is kinematically independent, and uses objects to define the subspace. We use each of these definitions to create a teleoperation mapping for different hands. One of the main contributions of this paper is the validation of both the empirical and algorithmic mappings with teleoperation experiments controlled by ten novices and performed on two kinematically distinct hands. The experiments show that the proposed subspace is relevant to teleoperation, intuitive enough to enable control by novices, and can generalize to non-anthropomorphic hands with different kinematics. △ Less

Submitted 28 October, 2020; v1 submitted 21 November, 2019; originally announced November 2019.

Comments: 14 pages, 6 tables, 8 figures, accepted October 2020 IEEE T-ASE

User-Driven Functional Movement Training with a Wearable Hand Robot after Stroke

Authors: Sangwoo Park, Michaela Fraser, Lynne M. Weber, Cassie Meeker, Lauri Bishop, Daniel Geller, Joel Stein, Matei Ciocarlie

Abstract: We studied the performance of a robotic orthosis designed to assist the paretic hand after stroke. It is wearable and fully user-controlled, serving two possible roles: as a therapeutic tool that facilitates device mediated hand exercises to recover neuromuscular function or as an assistive device for use in everyday activities to aid functional use of the hand. We present the clinical outcomes of… ▽ More We studied the performance of a robotic orthosis designed to assist the paretic hand after stroke. It is wearable and fully user-controlled, serving two possible roles: as a therapeutic tool that facilitates device mediated hand exercises to recover neuromuscular function or as an assistive device for use in everyday activities to aid functional use of the hand. We present the clinical outcomes of a pilot study designed as a feasibility test for these hypotheses. 11 chronic stroke (> 2 years) patients with moderate muscle tone (Modified Ashworth Scale less than or equal to 2 in upper extremity) engaged in a month-long training protocol using the orthosis. Individuals were evaluated using standardized outcome measures, both with and without orthosis assistance. Fugl-Meyer post intervention scores without robotic assistance showed improvement focused specifically at the distal joints of the upper limb, suggesting the use of the orthosis as a rehabilitative device for the hand. Action Research Arm Test scores post intervention with robotic assistance showed that the device may serve an assistive role in grasping tasks. These results highlight the potential for wearable and user-driven robotic hand orthoses to extend the use and training of the affected upper limb after stroke. △ Less

Submitted 2 September, 2020; v1 submitted 18 November, 2019; originally announced November 2019.

Comments: 10 pages, 15 figures, Camera ready version

Automatic Snake Gait Generation Using Model Predictive Control

Authors: Emily Hannigan, Bing Song, Gagan Khandate, Maximilian Haas-Heger, Ji Yin, Matei Ciocarlie

Abstract: In this paper, we propose a method for generating undulatory gaits for snake robots. Instead of starting from a pre-defined movement pattern such as a serpenoid curve, we use a Model Predictive Control approach to automatically generate effective locomotion gaits via trajectory optimization. An important advantage of this approach is that the resulting gaits are automatically adapted to the enviro… ▽ More In this paper, we propose a method for generating undulatory gaits for snake robots. Instead of starting from a pre-defined movement pattern such as a serpenoid curve, we use a Model Predictive Control approach to automatically generate effective locomotion gaits via trajectory optimization. An important advantage of this approach is that the resulting gaits are automatically adapted to the environment that is being modeled as part of the snake dynamics. To illustrate this, we use a novel model for anisotropic dry friction, along with existing models for viscous friction and fluid dynamic effects such as drag and added mass. For each of these models, gaits generated without any change in the method or its parameters are as efficient as Pareto-optimal serpenoid gaits tuned individually for each environment. Furthermore, the proposed method can also produce more complex or irregular gaits, e.g. for obstacle avoidance or executing sharp turns. △ Less

Submitted 15 October, 2020; v1 submitted 24 September, 2019; originally announced September 2019.

Underactuation Design for Tendon-driven Hands via Optimization of Mechanically Realizable Manifolds in Posture and Torque Spaces

Authors: Tianjian Chen, Long Wang, Maximilan Haas-Heger, Matei Ciocarlie

Abstract: Grasp synergies represent a useful idea to reduce grasping complexity without compromising versatility. Synergies describe coordination patterns between joints, either in terms of position (joint angles) or effort (joint torques). In both of these cases, a grasp synergy can be represented as a low-dimensional manifold lying in the high-dimensional joint posture or torque space. In this paper, we u… ▽ More Grasp synergies represent a useful idea to reduce grasping complexity without compromising versatility. Synergies describe coordination patterns between joints, either in terms of position (joint angles) or effort (joint torques). In both of these cases, a grasp synergy can be represented as a low-dimensional manifold lying in the high-dimensional joint posture or torque space. In this paper, we use the term \textit{Mechanically Realizable Manifolds} to refer to the subset of such manifolds (in either posture or torque space) that can be achieved via mechanical coupling of the joints in underactuated hands. We present a method to optimize the design parameters of an underactuated hand in order to shape the Mechanically Realizable Manifolds to fit a pre-defined set of desired grasps. Our method guarantees that the resulting synergies can be physically implemented in an underactuated hand, and will enable the resulting hand to both reach the desired grasp postures and achieve quasistatic equilibrium while loading the grasps. We demonstrate this method on three concrete design examples motivated by a real use case, and evaluate and compare their performance in practice. △ Less

Submitted 26 June, 2020; v1 submitted 27 May, 2019; originally announced May 2019.

Comments: IEEE Transactions on Robotics (T-RO)

Accurate Energetic Constraints for Passive Grasp Stability Analysis

Authors: Maximilian Haas-Heger, Matei Ciocarlie

Abstract: Passive reaction effects in grasp stability analysis occur when the contact forces and joint torques applied by a grasp change in response to external disturbances applied to the grasped object. For example, nonbackdrivable actuators (e.g. highly geared servos) will passively resist external disturbances without an actively applied command; for numerous robot hands using such motors, these effects… ▽ More Passive reaction effects in grasp stability analysis occur when the contact forces and joint torques applied by a grasp change in response to external disturbances applied to the grasped object. For example, nonbackdrivable actuators (e.g. highly geared servos) will passively resist external disturbances without an actively applied command; for numerous robot hands using such motors, these effects can be highly beneficial as they increase grasp resistance without requiring active control. We introduce a grasp stability analysis method that can model these effects, and, for a given grasp, distinguish between disturbances that will be passively resisted and those that will not. We find that, in order to achieve this, the grasp model must include accurate energetic constraints. One way to achieve this is to consider the Maximum Dissipation Principle (MDP), a part of the Coulomb friction model that is rarely used in grasp stability analysis. However, the MDP constraints are non-convex, and difficult to solve efficiently. We thus introduce a convex relaxation method, along with an algorithm that successively refines this relaxation locally in order to obtain solutions to arbitrary accuracy efficiently. Our resulting algorithm can determine if a grasp is passively stable, solve for equilibrium contact forces and compute optimal actuator commands for stability. Its implementation is publicly available as part of the open-source GraspIt! simulator. △ Less

Submitted 8 July, 2020; v1 submitted 30 April, 2019; originally announced May 2019.

Comments: 18 pages, 13 figures, 2 tables, 1 algorithm

EMG-Controlled Non-Anthropomorphic Hand Teleoperation Using a Continuous Teleoperation Subspace

Authors: Cassie Meeker, Matei Ciocarlie

Abstract: We present a method for EMG-driven teleoperation of non-anthropomorphic robot hands. EMG sensors are appealing as a wearable, inexpensive, and unobtrusive way to gather information about the teleoperator's hand pose. However, mapping from EMG signals to the pose space of a non-anthropomorphic hand presents multiple challenges. We present a method that first projects from forearm EMG into a subspac… ▽ More We present a method for EMG-driven teleoperation of non-anthropomorphic robot hands. EMG sensors are appealing as a wearable, inexpensive, and unobtrusive way to gather information about the teleoperator's hand pose. However, mapping from EMG signals to the pose space of a non-anthropomorphic hand presents multiple challenges. We present a method that first projects from forearm EMG into a subspace relevant to teleoperation. To increase robustness, we use a model which combines continuous and discrete predictors along different dimensions of this subspace. We then project from the teleoperation subspace into the pose space of the robot hand. Our method is effective and intuitive, as it enables novice users to teleoperate pick and place tasks faster and more robustly than state-of-the-art EMG teleoperation methods when applied to a non-anthropomorphic, multi-DOF robot hand. △ Less

Submitted 6 March, 2019; v1 submitted 25 September, 2018; originally announced September 2018.

Comments: 7 pages, 4 figures, ICRA2019 conference submission

Multimodal Sensing and Interaction for a Robotic Hand Orthosis

Authors: Sangwoo Park, Cassie Meeker, Lynne M. Weber, Lauri Bishop, Joel Stein, Matei Ciocarlie

Abstract: Wearable robotic hand rehabilitation devices can allow greater freedom and flexibility than their workstation-like counterparts. However, the field is generally lacking effective methods by which the user can operate the device: such controls must be effective, intuitive, and robust to the wide range of possible impairment patterns. Even when focusing on a specific condition, such as stroke, the v… ▽ More Wearable robotic hand rehabilitation devices can allow greater freedom and flexibility than their workstation-like counterparts. However, the field is generally lacking effective methods by which the user can operate the device: such controls must be effective, intuitive, and robust to the wide range of possible impairment patterns. Even when focusing on a specific condition, such as stroke, the variety of encountered upper limb impairment patterns means that a single sensing modality, such as electromyography (EMG), might not be sufficient to enable controls for a broad range of users. To address this significant gap, we introduce a multimodal sensing and interaction paradigm for an active hand orthosis. In our proof-of-concept implementation, EMG is complemented by other sensing modalities, such as finger bend and contact pressure sensors. We propose multimodal interaction methods that utilize this sensory data as input, and show they can enable tasks for stroke survivors who exhibit different impairment patterns. We believe that robotic hand orthoses developed as multimodal sensory platforms with help address some of the key challenges in physical interaction with the user. △ Less

Submitted 18 December, 2018; v1 submitted 31 July, 2018; originally announced August 2018.

Comments: 8 pages. 9 Figures. IEEE Robotics and Automation Letters. Preprint version. Accepted Dec, 2018

Journal ref: IEEE Robotics and Automation Letters, Vol. 4, No. 2, pp. 315 - 322, April 2019

Passive Static Equilibrium with Frictional Contacts and Application to Grasp Stability Analysis

Authors: Maximilian Haas-Heger, Christos Papadimitriou, Mihalis Yannakakis, Garud Iyengar, Matei Ciocarlie

Abstract: This paper studies the problem of passive grasp stability under an external disturbance, that is, the ability of a grasp to resist a disturbance through passive responses at the contacts. To obtain physically consistent results, such a model must account for friction phenomena at each contact; the difficulty is that friction forces depend in non-linear fashion on contact behavior (stick or slip).… ▽ More This paper studies the problem of passive grasp stability under an external disturbance, that is, the ability of a grasp to resist a disturbance through passive responses at the contacts. To obtain physically consistent results, such a model must account for friction phenomena at each contact; the difficulty is that friction forces depend in non-linear fashion on contact behavior (stick or slip). We develop the first polynomial-time algorithm which either solves such complex equilibrium constraints for two-dimensional grasps, or otherwise concludes that no solution exists. To achieve this, we show that the number of possible `slip states' (where each contact is labeled as either sticking or slipping) that must be considered is polynomial (in fact quadratic) in the number of contacts, and not exponential as previously thought. Our algorithm captures passive response behaviors at each contact, while accounting for constraints on friction forces such as the maximum dissipation principle. △ Less

Submitted 13 June, 2018; v1 submitted 4 June, 2018; originally announced June 2018.

Comments: Robotics: Science and Systems June 26-30, 2018 (9 pages, 7 figures)

Proprioception-Based Grasping for Unknown Objects Using a Series-Elastic-Actuated Gripper

Authors: Tianjian Chen, Matei Ciocarlie

Abstract: Grasping unknown objects has been an active research topic for decades. Approaches range from using various sensors (e.g. vision, tactile) to gain information about the object, to building passively compliant hands that react appropriately to contacts. In this paper, we focus on grasping unknown objects using proprioception (the combination of joint position and torque sensing). Our hypothesis is… ▽ More Grasping unknown objects has been an active research topic for decades. Approaches range from using various sensors (e.g. vision, tactile) to gain information about the object, to building passively compliant hands that react appropriately to contacts. In this paper, we focus on grasping unknown objects using proprioception (the combination of joint position and torque sensing). Our hypothesis is that proprioception alone can be the basis for versatile performance, including multiple types of grasps for objects with multiple shapes and sizes, and transitions between grasps. Using a series-elastic-actuated gripper, we propose a method for performing stable fingertip grasps for unknown objects with unknown contacts, formulated as multi-input-multi-output (MIMO) control. We also show that the proprioceptive gripper can perform enveloping grasps, as well as the transition from fingertip grasps to enveloping grasps. △ Less

Submitted 1 August, 2018; v1 submitted 26 March, 2018; originally announced March 2018.

Comments: 7 pages, 9 figures, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Underactuated Hand Design Using Mechanically Realizable Manifolds

Authors: Tianjian Chen, Maximilian Haas-Heger, Matei Ciocarlie

Abstract: Hand synergies, or joint coordination patterns, have become an effective tool for achieving versatile robotic grasping with simple hands or planning algorithms. Here we propose a method to determine the hand synergies such that they can be physically implemented in an underactuated fashion. Given a kinematic hand model and a set of desired grasps, our algorithm optimizes a Mechanically Realizable… ▽ More Hand synergies, or joint coordination patterns, have become an effective tool for achieving versatile robotic grasping with simple hands or planning algorithms. Here we propose a method to determine the hand synergies such that they can be physically implemented in an underactuated fashion. Given a kinematic hand model and a set of desired grasps, our algorithm optimizes a Mechanically Realizable Manifold designed to be achievable by a physical underactuation mechanism, enabling the resulting hand to achieve the desired grasps with few actuators. Furthermore, in contrast to existing methods for determining synergies which are only concerned with hand posture, our method explicitly optimizes the stability of the target grasps. We implement this method in the design of a three-finger single-actuator hand as an example, and evaluate its effectiveness numerically and experimentally. △ Less

Submitted 1 August, 2018; v1 submitted 26 March, 2018; originally announced March 2018.

Comments: 7 pages, 6 figures, 2018 IEEE International Conference on Robotics and Automation (ICRA)

Data-driven Super-resolution on a Tactile Dome

Authors: Pedro Piacenza, Sydney Sherman, Matei Ciocarlie

Abstract: While tactile sensor technology has made great strides over the past decades, applications in robotic manipulation are limited by aspects such as blind spots, difficult integration into hands, and low spatial resolution. We present a method for localizing contact with high accuracy over curved, three dimensional surfaces, with a low wire count and reduced integration complexity. To achieve this, w… ▽ More While tactile sensor technology has made great strides over the past decades, applications in robotic manipulation are limited by aspects such as blind spots, difficult integration into hands, and low spatial resolution. We present a method for localizing contact with high accuracy over curved, three dimensional surfaces, with a low wire count and reduced integration complexity. To achieve this, we build a volume of soft material embedded with individual off-the-shelf pressure sensors. Using data driven techniques, we map the raw signals from these pressure sensors to known surface locations and indentation depths. Additionally, we show that a finite element model can be used to improve the placement of the pressure sensors inside the volume and to explore the design space in simulation. We validate our approach on physically implemented tactile domes which achieve high contact localization accuracy ($1.1mm$ in the best case) over a large, curved sensing area ($1,300mm^2$ hemisphere). We believe this approach can be used to deploy tactile sensing capabilities over three dimensional surfaces such as a robotic finger or palm. △ Less

Submitted 26 February, 2018; originally announced February 2018.

Comments: 8 pages, 9 figures

Journal ref: IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1434-1441, July 2018

Touch Sensors with Overlapping Signals: Concept Investigation on Planar Sensors with Resistive or Optical Transduction

Authors: Pedro Piacenza, Emily Hannigan, Clayton Baumgart, Yuchen Xiao, Steve Park, Keith Behrman, Weipeng Dang, Jeremy Espinal, Ikram Hussain, Ioannis Kymissis, Matei Ciocarlie

Abstract: Traditional methods for achieving high localization accuracy on tactile sensors usually involve a matrix of miniaturized individual sensors distributed on the area of interest. This approach usually comes at a price of increased complexity in fabrication and circuitry, and can be hard to adapt to non-planar geometries. We propose a method where sensing terminals are embedded in a volume of soft ma… ▽ More Traditional methods for achieving high localization accuracy on tactile sensors usually involve a matrix of miniaturized individual sensors distributed on the area of interest. This approach usually comes at a price of increased complexity in fabrication and circuitry, and can be hard to adapt to non-planar geometries. We propose a method where sensing terminals are embedded in a volume of soft material. Mechanical strain in this material results in a measurable signal between any two given terminals. By having multiple terminals and pairing them against each other in all possible combinations, we obtain a rich signal set using few wires. We mine this data to learn the mapping between the signals we extract and the contact parameters of interest. Our approach is general enough that it can be applied with different transduction methods, and achieves high accuracy in identifying indentation location and depth. Moreover, this method lends itself to simple fabrication techniques and makes no assumption about the underlying geometry, potentially simplifying future integration in robot hands. △ Less

Submitted 12 July, 2019; v1 submitted 22 February, 2018; originally announced February 2018.

Comments: 15 pages, 13 figures, 6 tables

Accurate Contact Localization and Indentation Depth Prediction With an Optics-based Tactile Sensor

Authors: Pedro Piacenza, Weipeng Dang, Emily Hannigan, Jeremy Espinal, Ikram Hussain, Ioannis Kymissis, Matei Ciocarlie

Abstract: Traditional methods to achieve high localization accuracy with tactile sensors usually use a matrix of miniaturized individual sensors distributed on the area of interest. This approach usually comes at a price of increased complexity in fabrication and circuitry, and can be hard to adapt for non planar geometries. We propose to use low cost optic components mounted on the edges of the sensing are… ▽ More Traditional methods to achieve high localization accuracy with tactile sensors usually use a matrix of miniaturized individual sensors distributed on the area of interest. This approach usually comes at a price of increased complexity in fabrication and circuitry, and can be hard to adapt for non planar geometries. We propose to use low cost optic components mounted on the edges of the sensing area to measure how light traveling through an elastomer is affected by touch. Multiple light emitters and receivers provide us with a rich signal set that contains the necessary information to pinpoint both the location and depth of an indentation with high accuracy. We demonstrate sub-millimeter accuracy on location and depth on a 20mm by 20mm active sensing area. Our sensor provides high depth sensitivity as a result of two different modalities in how light is guided through our elastomer. This method results in a low cost, easy to manufacture sensor. We believe this approach can be adapted to cover non-planar surfaces, simplifying future integration in robot skin applications. △ Less

Submitted 19 February, 2018; originally announced February 2018.

Comments: 7 pages, 7 figures, 3 tables, originally published on ICRA 2017

Design and Development of Effective Transmission Mechanisms on a Tendon Driven Hand Orthosis for Stroke Patients

Authors: Sangwoo Park, Lynne M. Weber, Lauri Bishop, Joel Stein, Matei Ciocarlie

Abstract: Tendon-driven hand orthoses have advantages over exoskeletons with respect to wearability and safety because of their low-profile design and ability to fit a range of patients without requiring custom joint alignment. However, no existing study on a wearable tendon-driven hand orthosis for stroke patients presents evidence that such devices can overcome spasticity given repeated use and fatigue, o… ▽ More Tendon-driven hand orthoses have advantages over exoskeletons with respect to wearability and safety because of their low-profile design and ability to fit a range of patients without requiring custom joint alignment. However, no existing study on a wearable tendon-driven hand orthosis for stroke patients presents evidence that such devices can overcome spasticity given repeated use and fatigue, or discusses transmission efficiency. In this study, we propose two designs that provide effective force transmission by increasing moment arms around finger joints. We evaluate the designs with geometric models and experiment using a 3D-printed artificial finger to find force and joint angle characteristics of the suggested structures. We also perform clinical tests with stroke patients to demonstrate the feasibility of the designs. The testing supports the hypothesis that the proposed designs efficiently elicit extension of the digits in patients with spasticity as compared to existing baselines. △ Less

Submitted 31 July, 2018; v1 submitted 16 February, 2018; originally announced February 2018.

Comments: 7 pages, ICRA 2018

Intuitive Hand Teleoperation by Novice Operators Using a Continuous Teleoperation Subspace

Authors: Cassie Meeker, Thomas Rasmussen, Matei Ciocarlie

Abstract: Human-in-the-loop manipulation is useful in when autonomous grasping is not able to deal sufficiently well with corner cases or cannot operate fast enough. Using the teleoperator's hand as an input device can provide an intuitive control method but requires mapping between pose spaces which may not be similar. We propose a low-dimensional and continuous teleoperation subspace which can be used as… ▽ More Human-in-the-loop manipulation is useful in when autonomous grasping is not able to deal sufficiently well with corner cases or cannot operate fast enough. Using the teleoperator's hand as an input device can provide an intuitive control method but requires mapping between pose spaces which may not be similar. We propose a low-dimensional and continuous teleoperation subspace which can be used as an intermediary for mapping between different hand pose spaces. We present an algorithm to project between pose space and teleoperation subspace. We use a non-anthropomorphic robot to experimentally prove that it is possible for teleoperation subspaces to effectively and intuitively enable teleoperation. In experiments, novice users completed pick and place tasks significantly faster using teleoperation subspace mapping than they did using state of the art teleoperation methods. △ Less

Submitted 12 February, 2018; originally announced February 2018.

Comments: ICRA 2018, 7 pages, 7 figures, 2 tables

EMG Pattern Classification to Control a Hand Orthosis for Functional Grasp Assistance after Stroke

Authors: Cassie Meeker, Sangwoo Park, Lauri Bishop, Joel Stein, Matei Ciocarlie

Abstract: Wearable orthoses can function both as assistive devices, which allow the user to live independently, and as rehabilitation devices, which allow the user to regain use of an impaired limb. To be fully wearable, such devices must have intuitive controls, and to improve quality of life, the device should enable the user to perform Activities of Daily Living. In this context, we explore the feasibili… ▽ More Wearable orthoses can function both as assistive devices, which allow the user to live independently, and as rehabilitation devices, which allow the user to regain use of an impaired limb. To be fully wearable, such devices must have intuitive controls, and to improve quality of life, the device should enable the user to perform Activities of Daily Living. In this context, we explore the feasibility of using electromyography (EMG) signals to control a wearable exotendon device to enable pick and place tasks. We use an easy to don, commodity forearm EMG band with 8 sensors to create an EMG pattern classification control for an exotendon device. With this control, we are able to detect a user's intent to open, and can thus enable extension and pick and place tasks. In experiments with stroke survivors, we explore the accuracy of this control in both non-functional and functional tasks. Our results support the feasibility of developing wearable devices with intuitive controls which provide a functional context for rehabilitation. △ Less

Submitted 1 February, 2018; originally announced February 2018.

Comments: ICORR 2017, 8 pages, 5 figures

Contact Localization through Spatially Overlapping Piezoresistive Signals

Authors: Pedro Piacenza, Yuchen Xiao, Steve Park, Ioannis Kymissis, Matei Ciocarlie

Abstract: Achieving high spatial resolution in contact sensing for robotic manipulation often comes at the price of increased complexity in fabrication and integration. One traditional approach is to fabricate a large number of taxels, each delivering an individual, isolated response to a stimulus. In contrast, we propose a method where the sensor simply consists of a continuous volume of piezoresistive ela… ▽ More Achieving high spatial resolution in contact sensing for robotic manipulation often comes at the price of increased complexity in fabrication and integration. One traditional approach is to fabricate a large number of taxels, each delivering an individual, isolated response to a stimulus. In contrast, we propose a method where the sensor simply consists of a continuous volume of piezoresistive elastomer with a number of electrodes embedded inside. We measure piezoresistive effects between all pairs of electrodes in the set, and count on this rich signal set containing the information needed to pinpoint contact location with high accuracy using regression algorithms. In our validation experiments, we demonstrate submillimeter median accuracy in locating contact on a 10mm by 16mm sensor using only four electrodes (creating six unique pairs). In addition to extracting more information from fewer wires, this approach lends itself to simple fabrication methods and makes no assumptions about the underlying geometry, simplifying future integration on robot fingers. △ Less

Submitted 23 January, 2018; originally announced January 2018.

Comments: 7 pages, 7 figures, IROS 2016

Passive Reaction Analysis for Grasp Stability

Authors: Maximilian Haas-Heger, Garud Iyengar, Matei Ciocarlie

Abstract: In this paper we focus on the following problem in multi-fingered robotic grasping: assuming that an external wrench is being applied to a grasped object, will the contact forces between the hand and the object, as well as the hand joints, respond in such a way as to preserve quasi-static equilibrium? In particular, we assume that there is no change in the joint torques being actively exerted by t… ▽ More In this paper we focus on the following problem in multi-fingered robotic grasping: assuming that an external wrench is being applied to a grasped object, will the contact forces between the hand and the object, as well as the hand joints, respond in such a way as to preserve quasi-static equilibrium? In particular, we assume that there is no change in the joint torques being actively exerted by the motors; any change in contact forces and joint torques is due exclusively to passive effects arising in response to the external disturbance. Such passive effects include for example joints that are driven by highly geared motors (a common occurence in practice) and thus do not back drive in response to external torques. To account for non- linear phenomena encountered in such cases, and which existing methods do not consider, we formulate the problem as a mixed integer program used in the inner loop of an iterative solver. We present evidence showing that this formulation captures important effects for assessing the stability of a grasp employing some of the most commonly used actuation mechanisms. △ Less

Submitted 19 January, 2018; originally announced January 2018.

Comments: In press for IEEE Transactions on Automation Science and Engineering Special Issue 12 pages, 9 figures, 1 table

On the Feasibility of Wearable Exotendon Networks for Whole-Hand Movement Patterns in Stroke Patients

Authors: Sangwoo Park, Lauri Bishop, Tara Post, Yuchen Xiao, Joel Stein, Matei Ciocarlie

Abstract: Fully wearable hand rehabilitation and assistive devices could extend training and improve quality of life for patients affected by hand impairments. However, such devices must deliver meaningful manipulation capabilities in a small and lightweight package. In this context, this paper investigates the capability of single-actuator devices to assist whole-hand movement patterns through a network of… ▽ More Fully wearable hand rehabilitation and assistive devices could extend training and improve quality of life for patients affected by hand impairments. However, such devices must deliver meaningful manipulation capabilities in a small and lightweight package. In this context, this paper investigates the capability of single-actuator devices to assist whole-hand movement patterns through a network of exotendons. Our prototypes combine a single linear actuator (mounted on a forearm splint) with a network of exotendons (routed on the surface of a soft glove). We investigated two possible tendon network configurations: one that produces full finger extension (overcoming flexor spasticity), and one that combines proximal flexion with distal extension at each finger. In experiments with stroke survivors, we measured the force levels needed to overcome various levels of spasticity and open the hand for grasping using the first of these configurations, and qualitatively demonstrated the ability to execute fingertip grasps using the second. Our results support the feasibility of developing future wearable devices able to assist a range of manipulation tasks. △ Less

Submitted 19 December, 2017; originally announced December 2017.

Comments: Appeared at the 2016 IEEE Intl. Conf. on Robotics and Automation