This special issue of the Transactions on Human-Robot Interaction highlights, documents, and expl... more This special issue of the Transactions on Human-Robot Interaction highlights, documents, and explores the metrics, test methods, and artifacts used in human-robot interaction (HRI) research. This collection of articles brings to attention the commonalities between the application of measurement science for the assessment and assurance of human-centric robotics in a variety of application domains, including industry, education, and defense. This special issue draws specific attention to the use and impact of metrology toward the advancement of HRI technologies and algorithms, and it promotes the application of measurement science toward the benchmarking and replication of HRI research. Special attention is given to the use cases, data sets, test methodologies, measurement techniques, metrics, and statistical analyses used to evaluate system performance.
This work analyzes a total of 1464 papers through examination of seven years of HRI Conference pr... more This work analyzes a total of 1464 papers through examination of seven years of HRI Conference proceedings(2015 through 2021) and six years of Ro-Man Conference proceedings (2015 through 2020) to present a holistic snapshot of the state of methods and metrics in HRI research. Workshop proposals, late-breaking results, keynote talk abstracts, and demo presentations were omitted from this study due to varying methodologies and shorter study timeframes.
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successfu... more The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration since 2014. In that time, the related topic of trust in robotics has been rapidly growing, with major research efforts at universities and laboratories across the world. Indeed, many of the past participants in AI-HRI have been or are now involved with research into trust in HRI. While trust has no consensus definition, it is regularly associated with predictability, reliability, inciting confidence, and meeting expectations. Furthermore, it is generally believed that trust is crucial for adoption of both AI and robotics, particularly when transitioning technologies from the lab to industrial, social, and consumer applications. However, how does trust apply to the specific situations we encounter in the AI-HRI sphere? Is the notion of trust in AI the same as that in HRI? We see a growing need for research that lives directly at the intersection of ...
This report details the second annual, full-day workshop exploring the metrology necessary for re... more This report details the second annual, full-day workshop exploring the metrology necessary for repeatably and independently assessing the performance of robotic systems in realworld human-robot interaction (HRI) scenarios. This workshop continues the motion toward bridging the gaps between the theory and applications of HRI, enabling reproducible studies in HRI, and accelerating the adoption of cutting edge technologies as the industry state-of-practice. The second annual workshop, ‘Test Methods and Metrics for Effective HRI’, seeks to identify test methods and metrics for the holistic assessment and assurance of HRI performance in practical applications. The workshop was held on March 23, 2020, as part of the virtual ACM/IEEE International Conference on Human-Robot Interaction. The focus is on identifying the key performance indicators of seemingly disparate sectors, including manufacturing, social, medical, and service robot solutions among others, and to foster the community base...
2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2019
Verified and validated test methods, being necessary to measure the performance of complex system... more Verified and validated test methods, being necessary to measure the performance of complex systems, are important tools for driving innovation, benchmarking and improving performance, and establishing trust in collaborative human-robot teams. This full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the collaborative performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop aims to bridge the gaps between the theory and applications of HRI in industry, accelerating the adoption of cutting edge technologies as the industry state-of-practice. The interest in collaborative HRI is evident in the current market as well as standards efforts toward manufacturing, social, medical, and service robot solutions. Though these domains have been considered separate for many years, recent technological and scientific advancements show that, while their applications may differ, the underlying principles of HRI...
As part of the U.S. Department of Commerce, the National Institute of Standards and Technology (N... more As part of the U.S. Department of Commerce, the National Institute of Standards and Technology (NIST) is examining new approaches to increase the competitiveness of smaller U.S.-basedmanufacturers. Collaborative robotics is one way to increase automation, leveraging the specialized skills of humanworkers along with the increased repeatability and throughput of robots. However, in order for humans to efficiently work alongside robots, new types of interfaces are needed to streamline interactions. Augmented reality, especially wearable devices, are a new approach that can enable task training, hands-free interactions, and increased safety through situational awareness. This paper will explore some preliminary approaches to integrating augmented reality for a collaborative manufacturing environment.
Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, 2020
This second annual, full-day workshop aims to explore the metrology necessary for repeatably and ... more This second annual, full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop continues the motion toward bridging the gaps between the theory and applications of HRI, enabling reproducible studies in HRI, and accelerating the adoption of cutting edge technologies as the industry state-of-practice. The second annual workshop, "Test Methods and Metrics for Effective HRI," seeks to identify test methods and metrics for the holistic assessment and assurance of HRI performance in practical applications. The focus is on identifying the key performance indicators of seemingly disparate sectors, and to foster the community based on the principles of transparency, repeatability and reproducibility, and establishing trust. The goal is to aid in the advancement of HRI technologies through the development of experimental design, test metho...
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successfu... more The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration since 2014. During that time, these symposia provided a fertile ground for numerous collaborations and pioneered many discussions revolving trust in HRI, XAI for HRI, service robots, interactive learning, and more. This year, we aim to review the achievements of the AI-HRI community in the last decade, identify the challenges facing ahead, and welcome new researchers who wish to take part in this growing community. Taking this wide perspective, this year there will be no single theme to lead the symposium and we encourage AI-HRI submissions from across disciplines and research interests. Moreover, with the rising interest in AR and VR as part of an interaction and following the difficulties in running physical experiments during the pandemic, this year we specifically encourage researchers to submit works that do not include a physical robot in the...
Verified and validated test methods, being necessary to measure the performance of complex system... more Verified and validated test methods, being necessary to measure the performance of complex systems, are important tools for driving innovation, benchmarking and improving performance, and establishing trust in collaborative human-robot teams. This full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the collaborative performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop aims to bridge the gaps between the theory and applications of HRI in industry, accelerating the adoption of cutting edge technologies as the industry state-of-practice. The workshop was held on March 11, 2019, at the ACM/IEEE International Conference on Human-Robot Interaction in Daegu, South Korea. The interest in collaborative HRI is evident in the current market as well as standards efforts toward manufacturing, social, medical, and service robot solutions. Though these domains have been considered separate for many year...
This special issue of the Transactions on Human-Robot Interaction highlights, documents, and expl... more This special issue of the Transactions on Human-Robot Interaction highlights, documents, and explores the metrics, test methods, and artifacts used in human-robot interaction (HRI) research. This collection of articles brings to attention the commonalities between the application of measurement science for the assessment and assurance of human-centric robotics in a variety of application domains, including industry, education, and defense. This special issue draws specific attention to the use and impact of metrology toward the advancement of HRI technologies and algorithms, and it promotes the application of measurement science toward the benchmarking and replication of HRI research. Special attention is given to the use cases, data sets, test methodologies, measurement techniques, metrics, and statistical analyses used to evaluate system performance.
This work analyzes a total of 1464 papers through examination of seven years of HRI Conference pr... more This work analyzes a total of 1464 papers through examination of seven years of HRI Conference proceedings(2015 through 2021) and six years of Ro-Man Conference proceedings (2015 through 2020) to present a holistic snapshot of the state of methods and metrics in HRI research. Workshop proposals, late-breaking results, keynote talk abstracts, and demo presentations were omitted from this study due to varying methodologies and shorter study timeframes.
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successfu... more The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration since 2014. In that time, the related topic of trust in robotics has been rapidly growing, with major research efforts at universities and laboratories across the world. Indeed, many of the past participants in AI-HRI have been or are now involved with research into trust in HRI. While trust has no consensus definition, it is regularly associated with predictability, reliability, inciting confidence, and meeting expectations. Furthermore, it is generally believed that trust is crucial for adoption of both AI and robotics, particularly when transitioning technologies from the lab to industrial, social, and consumer applications. However, how does trust apply to the specific situations we encounter in the AI-HRI sphere? Is the notion of trust in AI the same as that in HRI? We see a growing need for research that lives directly at the intersection of ...
This report details the second annual, full-day workshop exploring the metrology necessary for re... more This report details the second annual, full-day workshop exploring the metrology necessary for repeatably and independently assessing the performance of robotic systems in realworld human-robot interaction (HRI) scenarios. This workshop continues the motion toward bridging the gaps between the theory and applications of HRI, enabling reproducible studies in HRI, and accelerating the adoption of cutting edge technologies as the industry state-of-practice. The second annual workshop, ‘Test Methods and Metrics for Effective HRI’, seeks to identify test methods and metrics for the holistic assessment and assurance of HRI performance in practical applications. The workshop was held on March 23, 2020, as part of the virtual ACM/IEEE International Conference on Human-Robot Interaction. The focus is on identifying the key performance indicators of seemingly disparate sectors, including manufacturing, social, medical, and service robot solutions among others, and to foster the community base...
2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2019
Verified and validated test methods, being necessary to measure the performance of complex system... more Verified and validated test methods, being necessary to measure the performance of complex systems, are important tools for driving innovation, benchmarking and improving performance, and establishing trust in collaborative human-robot teams. This full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the collaborative performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop aims to bridge the gaps between the theory and applications of HRI in industry, accelerating the adoption of cutting edge technologies as the industry state-of-practice. The interest in collaborative HRI is evident in the current market as well as standards efforts toward manufacturing, social, medical, and service robot solutions. Though these domains have been considered separate for many years, recent technological and scientific advancements show that, while their applications may differ, the underlying principles of HRI...
As part of the U.S. Department of Commerce, the National Institute of Standards and Technology (N... more As part of the U.S. Department of Commerce, the National Institute of Standards and Technology (NIST) is examining new approaches to increase the competitiveness of smaller U.S.-basedmanufacturers. Collaborative robotics is one way to increase automation, leveraging the specialized skills of humanworkers along with the increased repeatability and throughput of robots. However, in order for humans to efficiently work alongside robots, new types of interfaces are needed to streamline interactions. Augmented reality, especially wearable devices, are a new approach that can enable task training, hands-free interactions, and increased safety through situational awareness. This paper will explore some preliminary approaches to integrating augmented reality for a collaborative manufacturing environment.
Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, 2020
This second annual, full-day workshop aims to explore the metrology necessary for repeatably and ... more This second annual, full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop continues the motion toward bridging the gaps between the theory and applications of HRI, enabling reproducible studies in HRI, and accelerating the adoption of cutting edge technologies as the industry state-of-practice. The second annual workshop, "Test Methods and Metrics for Effective HRI," seeks to identify test methods and metrics for the holistic assessment and assurance of HRI performance in practical applications. The focus is on identifying the key performance indicators of seemingly disparate sectors, and to foster the community based on the principles of transparency, repeatability and reproducibility, and establishing trust. The goal is to aid in the advancement of HRI technologies through the development of experimental design, test metho...
The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successfu... more The Artificial Intelligence (AI) for Human-Robot Interaction (HRI) Symposium has been a successful venue of discussion and collaboration since 2014. During that time, these symposia provided a fertile ground for numerous collaborations and pioneered many discussions revolving trust in HRI, XAI for HRI, service robots, interactive learning, and more. This year, we aim to review the achievements of the AI-HRI community in the last decade, identify the challenges facing ahead, and welcome new researchers who wish to take part in this growing community. Taking this wide perspective, this year there will be no single theme to lead the symposium and we encourage AI-HRI submissions from across disciplines and research interests. Moreover, with the rising interest in AR and VR as part of an interaction and following the difficulties in running physical experiments during the pandemic, this year we specifically encourage researchers to submit works that do not include a physical robot in the...
Verified and validated test methods, being necessary to measure the performance of complex system... more Verified and validated test methods, being necessary to measure the performance of complex systems, are important tools for driving innovation, benchmarking and improving performance, and establishing trust in collaborative human-robot teams. This full-day workshop aims to explore the metrology necessary for repeatably and independently assessing the collaborative performance of robotic systems in real-world human-robot interaction (HRI) scenarios. This workshop aims to bridge the gaps between the theory and applications of HRI in industry, accelerating the adoption of cutting edge technologies as the industry state-of-practice. The workshop was held on March 11, 2019, at the ACM/IEEE International Conference on Human-Robot Interaction in Daegu, South Korea. The interest in collaborative HRI is evident in the current market as well as standards efforts toward manufacturing, social, medical, and service robot solutions. Though these domains have been considered separate for many year...
Uploads
Papers by Shelly Bagchi