Expertise

To develop capable robotic systems and understandable robot interactions with humans, Dr. Zhao Han has been addressing a wide range of topics to gain a deep level of expertise in the interdisciplinary field of HRI, as well as robotics, AI, and AR.

Curated List

Robot explanation, explainable AI (XAI) (AI-HRI ’19, ICRA ’20 WS, THRI ’21a, THRI ’21b, Ph.D. Dissertation, AAAI-SSS ’22, WYSD ’22, THRI ’22b)
- Preferences (THRI ’21b), generation algorithms (THRI ’21a), survey (AI-HRI ’19)
Mobile manipulation, research testbed (ICRA ’19 FetchIt Competition, ICRA ’20)
Augmented reality (AR)
- Projector-based AR, projection mapping (AI-HRI ’20, THRI ’21a, Paladyn ’21, HRI ’22b, AI-HRI ’22)
- Head-worn AR (VAM-HRI ’22, THRI ’22a, HRI ’23a)
Natural language generation — referring form selection (HRI ’22 LBR, INLG ’22, IROS ’22)
Robot failures (ICRA ’20 WS, THRI ’21a, THRI ’21b, WYSD ’22)
Assistive robotics
- Human-robot handovers (HRI ’19, ICRA ’20 WS, THRI ’21b)
  - Proactive release (HRI ’19), handover failures (ICRA ’20 WS, THRI ’21b)
- Robotic wheelchair (Paladyn ’21)
Robot competition (ICRA ’19 FetchIt Competition, Panasonic Challenge ’20, ICRA ’20)
Robot task representation — behavior trees (BTs) (THRI ’21a, WYSD ’22)
Robot ethics (HRI ’22a, HRI ’23b)
Mental workload — teleoperation (ICRA ’21)
Cognitive architecture (IROS ’20)
Open science — code (AI-HRI ’20, HRI ’22b)
Data visualization, visual analytics (WI ’16)
Data mining — frequent itemset mining (KES ’15, BigDAS ’15, M.S. Thesis)

Publications and Competitions by Expertise

The expertise below is ordered by the aforementioned curated list. Some publications may appear in others as they demonstrate multiple expertise.

Robot Explanation,
Explainable AI (XAI)

To Understand Indicators of Robots Vision Capabilities 1

To Understand Indicators of Robots’ Vision Capabilities, VAM-HRI 2024.

Designing Indicators to Show a Robots Physical Vision Capability

Designing Indicators to Show a Robot’s Physical Vision Capability, IEEE VR 2024 Poster.

Detect screw subtree with Assumption Checker nodes 1080p

Reactive or Proactive? How Robots Should Explain Failures, HRI 2024.

causality

Causal-HRI: Causal Learning for Human-Robot Interaction, HRI 2024 Workshop Abstract.

Mixed Reality Robot Behavior Replay

Communicating Missing Causal Information to Explain a Robot’s Past Behavior, THRI, 2023.

record replay workflow chart

Mixed-Reality Robot Behavior Replay: A System Implementation, AI-HRI 2022.

The “detect screw” subtree with Assumption Checker nodes (prefixed C, green names) and Action nodes (prefixed A, white names). The root RetryUntilSuccessful node retries the subtree up to three times if a failure or assumption violation occurs. The ReactiveSequenceNode asynchronously runs all actions including “lift torso”, “look at table”, and “detect screw” while continuously checking the assumption, “check updated head3DCamera”. An example of pre and post-conditions can be seen before and after the “detect screw” action.

“Why Didn’t I Do It?” A Study Design to Evaluate Robot Explanations, WYSD 2022.

Causal Robot Communication Inspired by Observational Learning Insights

Causal Robot Communication Inspired by Observational Learning Insights, AAAI-SSS 2022.

Classroom: One of four experimental contexts. We compared four norm violation responses across four contexts, in each of which the robot played fundamentally different roles, and bore fundamentally different role-obligations. The images shown above depict frames from videos shown to participants during this experiment. Each image depicts an Action explanation used in a different context.

Teacher, Teammate, Subordinate, Friend: Generating Norm Violation Responses Grounded in Role-based Relational Norms, HRI 2022.

Robot Explanations: Preferences, Generation, and Communication, Ph.D. Dissertation.

establish the handover task

The Need for Verbal Robot Explanations and How People Would Like a Robot To Explain Itself, THRI, 2021.

Screw placing, a representative subtask of the gearbox kitting task, modeled as a Behavior Tree. The leftmost node’s ancestors are not shown. For more detail, refer to Section 5.

Building The Foundation of Robot Explanation Generation Using Behavior Trees, THRI, 2021.

List of reasons why peopme want robots to

Reasons People Want Explanations After Unrecoverable Pre-Handover Failures, ICRA 2020 WS.

diagram of explanation system

Towards A Robot Explanation System: A Survey and Our Approach to State Summarization, Storage and Querying, and Human Interface, AI-HRI 2019.

Mobile Manipulation

Detect screw subtree with Assumption Checker nodes 1080p

Reactive or Proactive? How Robots Should Explain Failures, HRI 2024.

Mixed Reality Robot Behavior Replay

Communicating Missing Causal Information to Explain a Robot’s Past Behavior, THRI, 2023.

nav path to

Projecting Robot Navigation Paths: Hardware and Software for Projected AR, HRI 2022.

fetchit Fetch inserted a large gear

Towards Mobile Multi-Task Manipulation in a Confined and Integrated Environment with Irregular Objects, ICRA 2020.

We won second place in fetchit mobile manipulation challenge

FetchIt Mobile Manipulation Challenge at ICRA 2019, Second Place.

Augmented Reality (AR)

Do Results in Experiments with Virtual Robots in Augmented Reality Transfer To Physical Robots An Experiment Design

Do Results in Experiments with Virtual Robots in Augmented Reality Transfer To Physical Robots? An Experiment Design, WYSD 2024.

To Understand Indicators of Robots Vision Capabilities 1

To Understand Indicators of Robots’ Vision Capabilities, VAM-HRI 2024.

Designing Indicators to Show a Robots Physical Vision Capability

Designing Indicators to Show a Robot’s Physical Vision Capability, IEEE VR 2024 Poster.

Mixed Reality Robot Behavior Replay

Communicating Missing Causal Information to Explain a Robot’s Past Behavior, THRI, 2023.

together close 16x9 1

Best of Both Worlds? Combining Different Forms of Mixed Reality Deictic Gestures, THRI, 2023.

Crossing Reality Comparing Physical and Virtual Robot

Crossing Reality: Comparing Physical and Virtual Robot Deixis, HRI 2023.

record replay workflow chart

Mixed-Reality Robot Behavior Replay: A System Implementation, AI-HRI 2022.

ar p 16 x 9 scaled

Towards an Understanding of Physical vs Virtual Robot Appendage Design, VAM-HRI 2022.

nav path to

Projecting Robot Navigation Paths: Hardware and Software for Projected AR, HRI 2022.

paladyn scooter poster

Design Guidelines for Human-Robot Interaction with Assistive Robot Manipulation Systems, Paladyn, 2021.

table top projection

Projection Mapping Implementation: Enabling Direct Externalization of Perception Results and Action Intent to Improve Robot Explainability, AI-HRI 2020.

References,
Natural Language Generation

generalized experiment flow 1080p

Towards Reproducible Language-Based HRI Experiments: Open-Sourcing a Generalized Experiment Project, HRI 2024.

The Importance of Memory for Language Capable Robots

The Importance of Memory for Language-Capable Robots, XRDS, 2023.

Toward Open World Human Robot Interaction What Types of Gestures Are Used in Task Based Open World Referential Communication 1

Toward Open-World Human-Robot Interaction: What Types of Gestures Are Used in Task-Based Open-World Referential Communication?, SemDial 2023.

Exploring the Naturalness of Cognitive Status Informed Referring Form Selection Models

Exploring the Naturalness of Cognitive Status Informed Referring Form Selection Models, INLG 2023.

rfs pepper

Evaluating Cognitive Status-Informed Referring Form Selection for Human-Robot Interactions, CogSci 2023.

Givenness Hierarchy Informed Optimal Sentence Planning for Situated Human Robot Interaction

Givenness Hierarchy Informed Optimal Document Planning for Situated Human-Robot Interaction, IROS 2022.

referring form selection data decision tree model

Evaluating Referring Form Selection Models in Partially-Known Environments, INLG 2022.

Two of four quadrants of the task environment to study referring behaviors for linguistic HRI. Two rule cards are placed on each table, with learners' on the left hand side and instructors' (dashed box) on the right hand side (solid box). Objects are intentionally placed at the intersections of a 3×3 grid to encourage use of different referring forms whose use varies according to distance. Instructors teach learners to construct buildings whose constituents blocks are distributed across the visible and non-visible quadrants.

A Task Design for Studying Referring Behaviors for Linguistic HRI, HRI 2022 LBR.

Cognitive Robotics

The Importance of Memory for Language Capable Robots

The Importance of Memory for Language-Capable Robots, XRDS, 2023.

Exploring the Naturalness of Cognitive Status Informed Referring Form Selection Models

Exploring the Naturalness of Cognitive Status Informed Referring Form Selection Models, INLG 2023.

rfs pepper

Evaluating Cognitive Status-Informed Referring Form Selection for Human-Robot Interactions, CogSci 2023.

Givenness Hierarchy Informed Optimal Sentence Planning for Situated Human Robot Interaction

Givenness Hierarchy Informed Optimal Document Planning for Situated Human-Robot Interaction, IROS 2022.

referring form selection data decision tree model

Evaluating Referring Form Selection Models in Partially-Known Environments, INLG 2022.

Causal Robot Communication Inspired by Observational Learning Insights

Causal Robot Communication Inspired by Observational Learning Insights, AAAI-SSS 2022.

Two of four quadrants of the task environment to study referring behaviors for linguistic HRI. Two rule cards are placed on each table, with learners' on the left hand side and instructors' (dashed box) on the right hand side (solid box). Objects are intentionally placed at the intersections of a 3×3 grid to encourage use of different referring forms whose use varies according to distance. Instructors teach learners to construct buildings whose constituents blocks are distributed across the visible and non-visible quadrants.

A Task Design for Studying Referring Behaviors for Linguistic HRI, HRI 2022 LBR.

After teaching using cognitive architecture, Fetch is asked to describe how to assemble a caddy

Going Cognitive: A Demonstration of the Utility of Task-General Cognitive Architectures for Adaptive Robotic Task Performance, IROS 2020.

Robot Failures

Detect screw subtree with Assumption Checker nodes 1080p

Reactive or Proactive? How Robots Should Explain Failures, HRI 2024.

Mixed Reality Robot Behavior Replay

Communicating Missing Causal Information to Explain a Robot’s Past Behavior, THRI, 2023.

record replay workflow chart

Mixed-Reality Robot Behavior Replay: A System Implementation, AI-HRI 2022.

The “detect screw” subtree with Assumption Checker nodes (prefixed C, green names) and Action nodes (prefixed A, white names). The root RetryUntilSuccessful node retries the subtree up to three times if a failure or assumption violation occurs. The ReactiveSequenceNode asynchronously runs all actions including “lift torso”, “look at table”, and “detect screw” while continuously checking the assumption, “check updated head3DCamera”. An example of pre and post-conditions can be seen before and after the “detect screw” action.

“Why Didn’t I Do It?” A Study Design to Evaluate Robot Explanations, WYSD 2022.

establish the handover task

The Need for Verbal Robot Explanations and How People Would Like a Robot To Explain Itself, THRI, 2021.

Screw placing, a representative subtask of the gearbox kitting task, modeled as a Behavior Tree. The leftmost node’s ancestors are not shown. For more detail, refer to Section 5.

Building The Foundation of Robot Explanation Generation Using Behavior Trees, THRI, 2021.

List of reasons why peopme want robots to

Reasons People Want Explanations After Unrecoverable Pre-Handover Failures, ICRA 2020 WS.

Human-Robot Handovers

establish the handover task

The Need for Verbal Robot Explanations and How People Would Like a Robot To Explain Itself, THRI, 2021.

List of reasons why peopme want robots to

Reasons People Want Explanations After Unrecoverable Pre-Handover Failures, ICRA 2020 WS.

4 Baxter robot released the object and the participant took it scaled

The Effects of Proactive Release Behaviors During Human-Robot Handovers, HRI 2019.

Before Baxter hands the object it looks at the object full

Investigating Human-Robot Handover Release Behaviors, NEMS 2018.

Hardware

nav path to

Projecting Robot Navigation Paths: Hardware and Software for Projected AR, HRI 2022.

table top projection

Projection Mapping Implementation: Enabling Direct Externalization of Perception Results and Action Intent to Improve Robot Explainability, AI-HRI 2020.

Before Baxter hands the object it looks at the object full

Investigating Human-Robot Handover Release Behaviors, NEMS 2018.

Visualization

nav path to

Projecting Robot Navigation Paths: Hardware and Software for Projected AR, HRI 2022.

table top projection

Projection Mapping Implementation: Enabling Direct Externalization of Perception Results and Action Intent to Improve Robot Explainability, AI-HRI 2020.

BundleVis. A frequent itemset visualization. Course 472 is clicked.

Delivering Scalable Frequent Pattern Mining for Non-Expert Data Miners, M.S. Thesis.

The interactive visual analytics tool (icVAT) for frequent pattern mining.

An Interactive Circular Visual Analytic Tool for Visualization of Web Data, WI 2016.

Robot Competition

adi challenge title scaled

ADI Real-Time Sensor Fusion Challenge, Finalist.

panasonic challenge winner

Panasonic Prototype 3D LiDAR Challenge, First Place.

fetchit Fetch inserted a large gear

Towards Mobile Multi-Task Manipulation in a Confined and Integrated Environment with Irregular Objects, ICRA 2020.

We won second place in fetchit mobile manipulation challenge

FetchIt Mobile Manipulation Challenge at ICRA 2019, Second Place.

Behavior Trees (BTs),
Robot Task Representation

Detect screw subtree with Assumption Checker nodes 1080p

Reactive or Proactive? How Robots Should Explain Failures, HRI 2024.

Mixed Reality Robot Behavior Replay

Communicating Missing Causal Information to Explain a Robot’s Past Behavior, THRI, 2023.

record replay workflow chart

Mixed-Reality Robot Behavior Replay: A System Implementation, AI-HRI 2022.

The “detect screw” subtree with Assumption Checker nodes (prefixed C, green names) and Action nodes (prefixed A, white names). The root RetryUntilSuccessful node retries the subtree up to three times if a failure or assumption violation occurs. The ReactiveSequenceNode asynchronously runs all actions including “lift torso”, “look at table”, and “detect screw” while continuously checking the assumption, “check updated head3DCamera”. An example of pre and post-conditions can be seen before and after the “detect screw” action.

“Why Didn’t I Do It?” A Study Design to Evaluate Robot Explanations, WYSD 2022.

Screw placing, a representative subtask of the gearbox kitting task, modeled as a Behavior Tree. The leftmost node’s ancestors are not shown. For more detail, refer to Section 5.

Building The Foundation of Robot Explanation Generation Using Behavior Trees, THRI, 2021.

Assistive Robotics

paladyn scooter poster

Design Guidelines for Human-Robot Interaction with Assistive Robot Manipulation Systems, Paladyn, 2021.

List of reasons why peopme want robots to

Reasons People Want Explanations After Unrecoverable Pre-Handover Failures, ICRA 2020 WS.

4 Baxter robot released the object and the participant took it scaled

The Effects of Proactive Release Behaviors During Human-Robot Handovers, HRI 2019.

Before Baxter hands the object it looks at the object full

Investigating Human-Robot Handover Release Behaviors, NEMS 2018.

Virtual Reality (VR)

abstract gesture pepper

(Gestures Vaguely): The Effects of Robots’ Use of Abstract Pointing Gestures in Large-Scale Environments, HRI 2024.

Teleoperation,
Mental Workload

multi view

Investigation of Multiple Resource Theory Design Principles on Robot Teleoperation and Workload Management, ICRA 2021.

Robot Ethics

Fresh Start Encouraging Politeness in Wakeword Driven Human Robot Interaction

Fresh Start: Encouraging Politeness in Wakeword-Driven Human-Robot Interaction, HRI 2023.

Classroom: One of four experimental contexts. We compared four norm violation responses across four contexts, in each of which the robot played fundamentally different roles, and bore fundamentally different role-obligations. The images shown above depict frames from videos shown to participants during this experiment. Each image depicts an Action explanation used in a different context.

Teacher, Teammate, Subordinate, Friend: Generating Norm Violation Responses Grounded in Role-based Relational Norms, HRI 2022.

Diversity, Equity, and Inclusion

Inclusive HRI II

3rd Workshop on Inclusive HRI: Equity and Diversity in Design, Application, Methods, and Community, HRI 2024 Workshop Abstract.

Inclusive HRI II

Inclusive HRI II: Equity and Diversity in Design, Application, Methods, and Community, HRI 2023 Workshop Abstract.

Data Collection

hri data collection process

Towards Formalizing HRI Data Collection Processes, HRI Methods & Metrics 2022.

Data Mining

BundleVis. A frequent itemset visualization. Course 472 is clicked.

Delivering Scalable Frequent Pattern Mining for Non-Expert Data Miners, M.S. Thesis.

FIMaaS arch

FIMaaS: Scalable Frequent Itemset Mining-as-a-Service on Cloud for Non-Expert Miners, BigDAS 2015.

Edge-based mining of frequent subgraphs from graph streams, KES 2015.

Cloud Computing

FIMaaS arch

FIMaaS: Scalable Frequent Itemset Mining-as-a-Service on Cloud for Non-Expert Miners, BigDAS 2015.

Expertise Grid

Explainable AI (XAI)

14 Publications

Mobile Manipulation

Augmented Reality (AR)

11 Publications

Natural Language Generation

References

12 Publications

Cognitive Robotics

Robot Failures

Human-Robot Handovers

Hardware

Visualization

Robot Competition

Robot Task Representation

Assistive Robotics

Virtual Reality (VR)

Teleoperation

Mental Workload

Robot Ethics

Diversity, Equity, and Inclusion

Data Collection

Data Mining

Cloud Computing

Publications →

Competitions →