research-article

HeyTAP: Bridging the Gaps Between Users' Needs and Technology in IF-THEN Rules via Conversation

Authors:

Luigi De Russis,

Alberto Monge RoffarelloAuthors Info & Claims

AVI '20: Proceedings of the 2020 International Conference on Advanced Visual Interfaces

Article No.: 23, Pages 1 - 9

https://doi.org/10.1145/3399715.3399905

Published: 02 October 2020 Publication History

Abstract

In the Internet of Things era, users are willing to personalize the joint behavior of their connected entities, i.e., smart devices and online service, by means of IF-THEN rules. Unfortunately, how to make such a personalization effective and appreciated is still largely unknown. On the one hand, contemporary platforms to compose IF-THEN rules adopt representation models that strongly depend on the exploited technologies, thus making end-user personalization a complex task. On the other hand, the usage of technology-independent rules envisioned by recent studies opens up new questions, and the identification of available connected entities able to execute abstract users' needs become crucial. To this end, we present HeyTAP, a conversational and semantic-powered trigger-action programming platform able to map abstract users' needs to executable IF-THEN rules. By interacting with a conversational agent, the user communicates her personalization intentions and preferences. User's inputs, along with contextual and semantic information related to the available connected entities, are then used to recommend a set of IF-THEN rules that satisfies the user's needs. An exploratory study on 8 end users preliminary confirms the effectiveness and the appreciation of the approach, and shows that HeyTAP can successfully guide users from their needs to specific rules.

References

[1]

2020. Amazon Alexa. https://developer.amazon.com/en-US/alexa Accessed: 2020-01-20.

[2]

2020. Google Assistant. https://developers.google.com/assistant Accessed: 2020-01-20.

[3]

2020. IFTTT. https://ifttt.com/ Accessed: 2020-01-20.

[4]

2020. Zapier. https://zapier.com/ Accessed: 2020-01-20.

[5]

B. R. Barricelli and S. Valtolina. 2015. End-User Development: 5th International Symposium, IS-EUD 2015, Madrid, Spain, May 26-29, 2015. Proceedings. Springer International Publishing, Cham, Germany, Chapter Designing for End-User Development in the Internet of Things, 9--24. https://doi.org/10.1007/978-3-319-18425-8_2

[6]

Vint Cerf and Max Senges. 2016. Taking the Internet to the Next Physical Level. IEEE Computer 49, 2 (Feb 2016), 80--86. https://doi.org/10.1109/MC.2016.51

Digital Library

[7]

F. Corno, L. De Russis, and A. Monge Roffarello. 2017. A Semantic Web Approach to Simplifying Trigger-Action Programming in the IoT. Computer 50, 11 (2017), R@18--24. https://doi.org/10.1109/MC.2017.4041355

[8]

Fulvio Corno, Luigi De Russis, and Alberto Monge Roffarello. 2019. A high-level semantic approach to End-User Development in the Internet of Things. International Journal of Human-Computer Studies 125 (2019), 41--54. https://doi.org/10.1016/j.ijhcs.2018.12.008

Digital Library

[9]

Fulvio Corno, Luigi De Russis, and Alberto Monge Roffarello. 2019. RecRules: Recommending IF-THEN Rules for End-User Development. ACM Trans. Intell. Syst. Technol. 10, 5, Article 58 (Sept. 2019), 27 pages. https://doi.org/10.1145/3344211

Digital Library

[10]

Giuseppe Desolda, Carmelo Ardito, and Maristella Matera. 2017. Empowering End Users to Customize Their Smart Environments: Model, Composition Paradigms, and Domain-Specific Tools. ACM Transaction on Computer-Human Interaction (TOCHI) 24, 2, Article 12 (April 2017), 52 pages. https://doi.org/10.1145/3057859

Digital Library

[11]

Anind K. Dey, Timothy Sohn, Sara Streng, and Justin Kodama. 2006. iCAP: Interactive Prototyping of Context-aware Applications. In Proceedings of the 4th International Conference on Pervasive Computing (Dublin, Ireland) (PERVASIVE'06). Springer-Verlag, Berlin, Heidelberg, 254--271. https://doi.org/10.1007/11748625_16

Digital Library

[12]

Mathias Funk, L.-L Chen, S.-W Yang, and Y.-K Chen. 2018. Addressing the need to capture scenarios, intentions and preferences: Interactive intentional programming in the smart home. International Journal of Design 12 (04 2018), 53--66.

[13]

Giuseppe Ghiani, Marco Manca, Fabio Paternò, and Carmen Santoro. 2017. Personalization of Context-Dependent Applications Through Trigger-Action Rules. ACM Transactions on Computer-Human Interaction (TOCHI)24, 2, Article 14 (April 2017), 33 pages. https://doi.org/10.1145/3057861

Digital Library

[14]

Will Haines, Melinda Gervasio, Aaron Spaulding, and Bart Peintner. 2010. Recommendations for End-User Development. In Proceedings of the ACM RecSys 2010 Workshop on User-Centric Evaluation of Recommender Systems and Their Interfaces (UCERSTI).

[15]

J. Huang and M. Cakmak. 2015. Supporting Mental Model Accuracy in Trigger-action Programming. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Osaka, Japan) (UbiComp '15). ACM, New York, NY, USA, 215--225. https://doi.org/10.1145/2750858.2805830

Digital Library

[16]

Ting-Hao K. Huang, A. Azaria, and J. P. Bigham. 2016.: Creating IF-THEN Rules via Conversations with the Crowd. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (Santa Clara, California, USA) (CHI EA '16). ACM, New York, NY, USA, 1555--1562. https://doi.org/10.1145/2851581.2892502

Digital Library

[17]

Bart P. Knijnenburg, Martijn C. Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell. 2012. Explaining the user experience of recommender systems. User Modeling and User-Adapted Interaction 22, 4 (01 Oct 2012), 441--504. https://doi.org/10.1007/s11257-011-9118-4

Digital Library

[18]

Xianghang Mi, Feng Qian, Ying Zhang, and XiaoFeng Wang. 2017. An Empirical Characterization of IFTTT: Ecosystem, Usage, and Performance. In Proceedings of the 2017 Internet Measurement Conference (London, United Kingdom) (IMC '17). ACM, New York, NY, USA, 398--404. https://doi.org/10.1145/3131365.3131369

Digital Library

[19]

Blase Ur, Elyse McManus, Melwyn Pak Yong Ho, and Michael L. Littman. 2014. Practical Trigger-action Programming in the Smart Home. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI '14). ACM, New York, NY, USA, 803--812.

Digital Library

[20]

B. Ur, M. Pak Yong Ho, S. Brawner, J. Lee, S. Mennicken, N. Picard, D. Schulze, and M. L. Littman. 2016. Trigger-Action Programming in the Wild: An Analysis of 200,000 IFTTT Recipes. In Proceedings of the 34rd Annual ACM Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 3227--3231. https://doi.org/10.1145/2858036.2858556

Digital Library

[21]

Lina Yao, Quan Z. Sheng, Anne H.H. Ngu, Helen Ashman, and Xue Li. 2014. Exploring Recommendations in Internet of Things. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval (Gold Coast, Queensland, Australia) (SIGIR '14). ACM, New York, NY, USA, 855--858. https://doi.org/10.1145/2600428.2609458

Digital Library

Cited By

Gallo SPaterno FMalizia A(2023)Conversational Interfaces in IoT Ecosystems: Where We Are, What Is Still MissingProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3627775(279-293)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3627775
Mattioli APaternò F(2023)A Mobile Augmented Reality App for Creating, Controlling, Recommending Automations in Smart HomesProceedings of the ACM on Human-Computer Interaction10.1145/36042427:MHCI(1-22)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604242
Liu XShi YYu CGao CYang TLiang CShi Y(2023)Understanding In-Situ Programming for Smart Home AutomationProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962547:2(1-31)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3596254
Show More Cited By

Index Terms

HeyTAP: Bridging the Gaps Between Users' Needs and Technology in IF-THEN Rules via Conversation

Recommendations

From Users’ Intentions to IF-THEN Rules in the Internet of Things
In the Internet of Things era, users are willing to personalize the joint behavior of their connected entities, i.e., smart devices and online service, by means of trigger-action rules such as “IF the entrance Nest security camera detects a movement, THEN ...
TAPrec: supporting the composition of trigger-action rules through dynamic recommendations
IUI '20: Proceedings of the 25th International Conference on Intelligent User Interfaces

Nowadays, users can personalize the joint behavior of their connected entities, i.e., smart devices and online service, by means of trigger-action rules. As the number of supported technologies grows, however, so does the design space, i.e., the ...
Empowering End Users in Debugging Trigger-Action Rules
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

End users can program trigger-action rules to personalize the joint behavior of their smart devices and online services. Trigger-action programming is, however, a complex task for non-programmers and errors made during the composition of rules may lead ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AVI '20: Proceedings of the 2020 International Conference on Advanced Visual Interfaces

September 2020

613 pages

ISBN:9781450375351

DOI:10.1145/3399715

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AVI '20

AVI '20: International Conference on Advanced Visual Interfaces

September 28 - October 2, 2020

Salerno, Italy

Acceptance Rates

AVI '20 Paper Acceptance Rate 36 of 123 submissions, 29%;

Overall Acceptance Rate 128 of 490 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
164
Total Downloads

Downloads (Last 12 months)51
Downloads (Last 6 weeks)9

Reflects downloads up to 11 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gallo SPaterno FMalizia A(2023)Conversational Interfaces in IoT Ecosystems: Where We Are, What Is Still MissingProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3627775(279-293)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3627775
Mattioli APaternò F(2023)A Mobile Augmented Reality App for Creating, Controlling, Recommending Automations in Smart HomesProceedings of the ACM on Human-Computer Interaction10.1145/36042427:MHCI(1-22)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604242
Liu XShi YYu CGao CYang TLiang CShi Y(2023)Understanding In-Situ Programming for Smart Home AutomationProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962547:2(1-31)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3596254
Chen WYu CWang HWang ZYang LWang YShi WShi Y(2023)From Gap to Synergy: Enhancing Contextual Understanding through Human-Machine Collaboration in Personalized SystemsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606741(1-15)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606741
Liu LChen WWang TWang WWu GWei J(2023)Generating Scenario-Centric TAP Rules for Smart Homes by Mining Historical Event Logs2023 IEEE International Conference on Web Services (ICWS)10.1109/ICWS60048.2023.00015(21-27)Online publication date: Jul-2023
https://doi.org/10.1109/ICWS60048.2023.00015
Barricelli BBondioli AFogli DIemmolo LLocoro A(2023)Creating Routines for IoT Ecosystems through Conversation with Smart SpeakersInternational Journal of Human–Computer Interaction10.1080/10447318.2023.2247845(1-19)Online publication date: 20-Aug-2023
https://doi.org/10.1080/10447318.2023.2247845
Barricelli BFogli DLocoro A(2023)EUDabilityJournal of Systems and Software10.1016/j.jss.2022.111516195:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.jss.2022.111516
Salah AAdel GMohamed HBaghdady YMoussa S(2023)Towards personalized control of things using Arabic voice commands for elderly and with disabilities peopleInternational Journal of Information Technology10.1007/s41870-023-01628-6Online publication date: 15-Dec-2023
https://doi.org/10.1007/s41870-023-01628-6
Breve BDesolda GGreco FDeufemia V(2023)Democratizing Cybersecurity in Smart Environments: Investigating the Mental Models of Novices and ExpertsEnd-User Development10.1007/978-3-031-34433-6_9(145-161)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1007/978-3-031-34433-6_9
Andrao MBalducci FBreve BCena FDesolda GDeufemia VGena CMatera MMattioli APaternò FSantoro CTreccani BVernero FZancanaro M(2023)Understanding Concepts, Methods and Tools for End-User Control of Automations in Ecosystems of Smart Objects and ServicesEnd-User Development10.1007/978-3-031-34433-6_7(104-124)Online publication date: 6-Jun-2023
https://dl.acm.org/doi/10.1007/978-3-031-34433-6_7
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents