Abstract
The continuously increasing number of connected objects and sensors is opening up the possibility of introducing automations in many domains to better support people in their activities. However, such automations to be effective should be under the user control. Unfortunately, people often report difficulties in understanding the surrounding automations and how to modify them. The goal of this paper is to provide a multi-perspective view of what has been done in terms of design, tools, and evaluation in the area of end-user control of automations in ecosystems of smart objects and services. For each aspect we introduce the main challenge, the current possible approaches to address it, and the issues that still need further investigation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
https://www.statista.com/statistics/1183457/iot-connected-devices-worldwide/, last accessed 2023/04/12.
- 2.
https://www.upod.io/datasets.html, last accessed 2023/04/12.
- 3.
https://www-users.cse.umn.edu/~fengqian/ifttt_measurement/, last accessed 2023/04/12.
- 4.
https://doi.org/10.5281/zenodo.5572861, last accessed 2023/04/12.
- 5.
https://github.com/andrematt/trigger_action_rules, last accessed 2023/04/12.
References
Andrao, M., Desolda, G., Greco, F., Manfredi, R., Treccani, B., Zancanaro, M.: End-user programming and math teachers: an initial study. In: Proceedings of the 2022 International Conference on Advanced Visual Interfaces, pp. 1–3 (2022)
Andrao, M., Treccani, B., Zancanaro, M.: Therapists as designers: an initial investigation of end-user programming of a tangible tool for therapeutic interventions. In: Proceedings of the 2nd International Workshop on Empowering People in Dealing with Internet of Things Ecosystems co-located with INTERACT 2021, Bari, Italy, Online / Bari, Italy, September 30, 2021(CEUR Workshop Proceedings, vol. 3053). CEUR-WS.org, pp. 38–42 (2021). http://ceur-ws.org/Vol-3053/paper_8.pdf
Ardito, C., Desolda, G., Lanzilotti, R., Malizia, A., Matera, M.: Analysing trade-offs in frameworks for the design of smart environments. Behav. Inf. Technol. 39(1), 47–71 (2020)
Ardito, C., et al.: User-defined semantics for the design of IoT systems enabling smart interactive experiences. Pers. Ubiquit. Comput. 24(6), 781–796 (2020). https://doi.org/10.1007/s00779-020-01457-5
Ariano, R., Manca, M., Paternò, F., Santoro, C.: Smartphone-based augmented reality for end-user creation of home automations. Behav. Inf. Techno. 42(1), 124–140 (2023)
Balducci, F., Buono, P., Desolda, G., Impedovo, D., Piccinno, A.: Improving smart interactive experiences in cultural heritage through pattern recognition techniques. Pattern Recogn. Lett. 131, 142–149 (2020)
Bellucci, A., Vianello, A., Florack, Y., Micallef, L., Jacucci, G.: Augmenting objects at home through programmable sensor tokens: a design journey. Int. J. Hum. Comput Stud. 122, 211–231 (2019)
Brackenbury, W., et al.: How users interpret bugs in trigger-action programming. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI 2019). Association for Computing Machinery, New York, NY, USA, Article Paper 552, p. 12 (2019). https://doi.org/10.1145/3290605.330078
Breve, B., Cimino, G., Deufemia, V.: Towards explainable security for ECA rules. In: Proceedings of the 3rd International Workshop on Empowering End-Users in Dealing with Internet of Things Ecosystems (EMPATHY), CEUR-WS, vol. 3172, pp. 26–30 (2022)
Breve, B., Cimino, G., Deufemia, V.: Identifying security and privacy violation rules in trigger-action IoT platforms with NLP models. IEEE Internet Things J. 10(6), 5607–5622 (2023)
Brich, J., Walch, M., Rietzler, M., Weber, M., Schaub, F.: Exploring end user programming needs in home automation. ACM Trans. Comput. Human Interact. (TOCHI) 24(2), 1–35 (2017)
Cabitza, F., Fogli, D., Lanzilotti, R., Piccinno, A.: Rule-based tools for the configuration of ambient intelligence systems: a comparative user study. Multimed. Tools Appl. 76(4), 5221–5241 (2016). https://doi.org/10.1007/s11042-016-3511-2
Casati, F., Castano, S., Fugini, M., Mirbel, I., Pernici, B.: Using patterns to design rules in workflows. IEEE Trans. Softw. Eng. 26(8), 760–785 (2000)
Cena, F., et al.: Incorporating personality traits in user modeling for EUD. In: 3rd International Workshop on Empowering People in Dealing with Internet of Things Ecosystems, CEUR Workshop Proceedings, vol. 3172, pp. 41–48 (2022)
Chen, X., et al.: Fix the leaking tap: a survey of trigger-action programming (TAP) security issues, detection techniques and solutions. Comput. Secur. 102812 (2022)
Clark, M., Newman, M.W., Dutta, P.: ARticulate: one-shot interactions with intelligent assistants in unfamiliar smart spaces using augmented reality. Proc. ACM Interact. Mob. Wearabl. Ubiquit. Technol. 6(1), 1–24 (2022)
Cobb, C., et al.: How risky are real users’ IFTTT applets? In: Proceedings of the Sixteenth Symposium on Usable Privacy and Security (SOUPS 2020), pp. 505–529 (2020)
Coppers, S., Vanacken, D., Luyten, K.: FortClash: predicting and mediating unintended behavior in home automation. Proc. ACM Human Comput. Interact. 6(EICS), 1–20 (2022). https://doi.org/10.1145/3532204
Coppers, S., Vanacken, D., Luyten, K.: Fortniot: Intelligible predictions to improve user understanding of smart home behavior. Proc. ACM Interact. Mob. Wearabl. Ubiquit. Technol. 4(4), 1–24 (2020)
Corcella, L., Manca, M., Paternò, F.: Personalizing a student home behaviour. In: Barbosa, S., Markopoulos, P., Paternò, F., Stumpf, S., Valtolina, S. (eds.) End-User Development. LNCS, vol. 10303, pp. 18–33. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58735-6_2
Corno, F., De Russis, L., Roffarello, A.M.: A high-level semantic approach to end-user development in the Internet of Things. Int. J. Hum Comput Stud. 125, 41–54 (2019)
Corno, F., De Russis, L., Roffarello, A.M.: Empowering end users in debugging trigger-action rules. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–13 (2019)
Corno, F., De Russis, L., Roffarello, A.M.: RecRules: recommending IF-THEN rules for end-user development. ACM Trans. Intell. Syst. Technol. 10(5), 1–27 (2019). https://doi.org/10.1145/3344211
Corno, F., De Russis, L., Roffarello, A.M.: HeyTAP: bridging the gaps between users’ needs and technology in IF-THEN rules via conversation. In: Proceedings of the International Conference on Advanced Visual Interfaces, pp. 1–9 (2020)
Corno, F., De Russis, L., Roffarello, A.M.: TAPrec: supporting the composition of trigger-action rules through dynamic recommendations. In: Proceedings of the 25th International Conference on Intelligent User Interfaces, pp. 579–588 (2020)
Corno, F., De Russis, L., Roffarello, A.M.: Devices, information, and people: abstracting the internet of things for end-user personalization. In: Fogli, D., Tetteroo, D., Barricelli, B.R., Borsci, S., Markopoulos, P., Papadopoulos, G.A. (eds.) End-User Development. LNCS, vol. 12724, pp. 71–86. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-79840-6_5
Cypher, A.: Eager: programming repetitive tasks by example. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 33–39 (1991)
De Russis, L., Roffarello, A.M.: A debugging approach for trigger-action programming. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (2018). https://doi.org/10.1145/3170427.3188641
de Vega, M., Rinck, M., Diaz, J.M., León, I.: Figure and ground in temporal sentences: the role of the adverbs when and while. Discourse Process. 43(1), 1–23 (2007). https://doi.org/10.1080/01638530709336891
Desolda, G., Ardito, C., Matera, M.: Empowering end users to customize their smart environments: model, composition paradigms, and domain-specific tools. ACM Trans. Comput. Human Interact. (TOCHI) 24(2), 1–52 (2017)
Desolda, G., Greco, F., Guarnieri, F., Mariz, N., Zancanaro, M.: SENSATION: an authoring tool to support event–state paradigm in end-user development. In: Ardito, C., et al. (eds.) Human-Computer Interaction – INTERACT 2021. LNCS, vol. 12933, pp. 373–382. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85616-8_22
Epifania, F., Cremonesi, P.: User-centered evaluation of recommender systems with comparison between short and long profile. In: 2012 Sixth International Conference on Complex, Intelligent, and Software Intensive Systems, pp. 204–211. IEEE (2012)
Fitzgerald, S., et al.: Debugging: finding, fixing and flailing, a multi-institutional study of novice debuggers. Comput. Sci. Educ. 18(2), 93–116 (2008)
Fitzgerald, S., McCauley, R., Hanks, B., Murphy, L., Simon, B., Zander, C.: Debugging from the student perspective. IEEE Trans. Educ. 53(3), 390–396 (2010). https://doi.org/10.1109/TE.2009.2025266
Fogli, D., Peroni, M., Stefini, C.: ImAtHome: Making trigger-action programming easy and fun. J. Vis. Lang. Comput. 42, 60–75 (2017)
Gallitto, G., Treccani, B., Zancanaro, M.: If when is better than if (and while might help): on the importance of influencing mental models in EUD (a pilot study). In: Proceedings of the 1st International Workshop on Empowering People in Dealing with Internet of Things Ecosystems co-located with International Conference on Advanced Visual Interfaces (AVI), Ischia Island, Italy, 2020, pp. 7–11 (2020)
Gallo, S., Paterno, F.: A conversational agent for creating flexible daily automation. In: Proceedings of the 2022 International Conference on Advanced Visual Interfaces, pp. 1–8 (2022)
Gallo, S., Manca, M., Mattioli, A., Paternò, F., Santoro, C.: Comparative analysis of composition paradigms for personalization rules in IoT settings. In: Fogli, D., Tetteroo, D., Barricelli, B.R., Borsci, S., Markopoulos, P., Papadopoulos, G.A. (eds.) End-User Development. LNCS, vol. 12724, pp. 53–70. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-79840-6_4
Gao, L., Bai, X.: A unified perspective on the factors influencing consumer acceptance of internet of things technology. Asia Pac. J. Market. Logist. 26(2), 211–231 (2014). https://doi.org/10.1108/APJML-06-2013-0061
Gennari, R., Matera, M., Morra, D., Melonio, A., Rizvi, M.: Design for social digital well-being with young generations: Engage them and make them reflect. Int. J. Human Comput. Stud. 173, 103006 (2023). https://doi.org/10.1016/j.ijhcs.2023.103006
Ghiani, G., Manca, M., Paternò, F., Santoro, C.: Personalization of context-dependent applications through trigger-action rules. ACM Trans. Computer-Human Interaction (TOCHI) 24(2), 1–33 (2017)
Grigoreanu, V., Burnett, M., Wiedenbeck, S., Cao, J., Rector, K., Kwan, I.: End-user debugging strategies: a sensemaking perspective. ACM Trans. Comput. Human Interact. (TOCHI) 19(1), 1–28 (2012)
Gunawardana, A., Shani, G., Yogev, S.: Evaluating recommender systems. In: Recommender systems handbook, pp. 547–601. Springer US, New York, NY (2012)
Huang, J., Cakmak, M.: Supporting mental model accuracy in trigger-action programming. In: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing - UbiComp 2015, pp. 215–225. ACM Press, Osaka, Japan (2015). https://doi.org/10.1145/2750858.2805830
Jain, M., Kumar, P., Kota, R., Patel, S.N.: Evaluating and informing the design of chatbots. In: Proceedings of the 2018 Designing Interactive Systems Conference, pp. 895–906 (2018)
Knijnenburg, B.P., Willemsen, M.C., Gantner, Z., Soncu, H., Newell, C.: Explaining the user experience of recommender systems. User Model. User-Adap. Inter. 22, 441–504 (2012)
Ko, A., Myers, B.: Designing the whyline: a debugging interface for asking questions about program behavior. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 151–158. ACM (2004)
Li, C., Chan, E., Denny, P., Luxton-Reilly, A., Tempero, E.: Towards a framework for teaching debugging. In: Proceedings of the Twenty-First Australasian Computing Education Conference on - ACE 2019 (2019). https://doi.org/10.1145/3286960.3286970
Liao, Q.V., Gruen, D., Miller, S.: Questioning the AI: informing design practices for explainable AI user experiences. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1–15 (2020)
Manca, M., Paternò, F., Santoro, C.: Remote monitoring of end-user created automations in field trials. J. Ambient Intell. Human. Comput. 13(12), 5669–5697 (2021). https://doi.org/10.1007/s12652-021-03239-0
Manca, M., Paternò, F., Santoro, C., Corcella, L.: Supporting end-user debugging of trigger-action rules for IoT applications. Int. J. Hum. Comput. Stud. 123, 56–69 (2019)
Manfredi, R., Andrao, M., Greco, F., Desolda, G., Treccani, B. Zancanaro, M.: Toward a better understanding of end-user debugging strategies: a pilot study. In Proceedings of the 3rd international workshop on empowering people in dealing with Internet of Things ecosystems co-located with AVI 2022, Frascati, Rome, Italy, June 06, 2022. (CEUR Workshop Proceedings, vol. 3172). CEUR-WS.org, pp. 31–35 (2022). https://ceur-ws.org/Vol-3172/short6.pdf
Masui, T., Nakayama, K.: Repeat and predict—two keys to efficient text editing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 118–130 (1994)
Mattioli, A., Paternò, F.: A visual environment for end-user creation of IoT customization rules with recommendation support. In: Proceedings of the International Conference on Advanced Visual Interfaces, pp. 1–5 (2020)
Mattioli, A., Paternò, F.: Recommendations for creating trigger-action rules in a block-based environment. Behav. Inf. Technol. 40(10), 1024–1034 (2021)
Mi, X., Qian, F., Zhang, Y., Wang, X.F.: An empirical characterization of IFTTT: ecosystem, usage, and performance. In: Proceedings of the 2017 Internet Measurement Conference, pp. 398–404 (2017)
Paci, F., Bianchin, D., Quintarelli, E., Zannone, N.: IFTTT privacy checker. In: Saracino, A., Mori, P. (eds.) Emerging Technologies for Authorization and Authentication. LNCS, vol. 12515, pp. 90–107. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-64455-0_6
Paternò, F., Santoro, C.: End-user development for personalizing applications, things, and robots. Int. J. Hum Comput Stud. 131, 120–130 (2019)
Pianesi, F., Varzi, A.C.: Events and event talk: an introduction. In: Speaking of Events, pp. 3–47. Oxford University Press, New York, NY (2000)
Resnick, M., et al.: Scratch: programming for all. Commun. ACM 52(11), 60–67 (2009). https://doi.org/10.1145/1592761.1592779
Ruvini, J.-D., Dony, C.: Learning users’ habits to automate repetitive tasks. In: Your Wish is My Command, pp. 271-XIV. Morgan Kaufmann (2001) https://doi.org/10.1016/B978-155860688-3/50015-4
Saeidi, M., Calvert, M., Au, A.W., Sarma, A., Bobba, R.B.: If this context then that concern: exploring users’ concerns with IFTTT applets. In: Proceedings on Privacy Enhancing Technologies, vol. 2022(1), pp. 166–186 (2021)
Salovaara, A., Bellucci, A., Vianello, A., Jacucci, G.: Programmable smart home toolkits should better address households’ social needs. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–14 (2021)
Seiger, R., Kühn, R., Korzetz, M., Aßmann, U.: HoloFlows: modelling of processes for the Internet of Things in mixed reality. Softw. Syst. Model. 20(5), 1465–1489 (2021). https://doi.org/10.1007/s10270-020-00859-6
Soares, D., Dias, J.P., Restivo, A., Ferreira, H.S.: Programming IoT-spaces: a user-survey on home automation rules. In: Paszynski, M., Kranzlmüller, D., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds.) Computational Science – ICCS 2021. LNCS, vol. 12745, pp. 512–525. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77970-2_39
Srinivasan, V., Koehler, C., Jin, H.: RuleSelector: Selecting conditional action rules from user behavior patterns. Proc. ACM Interact. Mobile. Wearabl. Ubiquit. Technol. 2(1), 1–34 (2018)
Stefanidi, E., et al.: MagiPlay: an augmented reality serious game allowing children to program intelligent environments. Trans. Comput. Sci. XXXVII: Spec. Issue Comput. Graph. 144–169 (2020)
Surbatovich, M., Aljuraidan, J., Bauer, L., Das, A., Jia, L.: Some recipes can do more than spoil your appetite: analysing the security and privacy risks of IFTTT recipes. In: Proceedings of the 26th International Conference on World Wide Web (WWW 2017), pp. 1501–1510 (2017)
Ur, B., McManus, E., Yong Ho, M.P., Littman, M.L.: Practical trigger-action programming in the smart home. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 803–812 (2014)
Ur, B., et al.: Trigger-action programming in the wild: An analysis of 200,000 IFTTT recipes. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 3227–3231 (2016)
Valtolina, S., Barricelli, B.R., Di Gaetano, S.: Communicability of traditional interfaces VS chatbots in healthcare and smart home domains. Behav. Inf. Technol. 39(1), 108–132 (2020)
Wang, Q., Datta, P., Yang, W., Liu, S., Bates, A., Gunter, C.A.: Charting the attack surface of trigger-action IoT platforms. In: Proceedings of the ACM Conference on Computer and Communications Security, pp. 1439–1453 (2019)
Yang, F., Kalloori, S., Chalumattu, R., Gross, M.: Personalized information retrieval for touristic attractions in augmented reality. In: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pp. 1613–1616 (2022)
Yu, H., Hua, J., Julien, C.: Analysis of IFTTT recipes to study how humans use internet-of-things (iot) devices. In: Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems, pp. 537–541 (2021)
Yusuf, I.N.B., Jamal, D.B.A., Jiang, L., Lo, D.: RecipeGen++: an automated trigger action programs generator. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 1672–1676, November 2022
Zancanaro, M., Gallitto, G., Dina, Y., Treccani, B.: Improving mental models in IoT end-user development. Human-centric Comput. Inf. Sci. 2, 48 (2022)
Zheng, S., Apthorpe, N., Chetty, M., Feamster, N.: User perceptions of smart home IoT privacy. In: Proceedings of the ACM on Human-Computer Interaction, vol. 2, CSCW, pp. 1–20 (2018)
Acknowledgements
This work has been supported by the PRIN 2017 “EMPATHY: Empowering People in Dealing with Internet of Things Ecosystems”, https://www.empathy-project.eu/. Balducci acknowledges the support by the REFIN grant, POR Puglia FESR FSE 2014–2020 “Gestione di oggetti intelligenti per migliorare le esperienze di visita di siti di interesse culturale”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Andrao, M. et al. (2023). Understanding Concepts, Methods and Tools for End-User Control of Automations in Ecosystems of Smart Objects and Services. In: Spano, L.D., Schmidt, A., Santoro, C., Stumpf, S. (eds) End-User Development. IS-EUD 2023. Lecture Notes in Computer Science, vol 13917. Springer, Cham. https://doi.org/10.1007/978-3-031-34433-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-031-34433-6_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34432-9
Online ISBN: 978-3-031-34433-6
eBook Packages: Computer ScienceComputer Science (R0)