research-article

Natural Contextual Reasoning for End Users

Authors:

Georgios Metaxas,

Panos MarkopoulosAuthors Info & Claims

ACM Transactions on Computer-Human Interaction (TOCHI), Volume 24, Issue 2

Article No.: 13, Pages 1 - 36

https://doi.org/10.1145/3057860

Published: 27 April 2017 Publication History

Abstract

The realization and deployment of the Internet of Things require providing to non-programmers some level of programmatic control for tailoring system behaviour to their context and needs. We introduce a simple context-range semantics (CRS) and a context-range editor (CoRE) that support end users formulate and understand logical expressions regarding context. The editor builds on two key ideas (a) contextual information is used to evaluate and minimize logical expressions; (b) logical expressions are presented in a disjunctive normal form (DNF) thus applying a principle established in mental model theory. User tests reveal situations in which the theory regarding the intuitiveness of the DNF needs to be extended with a new element: Logical terms are easier to comprehend and formulate when grouped according to their semantic affinity. We report two experiments that demonstrate the intuitiveness of this approach and how it improves performance of non-programmers in specifying context sensitive system behaviour.

References

[1]

G. Abowd, Russell Beale, A. Dix, and J. Finlay. 1996. Human-Computer Interaction. Graduate computer science course offered at Georgia tech, winter.

[2]

Gregory D. Abowd and Elizabeth D. Mynatt. 2000. Charting past, present, and future research in ubiquitous computing. ACM Trans. Comput.-Hum. Interact. TOCHI 7, 1 (2000), 29--58.

Digital Library

[3]

Saeed Aghaee and Cesare Pautasso. 2014. End-user development of mashups with naturalmash. J. Vis. Lang. Comput. 25, 4 (2014), 414--432.

Digital Library

[4]

Carmelo Ardito et al. 2014. User-driven visual composition of service-based interactive spaces. J. Vis. Lang. Comput. 25, 4 (August 2014), 278--296.

[5]

Louise Barkhuus and Anind Dey. 2003. Is context-aware computing taking control away from the user? Three levels of interactivity examined. In Proceedings of the IEEE Ubiquitous Computing (UbiComp’03). Springer, 149--156.

[6]

Victoria Bellotti and Keith Edwards. 2001. Intelligibility and accountability: Human considerations in context-aware systems. Hum.--Comput. Interact. 16, 2--4 (2001), 193--212.

[7]

Alan F. Blackwell. 2002. First steps in programming: A rationale for attention investment models. In Proceedings of the IEEE 2002 Symposia on Human Centric Computing Languages and Environments. IEEE, 2--10.

[8]

Alan F. Blackwell and Rob Hague. 2001. AutoHAN: An architecture for programming the home. In Proceedings of the IEEE Symposia on Human-Centric Computing Languages and Environments. IEEE, 150--157.

[9]

Ruth M. J. Byrne and P. N. Johnson-Laird. 1992. The spontaneous use of propositional connectives. Q. J. Exp. Psychol. 45, 1 (1992), 89--110.

[10]

Keith Cheverst, Hee Eon Byun, Dan Fitton, Corina Sas, Chris Kray, and Nicolas Villar. 2005. Exploring issues of user model transparency and proactive behaviour in an office environment control system. User Model. User-Adapt. Interact. 15, 3--4 (2005), 235--273.

Digital Library

[11]

Eric Christensen, Francisco Curbera, Greg Meredith, and Sanjiva Weerawarana. 2009. Web services description language (WSDL) 1.0. (September 2009). Retrieved January 15, 2017 from http://xml.coverpages.org/wsdl20000929.html.

[12]

Philipp Cimiano, Andreas Hotho, and Steffen Staab. 2005. Learning concept hierarchies from text corpora using formal concept analysis. J. Artif. Intell. Res. 24 (2005), 305--339.

Digital Library

[13]

Jose Danado and Fabio Paternò. 2014. Puzzle: A mobile application development environment using a jigsaw metaphor. J. Vis. Lang. Comput. 25, 4 (2014), 297--315.

Digital Library

[14]

Anind K. Dey, Raffay Hamid, Chris Beckmann, Ian Li, and Daniel Hsu. 2004. A CAPpella: Programming by demonstration of context-aware applications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’04). ACM, New York, NY, 33--40.

Digital Library

[15]

Anind K. Dey, Timothy Sohn, Sara Streng, and Justin Kodama. 2006. iCAP: Interactive prototyping of context-aware applications. In Proceedings of the Pervasive Computing, Kenneth P. Fishkin, Bernt Schiele, Paddy Nixon, and Aaron Quigley (Eds.), Lecture Notes in Computer Science. Vol. 3968. Springer, Berlin, 254--271.

Digital Library

[16]

W. Keith Edwards and Rebecca E. Grinter. 2001. At home with ubiquitous computing: Seven challenges. In Proceedings of the Ubiquitous Computing (Ubicomp’01), Gregory D. Abowd, Barry Brumitt, and Steven Shafer (Eds.). Lecture Notes in Computer Science. Springer, Berlin. 256--272.

[17]

W. Keith Edwards, Mark W. Newman, Jana Sedivy, Trevor Smith, and Shahram Izadi. 2002. Challenge: Recombinant computing and the speakeasy approach. In Proceedings of the 8th Annual International Conference on Mobile Computing and Networking. ACM, 279--286.

Digital Library

[18]

Roger Evans, Paul Piwek, Lynne Cahill, and Neil Tipper. 2008. Natural language processing in C LIME, a multilingual legal advisory system. Nat. Lang. Eng. 14, 1 (2008), 101--132.

Digital Library

[19]

Giuseppe Ghiani, Fabio Paternò, Lucio Davide Spano, and Giuliano Pintori. 2016. An environment for end-user development of web mashups. Int. J. Hum.-Comput. Stud. 87 (2016), 38--64.

Digital Library

[20]

Hadley. 2009. Web application description language. Retrieved January 15, 2017 from https://www.w3.org/ Submission/wadl/.

[21]

Jan Humble et al. 2003. “Playing with the bits” user-configuration of ubiquitous domestic environments. In Proceedings of the Ubiquitous Computing (UbiComp’03). Springer, 256--263.

[22]

Philip N. Johnson-Laird. 1995. Mental models, deductive reasoning, and the brain. Cogn. Neurosci. 65 (1995), 999--1008.

[23]

Philip N. Johnson-Laird. 1983. Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness. Harvard University Press.

Digital Library

[24]

Philip Nicholas and Johnson-Laird. 2006. How We Reason. Oxford University Press.

[25]

M. Cameron Jones and Elizabeth F. Churchill. 2009. Conversations in developer communities: A preliminary analysis of the Yahoo&excl; Pipes community. In Proceedings of the 4th International Conference on Communities and Technologies. ACM, 195--204.

Digital Library

[26]

Steve Jones, Shona McInnes, and Mark S. Staveley. 1999. A graphical user interface for boolean query specification. Int. J. Digit. Libr. 2, 2--3 (1999), 207--223.

Digital Library

[27]

Vassilis-Javed Khan and Panos Markopoulos. 2009. Busy families’ awareness needs. Int. J. Hum.-Comput. Stud. 67, 2 (2009), 139--153.

Digital Library

[28]

Beata Beigman Klebanov. 2006. Measuring semantic relatedness using people and wordnet. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers (NAACL-Short’06). Association for Computational Linguistics, Stroudsburg, PA, 13--16.

[29]

Brian Y. Lim and Anind K. Dey. 2009. Assessing demand for intelligibility in context-aware applications. In Proceedings of the 11th International Conference on Ubiquitous Computing (UbiComp’09). ACM, New York, NY, 195--204.

[30]

Brian Y. Lim, Anind K. Dey, and Daniel Avrahami. 2009. Why and why not explanations improve the intelligibility of context-aware intelligent systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2119--2128.

Digital Library

[31]

Panos Markopoulos. 2016. Ambient intelligence: Vision, research, and life. J. Ambient Intell. Smart Environ. 8, 5 (January 2016), 491--499.

[32]

Panos Markopoulos. 2005. Designing ubiquitous computer-human interaction: The case of the connected family. In Future Interaction Design. Springer, 125--149.

[33]

Panos Markopoulos, Irene Mavrommati, and Achilles Kameas. 2004. End-user configuration of ambient intelligence environments: Feasibility from a user perspective. In Ambient Intelligence. Springer, Berlin, 243--254.

[34]

Irene Mavrommati, Achilles Kameas, and Panos Markopoulos. 2004. An editing tool that manages device associations in an in-home environment. Pers. Ubiquitous Comput. 8, 3--4 (2004), 255--263.

Digital Library

[35]

Nikolay Mehandjiev, Alistair Sutcliffe, and Darren Lee. 2006. Organizational view of end-user development. In Human-Computer Interaction Series. Henry Lieberman, Fabio Paternò, and Volker Wulf (Eds.), Springer, The Netherlands, 371--399.

[36]

G. Metaxas. 2010. End user programming of awareness systems: Addressing cognitive and social challenges for interaction with aware environments. PhD Thesis, Eindhoven University of Technology.

[37]

Georgios Metaxas, Panos Markopoulos, and Emile Aarts. 2009. Amelie: A recombinant computing framework for ambient awareness. In Proceedings of the Ambient Intelligence. Manfred Tscheligi, et al. (Eds.), Lecture Notes in Computer Science, Springer, Berlin, 88--100.

Digital Library

[38]

Georgios Metaxas, Barbaros Metin, Jutta Schneider, Panos Markopoulos, and Boris de Ruyter. 2007. Daily activities diarist: Supporting aging in place with semantically enriched narratives. In Proceedings of the Human-Computer Interaction (INTERACT’07). Cécilia Baranauskas, Philippe Palanque, Julio Abascal, and Simone Diniz Junqueira Barbosa (Eds.), Lecture Notes in Computer Science, Springer, Berlin, 390--403.

[39]

Carman Neustaedter and Saul Greenberg. 2003. The design of a context-aware home media space for balancing privacy and awareness. In Proceedings of the Ubiquitous Computing (UbiComp’03). Springer, 297--314.

[40]

Thomas C. Ormerod. 2000. Mechanisms and strategies for rephrasing. Deductive Reason. Strateg. W. Schaeken, G. de Vooght, Vandierendonckt, and G. d’ Ydewalle (2000), Lawrence Erlbaum, 131--152.

[41]

John F. Pane and Brad A. Myers. 2000. Tabular and textual methods for selecting objects from a group. In Proceedings of the IEEE International Symposium on Visual Languages. IEEE, 157--164.

[42]

John F. Pane, Brad A. Myers, et al. 2001. Studying the language and structure in non-programmers’ solutions to programming problems. Int. J. Hum.-Comput. Stud. 54, 2 (2001), 237--264.

Digital Library

[43]

Siddharth Patwardhan and Ellen Riloff. 2007. Effective information extraction with semantic affinity patterns and relevant regions. In Proceedings of the EMNLP-CoNLL. 717--727.

[44]

Ted Pedersen, Siddharth Patwardhan, and Jason Michelizzi. 2004. WordNet: Similarity: Measuring the relatedness of concepts. In Proceedings of the Demonstration Papers at HLT-NAACL’04. Association for Computational Linguistics, 38--41.

[45]

Charith Perera, Arkady Zaslavsky, Peter Christen, and Dimitrios Georgakopoulos. 2014. Context aware computing for the internet of things: A survey. IEEE Commun. Surv. Tutor. 16, 1 (2014), 414--454.

[46]

Richard Power, Donia Scott, and Roger Evans. 1998. What you see is what you meant: Direct knowledge editing with natural language feedback. In Proceedings of the ECAI. 677--681.

[47]

Christopher Scaffidi, Mary Shaw, and Brad Myers. 2005. An approach for categorizing end user programmers to guide software engineering research. In Proceedings of the 1st Workshop on End-User Software Engineering (WEUSE’05). ACM, New York, NY, 1--5.

Digital Library

[48]

ExistAlbrecht Schmidt. 2010. Ubiquitous computing: Are we there yet? Computer 43, 2 (2010), 95--97.

Digital Library

[49]

Vladimir M. Sloutsky and Yevgeniya Goldvarg. 2004. Mental representation of logical connectives. Q. J. Exp. Psychol. Sect. A 57, 4 (2004), 636--665.

[50]

Daniel Tetteroo et al. 2015. Lessons learnt from deploying an end-user development platform for physical rehabilitation. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 4133--4142.

Digital Library

[51]

Khai N. Truong, Elaine M. Huang, and Gregory D. Abowd. 2004. CAMP: A magnetic poetry interface for end-user programming of capture applications for the home. In Proceedings of the Ubiquitous Computing (UbiComp’04). Springer, 143--160.

[52]

Jo Vermeulen, Geert Vanderhulst, Kris Luyten, and Karin Coninx. 2010. PervasiveCrystal: Asking and answering why and why not questions about pervasive computing applications. In Proceedings of the 6th International Conference on Intelligent Environments (IE’10). IEEE, 271--276.

Digital Library

[53]

Jin Yu, Boualem Benatallah, Fabio Casati, and Florian Daniel. 2008. Understanding mashup development. IEEE Internet Comput. 12, 5 (2008), 44--52.

Digital Library

Cited By

Ponce VAbdulrazak B(2022)Context-Aware End-User Development ReviewApplied Sciences10.3390/app1201047912:1(479)Online publication date: 4-Jan-2022
https://doi.org/10.3390/app12010479
Abdollahinami SDucceschi LZancanaro MBottoni PPanizzi E(2022)End-user Development and Closed-Reading: an Initial InvestigationProceedings of the 2022 International Conference on Advanced Visual Interfaces10.1145/3531073.3531128(1-5)Online publication date: 6-Jun-2022
https://dl.acm.org/doi/10.1145/3531073.3531128
Manca MPaternò FSantoro C(2021)Remote monitoring of end-user created automations in field trialsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-021-03239-013:12(5669-5697)Online publication date: 12-Apr-2021
https://doi.org/10.1007/s12652-021-03239-0
Show More Cited By

Index Terms

Natural Contextual Reasoning for End Users
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Natural language interfaces
  2. Ubiquitous and mobile computing

Recommendations

Mobile Cloud Contextual Awareness with the Cloud Personal Assistant
FICLOUD '14: Proceedings of the 2014 International Conference on Future Internet of Things and Cloud

This paper presents our efforts to bridge the gap between mobile context awareness, and mobile cloud services, using the Cloud Personal Assistant (CPA). The CPA is a part of the Context Aware Mobile Cloud Services (CAMCS) middleware, which we continue ...
Context-aware content filtering & presentation for pervasive & mobile information systems
Ambi-Sys '08: Proceedings of the 1st international conference on Ambient media and systems

What constitutes relevant information to an individual may vary widely under different contexts. However, previous work on pervasive information systems has mostly focused on context-aware delivery of application-specific information. Such systems are ...
Context Reasoning Using Contextual Graph
CITWORKSHOPS '08: Proceedings of the 2008 IEEE 8th International Conference on Computer and Information Technology Workshops

Nowadays the combination of virtually free computation and ubiquitous environment has formed the new domain of pervasive computing. Context reasoning part in the context awareness seems to become one of the most important goals of that computing trend. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 24, Issue 2

Special Issue EUD for IoT

April 2017

284 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/3077620

Editor:
Ken Hinckley
Microsoft Research

Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 April 2017

Accepted: 01 January 2017

Revised: 01 January 2017

Received: 01 April 2016

Published in TOCHI Volume 24, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
390
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ponce VAbdulrazak B(2022)Context-Aware End-User Development ReviewApplied Sciences10.3390/app1201047912:1(479)Online publication date: 4-Jan-2022
https://doi.org/10.3390/app12010479
Abdollahinami SDucceschi LZancanaro MBottoni PPanizzi E(2022)End-user Development and Closed-Reading: an Initial InvestigationProceedings of the 2022 International Conference on Advanced Visual Interfaces10.1145/3531073.3531128(1-5)Online publication date: 6-Jun-2022
https://dl.acm.org/doi/10.1145/3531073.3531128
Manca MPaternò FSantoro C(2021)Remote monitoring of end-user created automations in field trialsJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-021-03239-013:12(5669-5697)Online publication date: 12-Apr-2021
https://doi.org/10.1007/s12652-021-03239-0
Desolda GGreco FGuarnieri FMariz NZancanaro M(2021)SENSATION: An Authoring Tool to Support Event–State Paradigm in End-User DevelopmentHuman-Computer Interaction – INTERACT 202110.1007/978-3-030-85616-8_22(373-382)Online publication date: 26-Aug-2021
https://doi.org/10.1007/978-3-030-85616-8_22
Abbas TKhan VMarkopoulos P(2020)Investigating the Crowd’s Creativity for Creating On-Demand IoT ScenariosInternational Journal of Human–Computer Interaction10.1080/10447318.2019.1709331(1-28)Online publication date: 16-Jan-2020
https://doi.org/10.1080/10447318.2019.1709331
Ardito CDesolda GLanzilotti RMalizia AMatera MBuono PPiccinno A(2020)User-defined semantics for the design of IoT systems enabling smart interactive experiencesPersonal and Ubiquitous Computing10.1007/s00779-020-01457-524:6(781-796)Online publication date: 26-Sep-2020
https://dl.acm.org/doi/10.1007/s00779-020-01457-5
Leonardi NManca MPaternò FSantoro CBrewster SFitzpatrick GCox AKostakos V(2019)Trigger-Action Programming for Personalising Humanoid Robot BehaviourProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300675(1-13)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300675
Goumopoulos CMavrommati I(2019)A Framework for Pervasive Computing Applications based on Smart Objects and End User DevelopmentJournal of Systems and Software10.1016/j.jss.2019.110496(110496)Online publication date: Dec-2019
https://doi.org/10.1016/j.jss.2019.110496
Paternò FSantoro C(2019)End-User Development for Personalizing Applications, Things, and RobotsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2019.06.002Online publication date: Jun-2019
https://doi.org/10.1016/j.ijhcs.2019.06.002
Bellucci AVianello AFlorack YMicallef LJacucci G(2019)Augmenting objects at home through programmable sensor tokens: A design journeyInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2018.09.002122(211-231)Online publication date: Feb-2019
https://doi.org/10.1016/j.ijhcs.2018.09.002
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents