research-article

Ontologies of Action and Object in Home Environment towards Injury Prevention

Authors:

Satoshi Nishimura,

Ken FukudaAuthors Info & Claims

IJCKG '21: Proceedings of the 10th International Joint Conference on Knowledge Graphs

Pages 126 - 130

https://doi.org/10.1145/3502223.3502239

Published: 24 January 2022 Publication History

Abstract

It is one of the critical applications for human-centric artificial intelligence that surveys the risky situation and inferring ways to prevent it by storing the situation information from surveillance cameras. Recognition of human activities in daily situations is an emerging topic in the computer vision domain. Significantly, the context information, such as objects involved in activities and the relationships between the objects and the activities, are attractive to improve the accuracy of the activity recognition task. However, the existing labels for actions and objects are not well considered for describing daily activities. This short research paper provides the ontologies of actions and objects in the home environment, so-called Primitive Action ontology, and Home Object ontology. The Primitive Action ontology contains a minimal set of primitive actions designed to abstract actions and discards objects and methods. The Home Object ontology has object types and properties such as affordance and attributes to describe daily situations. The properties represent both normal and abnormal effects, including intentional function and incidents in the home environment. We also discuss the prospect of using these ontologies as the conclusion of this paper.

References

[1]

Daniel Beßler, Robert Porzel, Mihai Pomarlan, Michael Beetz, Rainer Malaka, and John A. Bateman. 2020. A Formal Model of Affordances for Flexible Robotic Task Execution. In ECAI 2020 - 24th European Conference on Artificial Intelligence, 29 August-8 September 2020, Santiago de Compostela, Spain, August 29 - September 8, 2020 - Including 10th Conference on Prestigious Applications of Artificial Intelligence (PAIS 2020)(Frontiers in Artificial Intelligence and Applications, Vol. 325), Giuseppe De Giacomo, Alejandro Catalá, Bistra Dilkina, Michela Milano, Senén Barro, Alberto Bugarín, and Jérôme Lang (Eds.). IOS Press, 2425–2432. https://doi.org/10.3233/FAIA200374

[2]

Stefano Borgo, Maarten Franssen, Paweł Garbacz, Yoshinobu Kitamura, Riichiro Mizoguchi, and Pieter E. Vermaas. 2014. Technical Artifacts: An Integrated Perspective. Appl. Ontol. 9, 3–4 (jul 2014), 217–235.

[3]

Ginevra Castellano. 2020. What Kind of Human-Centric Robotics Do We Need? Investigations from Human-Robot Interactions in Socially Assistive Scenarios. In Proceedings of the 8th International Conference on Human-Agent Interaction (Virtual Event, USA) (HAI ’20). Association for Computing Machinery, New York, NY, USA, 1–2. https://doi.org/10.1145/3406499.3422313

Digital Library

[4]

Michael Compton, Payam Barnaghi, Luis Bermudez, Raúl García-Castro, Oscar Corcho, Simon Cox, John Graybeal, Manfred Hauswirth, Cory Henson, Arthur Herzog, Vincent Huang, Krzysztof Janowicz, W. David Kelsey, Danh Le Phuoc, Laurent Lefort, Myriam Leggieri, Holger Neuhaus, Andriy Nikolov, Kevin Page, Alexandre Passant, Amit Sheth, and Kerry Taylor. 2012. The SSN ontology of the W3C semantic sensor network incubator group. Journal of Web Semantics 17 (2012), 25–32. https://doi.org/10.1016/j.websem.2012.05.003

[5]

Shusaku Egami, Satoshi Nishimura, and Ken Fukuda. 2021. VirtualHome2KG: Constructing and Augmenting Knowledge Graphs of Daily Activities Using Virtual Space. In Proceedings of the ISWC 2021 Posters, Demos and Industry Tracks: From Novel Ideas to Industrial Practice co-located with 20th International Semantic Web Conference (ISWC 2021), Virtual Conference, October 24-28, 2021(CEUR Workshop Proceedings, Vol. 2980), Oshani Seneviratne, Catia Pesquita, Juan Sequeda, and Lorena Etcheverry (Eds.). CEUR-WS.org. http://ceur-ws.org/Vol-2980/paper381.pdf

[6]

Maria Fazio, Antonio Celesti, Fermín Galán Márquez, Alex Glikson, and Massimo Villari. 2015. Exploiting the FIWARE cloud platform to develop a remote patient monitoring system. In 2015 IEEE Symposium on Computers and Communication (ISCC). 264–270. https://doi.org/10.1109/ISCC.2015.7405526

Digital Library

[7]

Sten Hanke, Christopher Mayer, Oliver Hoeftberger, Henriette Boos, Reiner Wichert, Mohammed-R. Tazari, Peter Wolf, and Francesco Furfari. 2011. universAAL – An Open and Consolidated AAL Platform. Springer Berlin Heidelberg, Berlin, Heidelberg, 127–140. https://doi.org/10.1007/978-3-642-18167-2_10

[8]

Bin Huang, Siao Tang, Guangyao Shen, Guohao Li, Xin Wang, and Wenwu Zhu. 2020. Commonsense Learning: An Indispensable Path towards Human-Centric Multimedia. In Proceedings of the 1st International Workshop on Human-Centric Multimedia Analysis (Seattle, WA, USA) (HuMA’20). Association for Computing Machinery, New York, NY, USA, 91–100. https://doi.org/10.1145/3422852.3423484

Digital Library

[9]

Jingwei Ji, Ranjay Krishna, Li Fei-Fei, and Juan Carlos Niebles. 2020. Action Genome: Actions As Compositions of Spatio-Temporal Scene Graphs. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10233–10244. https://doi.org/10.1109/CVPR42600.2020.01025

[10]

Kristiina JOKINEN. 2020. Situated Agents, Personality and Human-Centric AI. Proceedings of the Annual Conference of JSAI2020 JSAI2020 (2020), 2G4ES402–2G4ES402. https://doi.org/10.11517/pjsai.JSAI2020.0_2G4ES402

[11]

H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre. 2011. HMDB: A large video database for human motion recognition. In 2011 International Conference on Computer Vision. 2556–2563. https://doi.org/10.1109/ICCV.2011.6126543

Digital Library

[12]

Xueting Li, Sifei Liu, Kihwan Kim, Xiaolong Wang, Ming Hsuan Yang, and Jan Kautz. 2019. Putting humans in a scene: Learning affordance in 3D indoor environments. In Proceedings - 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2019(Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition). IEEE Computer Society, United States, 12360–12368. https://doi.org/10.1109/CVPR.2019.01265

[13]

Fernando Martínez-Santiago, M Rosario García-Viedma, John A Williams, Luke T Slater, and Georgios V Gkoutos. 2020. Aging neuro-behavior ontology. Applied Ontology 15, 2 (2020), 219–239.

Digital Library

[14]

Georgios Meditskos and Ioannis Kompatsiaris. 2017. iKnow: Ontology-driven situational awareness for the recognition of activities of daily living. Pervasive and Mobile Computing 40 (2017), 17–41. https://doi.org/10.1016/j.pmcj.2017.05.003

Digital Library

[15]

National Institute of Advanced Industrial Science and Technology. 2020. Elderly Behavior Library. http://www.behavior-library-meti.com/behaviorLib/homes/about

[16]

World Health Organization. 2001. International Classification of Functioning, Disability and Health (ICF). https://www.who.int/standards/classifications/international-classification-of-functioning-disability-and-health

[17]

Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. 2018. VirtualHome: Simulating Household Activities Via Programs. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8494–8502. https://doi.org/10.1109/CVPR.2018.00886

[18]

Gunnar A. Sigurdsson, Olga Russakovsky, and Abhinav Gupta. 2017. What Actions are Needed for Understanding Human Actions in Videos?. In 2017 IEEE International Conference on Computer Vision (ICCV). 2156–2165. https://doi.org/10.1109/ICCV.2017.235

[19]

Gunnar A. Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta. 2016. Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding. In Computer Vision – ECCV 2016, Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, Cham, 510–526.

[20]

Khurram Soomro, Amir Roshan Zamir, Mubarak Shah, Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. [n.d.]. UCF101: A dataset of 101 human actions classes from videos in the wild. CoRR ([n. d.]), 2012.

[21]

Shinya Tarumi, Kouji Kozaki, Yoshinobu Kitamura, and Riichiro Mizoguchi. 2010. A Consideration of Quality-Attribute-Property for Interoperability of QualityData (In Japanese). Transactions of the Japanese Society for Artificial Intelligence 25, 5(2010), 579–592. https://doi.org/10.1527/tjsai.25.579

[22]

Alexandros Vassiliades, Nick Bassiliades, Filippos Gouidis, and Theodore Patkos. 2020. A Knowledge Retrieval Framework for Household Objects and Actions with External Knowledge. In Semantic Systems. In the Era of Knowledge Graphs, Eva Blomqvist, Paul Groth, Victor de Boer, Tassilo Pellegrini, Mehwish Alam, Tobias Käfer, Peter Kieseberg, Sabrina Kirrane, Albert Meroño-Peñuela, and Harshvardhan J. Pandit (Eds.). Springer International Publishing, Cham, 36–52.

[23]

Chao-Yuan Wu, Christoph Feichtenhofer, Haoqi Fan, Kaiming He, Philipp Krähenbühl, and Ross Girshick. 2019. Long-Term Feature Banks for Detailed Video Understanding. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 284–293. https://doi.org/10.1109/CVPR.2019.00037

[24]

Natsuki Yamanobe, Weiwei Wan, Ixchel G. Ramirez-Alpizar, Damien Petit, Tokuo Tsuji, Shuichi Akizuki, Manabu Hashimoto, Kazuyuki Nagata, and Kensuke Harada. 2017. A brief review of affordance in robotic manipulation research. Advanced Robotics 31, 19-20 (2017), 1086–1101. https://doi.org/10.1080/01691864.2017.1394912 arXiv:https://doi.org/10.1080/01691864.2017.1394912

Cited By

Htun SEgami SFukuda K(2023)A Survey and Comparison of Activities of Daily Living Datasets in Real-life and Virtual Spaces2023 IEEE/SICE International Symposium on System Integration (SII)10.1109/SII55687.2023.10039226(1-7)Online publication date: 17-Jan-2023
https://doi.org/10.1109/SII55687.2023.10039226
Egami SUgai TOono MKitamura KFukuda K(2023)Synthesizing Event-Centric Knowledge Graphs of Daily Activities Using Virtual SpaceIEEE Access10.1109/ACCESS.2023.325380711(23857-23873)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3253807
Egami SOono MOtsuki MUgai TFukuda K(2023)Analysis of Annotation Quality of Human Activities Using Knowledge GraphsHCI International 2023 Posters10.1007/978-3-031-36001-5_62(483-489)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-36001-5_62

Index Terms

Ontologies of Action and Object in Home Environment towards Injury Prevention

Index terms have been assigned to the content through auto-classification.

Recommendations

A New Way to Build Multifacetted Ontologies for Elderly Care
PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

The care of the elderly is complex, with multiple agencies and individuals involved. We propose a new way of developing an ontology to reflect these aspects within a real-time home monitoring system so that it captures real-life circumstances and ...
Semantic enrichment for medical ontologies

The Unified Medical Language System (UMLS) contains two separate but interconnected knowledge structures, the Semantic Network (upper level) and the Metathesaurus (lower level). In this paper, we have attempted to work out better how the use of such a ...
Haptic characteristics of some activities of daily living
HAPTIC '10: Proceedings of the 2010 IEEE Haptics Symposium

Activities of daily living (ADLs) are of interest in rehabilitation, independent living for the elderly and infirm, and to the designers of everyday objects. This paper reports measurements of forces and torques at the interaction point between users ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

IJCKG '21: Proceedings of the 10th International Joint Conference on Knowledge Graphs

December 2021

204 pages

ISBN:9781450395656

DOI:10.1145/3502223

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 January 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

New Energy and Industrial Technology Development Organization

Conference

IJCKG'21

IJCKG'21: The 10th International Joint Conference on Knowledge Graphs

December 6 - 8, 2021

Virtual Event, Thailand

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
105
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)2

Reflects downloads up to 06 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Htun SEgami SFukuda K(2023)A Survey and Comparison of Activities of Daily Living Datasets in Real-life and Virtual Spaces2023 IEEE/SICE International Symposium on System Integration (SII)10.1109/SII55687.2023.10039226(1-7)Online publication date: 17-Jan-2023
https://doi.org/10.1109/SII55687.2023.10039226
Egami SUgai TOono MKitamura KFukuda K(2023)Synthesizing Event-Centric Knowledge Graphs of Daily Activities Using Virtual SpaceIEEE Access10.1109/ACCESS.2023.325380711(23857-23873)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3253807
Egami SOono MOtsuki MUgai TFukuda K(2023)Analysis of Annotation Quality of Human Activities Using Knowledge GraphsHCI International 2023 Posters10.1007/978-3-031-36001-5_62(483-489)Online publication date: 9-Jul-2023
https://doi.org/10.1007/978-3-031-36001-5_62

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents