Abstract
This work is focused on advancing automatic scene analysis and ambient assisted living systems to support individuals requiring special care, such as the elderly or those visually impaired. The study explores the most effective techniques in Video Captioning and Object Detection, proposing a Deep Learning pipeline for Risks Assessment in home environments. Key elements include the integration of SwinBERT for Video Captioning and YOLOv7 for Object Recognition. Additionally, the effectiveness and limitations of the Risks Assessment pipeline are evaluated through various architectures, utilizing the Charades dataset, known for its natural and spontaneous depiction of household activities. The experimentation demonstrates how the integration of both models increases the results up to 7% in the Object Detection task, which is fundamental for the correct identification of potential risks. This comprehensive approach aims to develop more human-aligned and accurate systems for aiding vulnerable populations in their daily lives.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
Pre-trained model word2vec-google-news-300 are used.
References
Agrawal, D.K., et al.: Fall risk prediction using wireless sensor insoles with machine learning. IEEE Access 11, 23119–23126 (2023)
Azorin-Lopez, J., et al.: A novel prediction method for early recognition of global human behaviour in image sequences. Neural Process. Lett. 43(2), 363–387 (2016)
Azorín-López, J., et al.: Human behaviour recognition based on trajectory analysis using neural networks. In: IJCNN, pp. 1–7 (2013)
Chen, S.: Toward ambient assistance: a spatially-aware virtual assistant enabled by object detection. In: ICCEA, pp. 494–501 (2020)
Flórez-Revuelta, F., et al.: Representation of 2d objects with a topology preserving network, April 2002
José García-Rodríguez and Juan Manuel García-Chamizo: Surveillance and human-computer interaction applications of self-growing models. Appl. Soft Comput. 11(7), 4413–4431 (2011)
Gomez-Donoso, F., et al.: A robotic platform for customized and interactive rehabilitation of persons with disabilities. Pattern Recogn. Lett. 99, 105–113 (2017)
Islam, S., Dash, A., Seum, A., Raj, A.H., Hossain, T., Shah, F.M.: Exploring video captioning techniques: a comprehensive survey on deep learning methods. SN Comput. Sci. 2(2), 120 (2021)
Lin, K., Li, L., Lin, C.-C., Ahmed, F., Gan, Z., Liu, Z., Yumao, L., Wang, L.: End-to-end transformers with sparse attention for video captioning, Swinbert (2022)
Lin, T.-Y., et al.: Microsoft coco: Common objects in context (2015)
Liu, J., Luo, H., Liu, H.: Deep learning-based data analytics for safety in construction. Autom. Constr. 140, 104302 (2022)
Luperto, M., Monroy, J., Jennifer, R., et al.: Integrating social assistive robots, iot, virtual communities and smart objects to assist at-home independently living elders: the movecare project. Int. J. Soc. Robot. 15(3), 517–545 (2023)
Naik, D., Jaidhar, C.D.: Video captioning using sentence vector-enabled convolutional framework with short-connected lstm. Multimed. Tools Appl. 83(4), 11187–11213 (2024)
Puig, X., Ra, K., Boben, M., Li, J., Wang, T., Fidler, S.: and Antonio Torralba. Simulating household activities via programs, Virtualhome (2018)
Redmon, J., Farhadi, A.: Yolo9000: Better, faster, stronger (2016)
Ren, S., He, K., Girshick, R., Sun, J.: Towards real-time object detection with region proposal networks, Faster r-cnn (2016)
Rodríguez-Juan, J., Ortiz-Perez, D., Garcia-Rodriguez, J., et al.: Indoor scenes video captioning. In: SOCO, pp. 153–162. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-42536-3_15
Savadkoohi, M., Oladunni, T., Thompson, L.A.: Deep neural networks for human’s fall-risk prediction using force-plate time series signal. Expert Syst. Appl. 182, November 2021
Sigurdsson, G.A., et al.: Hollywood in homes: Crowdsourcing data collection for activity understanding (2016). https://arxiv.org/abs/1604.01753
Viejo, D., Garcia, J., Cazorla, M., Gil, D., Johnsson, M.: Using gng to improve 3d feature extraction-application to 6dof egomotion. Neural Networks 32, 138–146 (2012). Selected Papers from IJCNN 2011
Wang, C.-Y., Bochkovskiy, A., Liao, H.-Y.M.: Yolov7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (2022)
Yang, A., et al.: Vid2seq: large-scale pretraining of a visual language model for dense video captioning (2023)
Yared, R., Abdulrazak, B.: Ambient technology to assist elderly people in indoor risks. Computers, 5(4) (2016)
Zaidi, S.S.A., et al.: A survey of modern deep learning based object detection models. Digital Signal Process. 126, 103514 (2022)
Zhao, Y., Misra, I., Krähenbühl, P., Girdhar, R.: Learning video representations from large language models. In: CVPR, pp. 6586–6597, June 2023
Górriz, J.M., et al.: Computational approaches to explainable artificial intelligence: advances in theory, applications and trends. Inf. Fusion 100, 101945 (2023)
Acknowledgment
We would like to thank “A way of making Europe” European Regional Development Fund (ERDF) and MCIN/AEI/10.13039/501100011033 for supporting this work under the “CHAN-TWIN” project (grant TED2021-130890B- C21. HORIZON-MSCA-2021-SE-0 action number: 101086387, REMARKABLE, Rural Environmental Monitoring via ultra wide-ARea networKs And distriButed federated Learning. CIAICO/2022/132 Consolidated group project “AI4Health” funded by Valencian government and International Center for Aging Research ICAR funded project “IASISTEM”. This work has also been supported by a Spanish regional grant for PhD studies, CIACIF/2022/175 and a research initiation grant from the University of Alicante, AII23-12. Finally we would like to thanks the support of the University Institute for Computer Research at the UA.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rodriguez-Juan, J., Ortiz-Perez, D., Garcia-Rodriguez, J., Tomás, D. (2024). Deep Learning for Enhanced Risk Assessment in Home Environments. In: Ferrández Vicente, J.M., Val Calvo, M., Adeli, H. (eds) Bioinspired Systems for Translational Applications: From Robotics to Social Engineering. IWINAC 2024. Lecture Notes in Computer Science, vol 14675. Springer, Cham. https://doi.org/10.1007/978-3-031-61137-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-61137-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-61136-0
Online ISBN: 978-3-031-61137-7
eBook Packages: Computer ScienceComputer Science (R0)