research-article

Detection of Semantic Risk Situations in Lifelog Data for Improving Life of Frail People

Authors:

Thinhinane Yebda,

Jenny Benois-Pineau,

Hélène Amièva,

Cathal GurrinAuthors Info & Claims

ICMR '20: Proceedings of the 2020 International Conference on Multimedia Retrieval

Pages 402 - 406

https://doi.org/10.1145/3372278.3391931

Published: 08 June 2020 Publication History

Abstract

The automatic recognition of risk situations for frail people is an urgent research topic for the interdisciplinary artificial intelligence and multimedia community. Risky situations can be recognized from lifelog data recorded with wearable devices. In this paper, we present a new approach for the detection of semantic risk situations for frail people in lifelog data. Concept matching between general lifelog and risk taxonomies was realized and tuned AlexNet was deployed for detection of two semantic risks situations such as risk of domestic accident and risk of fraud with promising results.

References

[1]

A. Parant, "Blanchet mickaël, 2017, atlas des séniors et du grand âge en france. 100 cartes et graphiques pour analyser et comprendre, rennes, presses de l'ehesp, 120 p." Population, vol. 73, no. 4, pp. 852--853, 2018.

[2]

N. Fedarko, "The biology of aging and frailty," Clinics in Geriatric Medicine, vol. 27, no. 1, pp. 27--37, 2 2011.

[3]

S. S. Vemulapalli, M. B. Ulak, E. E. Ozguven, T. Sando, M. W. Horner, Y. Abdelrazig, and R. Moses, "Gis-based spatial and temporal analysis of aging-involved accidents: a case study of three counties in florida," Applied Spatial Analysis and Policy, vol. 10, no. 4, pp. 537--563, Dec 2017.

[4]

M. E. Tinetti and C. Kumar, "The patient who falls:"it's always a trade-off", Jama, vol. 303, no. 3, pp. 258--266, 2010.

[5]

S. Sabesan and R. Sankar, "Improving long-term management of epilepsy using a wearable multimodal seizure detection system," Epilepsy & Behavior, vol. 46, pp. 56--57, 2015.

[6]

N. Hegde, M. Bries, T. Swibas, E. Melanson, and E. Sazonov, "Automatic recognition of activities of daily living utilizing insole-based and wrist-worn wearable sensors," IEEE journal of biomedical and health informatics, vol. 22, no. 4, pp. 979--988, 2017.

[7]

N. Hegde and E. Sazonov, "Smartstep: A fully integrated, low-power insole monitor," Electronics, vol. 3, no. 2, pp. 381--397, 2014.

[8]

P. Bellagente, C. Crema, A. Depari, P. Ferrari, A. Flammini, G. Lanfranchi, G. Lenzi, M. Maddiona, S. Rinaldi, E. Sisinni et al., "Remote and non-invasive monitoring of elderly in a smart city context," in 2018 IEEE Sensors Applications Symposium (SAS). IEEE, 2018, pp. 1--6.

[9]

M. Kheirkhahan, S. Nair, A. Davoudi, P. Rashidi, A. A.Wanigatunga, D. B. Corbett, T. Mendoza, T. M. Manini, and S. Ranka, "A smartwatch-based framework for real-time and online assessment and mobility monitoring," Journal of biomedical informatics, vol. 89, pp. 29--40, 2019.

[10]

S. B. Khojasteh, J. R. Villar, C. Chira, V. M. González, and E. De la Cal, "Improving fall detection using an on-wrist wearable accelerometer," Sensors, vol. 18, no. 5, p. 1350, 2018.

[11]

L. Maimoon, J. Chuang, H. Zhu, S. Yu, K.-S. Peng, R. Prayakarao, J. Bai, D. Zeng, S.-H. Li, H. Lu et al., "Silverlink: developing an international smart and connected home monitoring system for senior care," in International Conference on Smart Health. Springer, 2016, pp. 65--77.

[12]

P. Pierleoni, A. Belli, L. Palma, M. Pellegrini, L. Pernini, and S. Valenti, "A high reliability wearable device for elderly fall detection," IEEE Sensors Journal, vol. 15, no. 8, pp. 4544--4553, 2015.

[13]

Y. Lin, C. Wang, J. Wang, and Z. Dou, "A novel dynamic spectrum access framework based on reinforcement learning for cognitive radio sensor networks," Sensors, vol. 16, no. 10, p. 1675, 2016.

[14]

L. Zhu, R. Wang, Z. Wang, and H. Yang, "Tagcare: Using rfids to monitor the status of the elderly living alone," IEEE Access, vol. 5, pp. 11 364--11 373, 2017.

[15]

J. Yuan, K. K. Tan, T. H. Lee, and G. C. H. Koh, "Power-efficient interrupt-driven algorithms for fall detection and classification of activities of daily living," IEEE Sensors Journal, vol. 15, no. 3, pp. 1377--1387, 2014.

[16]

D. Yacchirema, J. S. de Puga, C. Palau, and M. Esteve, "Fall detection system for elderly people using iot and ensemble machine learning algorithm," Personal and Ubiquitous Computing, vol. 23, no. 5--6, pp. 801--817, 2019.

[17]

D. Seo, B. Yoo, and H. Ko, "Data-driven smart home system for elderly people based on web technologies," in International Conference on Distributed, Ambient, and Pervasive Interactions. Springer, 2016, pp. 122--131.

[18]

M. M. Baig, S. Afifi, H. GholamHosseini, and F. Mirza, "A systematic review of wearable sensors and iot-based monitoring applications for older adults--a focus on ageing population and independent living," Journal of medical systems, vol. 43, no. 8, p. 233, 2019.

Digital Library

[19]

B. Klaassen, B. J. van Beijnum, and H. J. Hermens, "Usability in telemedicine systems-a literature survey," International journal of medical informatics, vol. 93, pp. 57--69, 2016.

[20]

A. R. Doherty, K. Pauly-Takacs, N. Caprani, C. Gurrin, C. J. A. Moulin, N. E. O'Connor, and A. F. Smeaton, "Experiences of aiding autobiographical memory using the sensecam," Human-Computer Interaction, vol. 27, no. 1--2, pp. 151--174, 2012.

[21]

R. J. A. S. K. I. Louise Hopper, Eamonn Newman, "Dementia ambient care: Homebased monitoring and enablement of people with dementia," in 9th Panhellenic Conference on Alzheimer's Disease and 1st Mediterranean on Neurodegenerative Diseases Conference,PICAD 2015, 2015, pp. 1--4.

[22]

E. Talavera, N. Petkov, and P. Radeva, "Towards unsupervised familiar scene recognition in egocentric videos," CoRR, vol. abs/1905.04093, 2019.

[23]

C. Gurrin, K. Schoeffmann, H. Joho, A. Leibetseder, L. Zhou, A. Duane, D.-T. Dang-Nguyen, M. Riegler, L. Piras, M.-T. Tran, J. Lokoc, and W. Huerst, "[invited papers] comparing approaches to interactive lifelog search at the lifelog search challenge (lsc2018)," ITE Transactions on Media Technology and Applications, vol. 7, no. 2, pp. 46--59, 2019.

[24]

E. S. Aimar, P. Radeva, and M. Dimiccoli, "Social relation recognition in egocentric photostreams," in ICIP. IEEE, 2019, pp. 3227--3231.

[25]

K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," 2015, cite arxiv:1512.03385Comment: Tech report. [Online]. Available: http://arxiv.org/abs/1512.03385

[26]

I. González-Díaz, J. Benois-Pineau, J. Domenger, D. Cattaert, and A. de Rugy, "Perceptually-guided deep neural networks for ego-action prediction: Object grasping," Pattern Recognit., vol. 88, pp. 223--235, 2019.

Digital Library

[27]

C. Gurrin, K. Schoe, H. Joho, and B. Munzer, "A Test Collection for Interactive Lifelog Retrieval," in MMM 2019, the 25th International Conference on MultiMedia Modeling, Thessaloniki, Greece, 2019.

[28]

A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2012, pp. 1097--1105. [Online]. Available: http://papers.nips.cc/paper/4824-imagenet-classification-with-deepconvolutional- neural-networks.pdf

[29]

B. Tavernier, C. Helmer, F. Portet, H. Amieva, I. Carrière, J.-F. Dartigues, J. A. Ávila Funes, K. Ritchie, L. M. Gutiérrez-Robledo, M. L. Goff, and P. Barberger-Gateau, "Frailty Among Community-Dwelling Elderly People in France: The Three-City Study," The Journals of Gerontology: Series A, vol. 63, no. 10, pp. 1089--1096, 10 2008.

[30]

T. Yebda, J. Benois-Pineau, H. Amieva, and B. Frolicher, "Multi-sensing of fragile persons for risk situation detection: devices, methods, challenges," in 2019 International Conference on Content-Based Multimedia Indexing (CBMI). IEEE, 2019, pp. 1--6.

[31]

A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, 2012, pp. 1097--1105.

Digital Library

[32]

Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, "Caffe: Convolutional architecture for fast feature embedding," in Proceedings of the 22nd ACM international conference on Multimedia, 2014, pp. 675--678.

[33]

A. M. Obeso, J. Benois-Pineau, M. S. García-Vázquez, and A. A. Ramírez-Acosta, "Saliency-based selection of visual content for deep convolutional neural networks - application to architectural style classification," Multimedia Tools Appl., vol. 78, no. 8, pp. 9553--9576, 2019.

Digital Library

[34]

T. Chen, M. Li, Y. Li, M. Lin, N. Wang, M. Wang, T. Xiao, B. Xu, C. Zhang, and Z. Zhang, "Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems," arXiv preprint arXiv:1512.01274, 2015.

Cited By

Jang HLee SPark HLee W(2024)Development of a Low-Power Touch-Based Lifelogging ProcessorIEEE Transactions on Circuits and Systems II: Express Briefs10.1109/TCSII.2024.336841071:8(3910-3914)Online publication date: Aug-2024
https://doi.org/10.1109/TCSII.2024.3368410
Mallick RYebda TBenois-Pineau JZemmari APech MAmieva H(2021)A GRU Neural Network with attention mechanism for detection of risk situations on multimodal lifelog data2021 International Conference on Content-Based Multimedia Indexing (CBMI)10.1109/CBMI50038.2021.9461910(1-6)Online publication date: 28-Jun-2021
https://doi.org/10.1109/CBMI50038.2021.9461910

Index Terms

Detection of Semantic Risk Situations in Lifelog Data for Improving Life of Frail People
1. Applied computing
  1. Life and medical sciences
    1. Health care information systems
2. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Multimodal Sensor Data Analysis for Detection of Risk Situations of Fragile People in @home Environments
MultiMedia Modeling
Abstract
Multimedia (MM) nowadays often means “Multimodality”. The target application area of MM technologies further extends to healthcare. Health parameters monitoring, context and situational recognition in ambient assisted living - all these ...
An innovative AAL system based on neural networks and IoT-aware technologies to improve the quality of life in elderly people

Nowadays more and more elderly people need support in daily activities. This is due to the increase of cognitive diseases and other conditions which lead the elderly to not being self-sufficient. Considering this, providing an Ambient Assisted Living ...
Abnormality detection for improving elder's daily life independent
ICOST'10: Proceedings of the Aging friendly technology for health and independence, and 8th international conference on Smart homes and health telematics

Since the dramatic demographic change makes it inevitable that rapid aging of the population is an unprecedented phenomenon in Taiwan. A growing social problem is supporting older adults who want to live independently in their own homes. It needs a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '20: Proceedings of the 2020 International Conference on Multimedia Retrieval

June 2020

605 pages

ISBN:9781450370875

DOI:10.1145/3372278

General Chairs:
Cathal Gurrin
Dublin City University, Ireland
,
Björn Þór Jónsson
IT University of Copenhagen, Denmark
,
Noriko Kando
National Institute of Informatics, Tokyo
,
Program Chairs:
Klaus Schoeffmann
Klagenfurt University, Austria
,
Phoebe Chen
La Trobe University, Australia
,
Noel E. O'Connor
Dublin City University, Ireland

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

AAP 2019
French national ANRT grant

Conference

ICMR '20

Sponsor:

SIGMM

ICMR '20: International Conference on Multimedia Retrieval

June 8 - 11, 2020

Dublin, Ireland

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
102
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jang HLee SPark HLee W(2024)Development of a Low-Power Touch-Based Lifelogging ProcessorIEEE Transactions on Circuits and Systems II: Express Briefs10.1109/TCSII.2024.336841071:8(3910-3914)Online publication date: Aug-2024
https://doi.org/10.1109/TCSII.2024.3368410
Mallick RYebda TBenois-Pineau JZemmari APech MAmieva H(2021)A GRU Neural Network with attention mechanism for detection of risk situations on multimodal lifelog data2021 International Conference on Content-Based Multimedia Indexing (CBMI)10.1109/CBMI50038.2021.9461910(1-6)Online publication date: 28-Jun-2021
https://doi.org/10.1109/CBMI50038.2021.9461910

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents