poster

Multimodal multisensor activity annotation tool

Authors:

Michael Barz,

Mohammad Mehdi Moniri,

Markus Weber,

Daniel SonntagAuthors Info & Claims

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

Pages 17 - 20

https://doi.org/10.1145/2968219.2971459

Published: 12 September 2016 Publication History

Get Access

Abstract

In this paper we describe a multimodal-multisensor annotation tool for physiological computing; for example mobile gesture-based interaction devices or health monitoring devices can be connected. It should be used as an expert authoring tool to annotate multiple video-based sensor streams for domain-specific activities. Resulting datasets can be used as supervised datasets for new machine learning tasks. Our tool provides connectors to commercially available sensor systems (e.g., Intel RealSense F200 3D camera, Leap Motion, and Myo) and a graphical user interface for annotation.

References

[1]

Stamatia Dasiopoulou, Eirini Giannakidou, Georgios Litos, Polyxeni Malasioti, and Yiannis Kompatsiaris. 2011. Knowledge-driven Multimedia Information Extraction and Ontology Evolution. Springer-Verlag, Chapter A Survey of Semantic Image and Video Annotation Tools, 196--239. http://dl.acm.org/citation.cfm?id=2001069.2001077

Digital Library

Google Scholar

[2]

Michael Kipp. 2008. Spatiotemporal Coding in ANVIL. In Proceedings of the 6th international conference on Language Resources and Evaluation. ELRA.

Google Scholar

[3]

Z. Palotai, M. Lang, A. Sarkany, Z. Töser, D. Sonntag, T. Toyama, and A. Lörincz. 2014. LabelMovie: Semi-supervised machine annotation tool with quality assurance and crowd-sourcing options for videos. In 12th International Workshop on Content-Based Multimedia Indexing. 1--4.

Crossref

Google Scholar

[4]

Klaus Schoeffmann, Marco A. Hudelist, and Jochen Huber. 2015. Video Interaction Tools: A Survey of Recent Work. ACM Comput. Surv. 48, 1, Article 14 (2015), 14:1--14:34 pages.

Digital Library

Google Scholar

[5]

Daniel Sonntag. 2014. ERmed -- Towards Medical Multimodal Cyber-Physical Environments. In Foundations of Augmented Cognition. Advancing Human Performance and Decision-Making through Adaptive Systems. Springer, 359--370.

Crossref

Google Scholar

[6]

Thomas Stiefmeier, Daniel Roggen, Georg Ogris, Paul Lukowicz, and Gerhard Tröster. 2008. Wearable Activity Tracking in Car Manufacturing. IEEE Pervasive Computing 7, 2 (April 2008), 42--50.

Digital Library

Google Scholar

Cited By

View all

Cuadra ABreuch JEstrada SIhim DHung IAskaryar DHassanien MFessele KLanday J(2024)Digital Forms for All: A Holistic Multimodal Large Language Model Agent for Health Data EntryProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596248:2(1-39)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659624
Gunthermann L(2023)Application for Doctoral Consortium IUI 2023Companion Proceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581754.3584112(233-236)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581754.3584112
Ollenschläger MKüderle AMehringer WSeifer AWinkler JGaßner HKluge FEskofier B(2022)MaD GUI: An Open-Source Python Package for Annotation and Analysis of Time-Series DataSensors10.3390/s2215584922:15(5849)Online publication date: 5-Aug-2022
https://doi.org/10.3390/s22155849
Show More Cited By

Index Terms

Multimodal multisensor activity annotation tool
1. Human-centered computing

Recommendations

Multimodal mood-based annotation

The paper presents an architecture for multimodal mood-based annotation systems. The architecture aims at the implementation of interactive multimodal systems to support communities of users in the creation and management of annotations in locative ...
Capturing user tests in a multimodal, multidevice informal prototyping tool
ICMI '03: Proceedings of the 5th international conference on Multimodal interfaces

Interaction designers are increasingly faced with the challenge of creating interfaces that incorporate multiple input modalities, such as pen and speech, and span multiple devices. Few early stage prototyping tools allow non-programmers to prototype ...
Flexible and scalable annotation tool to develop scene understanding datasets
HILDA '22: Proceedings of the Workshop on Human-In-the-Loop Data Analytics

Recent progress in data-driven vision and language-based tasks demands developing training datasets enriched with multiple modalities representing human intelligence. The link between text and image data is one of the crucial modalities for developing AI ...

Comments

Information & Contributors

Information

Published In

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

September 2016

1807 pages

ISBN:9781450344623

DOI:10.1145/2968219

General Chairs:
Paul Lukowicz
DFKI and University of Kaiserslautern
,
Antonio Krüger
DFKI and Saarland University
,
Program Chairs:
Andreas Bulling
Max Planck Institute for Informatics
,
Youn-Kyung Lim
KAIST
,
Shwetak N. Patel
University of Washington

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2016

Check for updates

Author Tags

Qualifiers

Poster

Funding Sources

European Institute of Innovation and Technology

Conference

UbiComp '16

Sponsor:

UbiComp '16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 12 - 16, 2016

Germany, Heidelberg

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
354
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Cuadra ABreuch JEstrada SIhim DHung IAskaryar DHassanien MFessele KLanday J(2024)Digital Forms for All: A Holistic Multimodal Large Language Model Agent for Health Data EntryProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596248:2(1-39)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659624
Gunthermann L(2023)Application for Doctoral Consortium IUI 2023Companion Proceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581754.3584112(233-236)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581754.3584112
Ollenschläger MKüderle AMehringer WSeifer AWinkler JGaßner HKluge FEskofier B(2022)MaD GUI: An Open-Source Python Package for Annotation and Analysis of Time-Series DataSensors10.3390/s2215584922:15(5849)Online publication date: 5-Aug-2022
https://doi.org/10.3390/s22155849
Jones MByun CJohnson NSeppi K(2022)Understanding the Roles of Video and Sensor Data in the Annotation of Human ActivitiesInternational Journal of Human–Computer Interaction10.1080/10447318.2022.210158939:18(3634-3648)Online publication date: Aug-2022
https://doi.org/10.1080/10447318.2022.2101589
Ponnada ACooper STang QThapa-Chhetry BMiller JJohn DIntille S(2021)Signaligner Pro: A Tool to Explore and Annotate Multi-day Raw Accelerometer Data2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops51409.2021.9431110(475-480)Online publication date: 22-Mar-2021
https://doi.org/10.1109/PerComWorkshops51409.2021.9431110
Kwon HAbowd GPlötz TFarrahi KHarle RLane N(2019)Handling annotation uncertainty in human activity recognitionProceedings of the 2019 ACM International Symposium on Wearable Computers10.1145/3341163.3347744(109-117)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3341163.3347744
Ponnada ACooper SThapa-Chhetry BMiller JJohn DIntille SArnedo JNacke LVanden Abeele VToups Dugas P(2019)Designing Videogames to Crowdsource Accelerometer Data Annotation for Activity Recognition ResearchProceedings of the Annual Symposium on Computer-Human Interaction in Play10.1145/3311350.3347153(135-147)Online publication date: 17-Oct-2019
https://dl.acm.org/doi/10.1145/3311350.3347153
Buddhan AEswaran SBuddhan DSripurushottama S(2019)Even Driven Multimodal Augmented Reality based Command and Control Systems for Mining IndustryProcedia Computer Science10.1016/j.procs.2019.04.135151(965-970)Online publication date: 2019
https://doi.org/10.1016/j.procs.2019.04.135
Diete ASztyler TStuckenschmidt H(2018)Exploring Semi-Supervised Methods for Labeling Support in Multimodal DatasetsSensors10.3390/s1808263918:8(2639)Online publication date: 11-Aug-2018
https://doi.org/10.3390/s18082639
Radu VTong CBhattacharya SLane NMascolo CMarina MKawsar F(2018)Multimodal Deep Learning for Activity and Context RecognitionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31611741:4(1-27)Online publication date: 8-Jan-2018
https://dl.acm.org/doi/10.1145/3161174
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Multimodal mood-based annotation

Capturing user tests in a multimodal, multidevice informal prototyping tool

Flexible and scalable annotation tool to develop scene understanding datasets