research-article

CrowdAct: Achieving High-Quality Crowdsourced Datasets in Mobile Activity Recognition

Authors:

Nattaya Mairittha,

Tittaya Mairittha,

Sozo InoueAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 5, Issue 1

Article No.: 50, Pages 1 - 32

https://doi.org/10.1145/3432222

Published: 30 March 2021 Publication History

Abstract

In this study, we propose novel gamified active learning and inaccuracy detection for crowdsourced data labeling for an activity recognition system using mobile sensing (CrowdAct). First, we exploit active learning to address the lack of accurate information. Second, we present the integration of gamification into active learning to overcome the lack of motivation and sustained engagement. Finally, we introduce an inaccuracy detection algorithm to minimize inaccurate data. To demonstrate the capability and feasibility of the proposed model in realistic settings, we developed and deployed the CrowdAct system to a crowdsourcing platform. For our experimental setup, we recruited 120 diverse workers. Additionally, we gathered 6,549 activity labels from 19 activity classes by using smartphone sensors and user engagement information. We empirically evaluated the quality of CrowdAct by comparing it with a baseline using techniques such as machine learning and descriptive and inferential statistics. Our results indicate that CrowdAct was effective in improving activity accuracy recognition, increasing worker engagement, and reducing inaccurate data in crowdsourced data labeling. Based on our findings, we highlight critical and promising future research directions regarding the design of efficient activity data collection with crowdsourcing.

References

[1]

Saeed Abdullah, Nicholas D Lane, and Tanzeem Choudhury. 2012. Towards population scale activity recognition: A framework for handling data diversity. In Twenty-Sixth AAAI Conference on Artificial Intelligence.

[2]

Utku Günay Acer, Marc van den Broeck, Claudio Forlivesi, Florian Heller, and Fahim Kawsar. 2019. Scaling Crowdsourcing with Mobile Workforce: A Case Study with Belgian Postal Service. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2 (2019), 1--32.

Digital Library

[3]

Hande Alemdar, Tim LM van Kasteren, and Cem Ersoy. 2011. Using active learning to allow activity recognition on a large scale. In International Joint Conference on Ambient Intelligence. Springer, 105--114.

Digital Library

[4]

Shahriyar Amini and Yang Li. 2013. CrowdLearner: rapidly creating mobile recognizers using crowdsourcing. In Proceedings of the 26th annual ACM symposium on User interface software and technology. 163--172.

Digital Library

[5]

Dana Angluin. 1988. Queries and concept learning. Machine learning 2, 4 (1988), 319--342.

[6]

Les E Atlas, David A Cohn, and Richard E Ladner. 1990. Training connectionist networks with queries and selective sampling. In Advances in neural information processing systems. 566--573.

[7]

Salikh Bagaveyev and Diane J Cook. 2014. Designing and evaluating active learning methods for activity recognition. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. 469--478.

Digital Library

[8]

Anahid Basiri, Muki Haklay, Giles Foody, and Peter Mooney. 2019. Crowdsourced geospatial data quality: Challenges and future directions.

[9]

Martin Berchtold, Matthias Budde, Dawud Gordon, Hedda R Schmidtke, and Michael Beigl. 2010. Actiserv: Activity recognition service for mobile phones. In International Symposium on Wearable Computers (ISWC) 2010. IEEE, 1--8.

[10]

Dimitar Bounov, Anthony DeRossi, Massimiliano Menarini, William G Griswold, and Sorin Lerner. 2018. Inferring loop invariants through gamification. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--13.

Digital Library

[11]

Hennie Brugman, Albert Russel, and Xd Nijmegen. 2004. Annotating Multi-media/Multi-modal Resources with ELAN. In LREC.

[12]

Andreas Bulling, Ulf Blanke, and Bernt Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys (CSUR) 46, 3 (2014), 1--33.

Digital Library

[13]

Fabrizio Carcillo, Yann-Aël Le Borgne, Olivier Caelen, Yacine Kessaci, Frédéric Oblé, and Gianluca Bontempi. 2019. Combining unsupervised and supervised learning in credit card fraud detection. Information sciences (2019).

[14]

Yung-Ju Chang, Gaurav Paruthi, and Mark W Newman. 2015. A field study comparing approaches to collecting annotated activity data in real-world settings. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 671--682.

Digital Library

[15]

Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. 2002. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research 16 (2002), 321--357.

[16]

Tianqi Chen, Tong He, Michael Benesty, Vadim Khotilovich, and Yuan Tang. 2015. Xgboost: extreme gradient boosting. R package version 0.4-2 (2015), 1--4.

[17]

Allan H Church. 1993. Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public opinion quarterly 57, 1 (1993), 62--79.

[18]

Federico Cruciani, Ian Cleland, Chris Nugent, Paul McCullagh, Kåre Synnes, and Josef Hallberg. 2018. Automatic annotation for human activity recognition in free living using a smartphone. Sensors 18, 7 (2018), 2203.

[19]

Sebastian Deterding, Dan Dixon, Rilla Khaled, and Lennart Nacke. 2011. From game design elements to gamefulness: defining" gamification". In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments. 9--15.

Digital Library

[20]

Sebastian Deterding, Miguel Sicart, Lennart Nacke, Kenton O'Hara, and Dan Dixon. 2011. Gamification. using game-design elements in non-gaming contexts. In CHI'11 extended abstracts on human factors in computing systems. 2425--2428.

Digital Library

[21]

Zhiguo Ding and Minrui Fei. 2013. An anomaly detection approach based on isolation forest algorithm for streaming data using sliding window. IFAC Proceedings Volumes 46, 20 (2013), 12--17.

[22]

Souad Djelassi and Isabelle Decoopman. 2013. Customers' participation in product development through crowdsourcing: Issues and implications. Industrial Marketing Management 42, 5 (2013), 683--692.

[23]

Olive Jean Dunn. 1961. Multiple comparisons among means. Journal of the American statistical association 56, 293 (1961), 52--64.

[24]

Tom Fawcett. 2006. An introduction to ROC analysis. Pattern recognition letters 27, 8 (2006), 861--874.

[25]

Zachary Fitz-Walter and Dian W Tjondronegoro. 2011. Exploring the opportunities and challenges of using mobile sensing for gamification and achievements. In UbiComp 11: Proceedings of the 2011 ACM Conference on Ubiquitous Computing. ACM Press, 1--5.

[26]

Jesús Fontecha, Fco Javier Navarro, Ramón Hervás, and José Bravo. 2013. Elderly frailty detection by using accelerometer-enabled smartphones and clinical information records. Personal and ubiquitous computing 17, 6 (2013), 1073--1083.

[27]

Juho Hamari, Jonna Koivisto, and Harri Sarsa. 2014. Does gamification work?-a literature review of empirical studies on gamification. In 2014 47th Hawaii international conference on system sciences. Ieee, 3025--3034.

Digital Library

[28]

Kotaro Hara, Abigail Adams, Kristy Milland, Saiph Savage, Chris Callison-Burch, and Jeffrey P Bigham. 2018. A data-driven analysis of workers' earnings on Amazon Mechanical Turk. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--14.

Digital Library

[29]

Kim Hartman. 2011. How do intrinsic and extrinsic motivation correlate with each other in open source software development?

[30]

Yu-chen Ho, Ching-hu Lu, I-han Chen, Shih-shinh Huang, Ching-yao Wang, Li-chen Fu, et al. 2009. Active-learning assisted self-reconfigurable activity recognition in a dynamic environment. In Proceedings of the 2009 IEEE international conference on Robotics and Automation. IEEE Press, 1567--1572.

Digital Library

[31]

HM Sajjad Hossain, Md Abdullah Al Hafiz Khan, and Nirmalya Roy. 2017. Active learning enabled activity recognition. Pervasive and Mobile Computing 38 (2017), 312--330.

[32]

Sozo Inoue, Paula Lago, Tahera Hossain, Tittaya Mairittha, and Nattaya Mairittha. 2019. Integrating Activity Recognition and Nursing Care Records: The System, Deployment, and a Verification Study. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--24.

Digital Library

[33]

Sozo Inoue and Xincheng Pan. 2016. Supervised and unsupervised transfer learning for activity recognition from simple in-home sensors. In Proceedings of the 13th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services. 20--27.

Digital Library

[34]

Eiichi Iwamoto, Masaki Matsubara, Chihiro Ota, Satoshi Nakamura, Tsutomu Terada, Hiroyuki Kitagawa, and Atsuyuki Morishima. 2018. Passerby Crowdsourcing: Workers' Behavior and Data Quality Management. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 4 (2018), 1--20.

Digital Library

[35]

Ashish Kapoor and Eric Horvitz. 2008. Experience sampling for building predictive user models: a comparative study. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 657--666.

Digital Library

[36]

Michael Kipp. 2001. Anvil-a generic annotation tool for multimodal dialogue. In Seventh European Conference on Speech Communication and Technology.

[37]

William H Kruskal and W Allen Wallis. 1952. Use of ranks in one-criterion variance analysis. Journal of the American statistical Association 47, 260 (1952), 583--621.

[38]

Jennifer R Kwapisz, Gary M Weiss, and Samuel A Moore. 2011. Activity recognition using cell phone accelerometers. ACM SigKDD Explorations Newsletter 12, 2 (2011), 74--82.

Digital Library

[39]

Nicholas D Lane, Ye Xu, Hong Lu, Shaohan Hu, Tanzeem Choudhury, Andrew T Campbell, and Feng Zhao. 2011. Enabling large-scale human activity inference on smartphones using community similarity networks (csn). In Proceedings of the 13th international conference on Ubiquitous computing. 355--364.

Digital Library

[40]

Tak Yeon Lee, Casey Dugan, Werner Geyer, Tristan Ratchford, Jamie Rasmussen, N Sadat Shami, and Stela Lupushor. 2013. Experiments on motivational feedback for crowdsourced workers. In Seventh International AAAI Conference on Weblogs and Social Media.

[41]

David D Lewis and William A Gale. 1994. A sequential algorithm for training text classifiers. In SIGIR'94. Springer, 3--12.

[42]

Blerina Lika, Kostas Kolomvatsos, and Stathes Hadjiefthymiades. 2014. Facing the cold start problem in recommender systems. Expert Systems with Applications 41, 4 (2014), 2065--2073.

Digital Library

[43]

Brent Longstaff, Sasank Reddy, and Deborah Estrin. 2010. Improving activity classification for health applications on mobile devices using active and semi-supervised learning. In 2010 4th International Conference on Pervasive Computing Technologies for Healthcare. IEEE, 1--7.

[44]

Nattaya Mairittha and Sozo Inoue. 2018. Gamification for High-Quality Dataset in Mobile Activity Recognition. In International Conference on Mobile Computing, Applications, and Services. Springer, 216--222.

[45]

Nattaya Mairittha and Sozo Inoue. 2019. Crowdsourcing System Management for Activity Data with Mobile Sensors. In 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR). IEEE, 85--90.

[46]

Nattaya Mairittha, Tittaya Mairittha, and Sozo Inoue. 2018. A Mobile App for Nursing Activity Recognition. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers. 400--403.

Digital Library

[47]

Nattaya Mairittha, Tittaya Mairittha, and Sozo Inoue. 2019. On-Device Deep Learning Inference for Efficient Activity Data Collection. Sensors 19, 15 (2019), 3434.

[48]

Nattaya Mairittha, Tittaya Mairittha, and Sozo Inoue. 2019. Optimizing activity data collection with gamification points using uncertainty based active learning. In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. 761--767.

Digital Library

[49]

Jane McGonigal. 2011. Reality is broken: Why games make us better and how they can change the world. Penguin.

Digital Library

[50]

Mohamed Musthag, Andrew Raij, Deepak Ganesan, Santosh Kumar, and Saul Shiffman. 2011. Exploring micro-incentive strategies for participant compensation in high-burden studies. In Proceedings of the 13th international conference on Ubiquitous computing. 435--444.

Digital Library

[51]

Heather L O'Brien, Paul Cairns, and Mark Hall. 2018. A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human-Computer Studies 112 (2018), 28--39.

[52]

Maria V Palacin-Silva, Antti Knutas, Maria Angela Ferrario, Jari Porras, Jouni Ikonen, and Chandara Chea. 2018. The role of gamification in participatory environmental sensing: A study in the wild. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--13.

Digital Library

[53]

Gabriele Paolacci, Jesse Chandler, and Panagiotis G Ipeirotis. 2010. Running experiments on amazon mechanical turk. Judgment and Decision making 5, 5 (2010), 411--419.

[54]

Martin Pielot, Karen Church, and Rodrigo De Oliveira. 2014. An in-situ study of mobile phone notifications. In Proceedings of the 16th international conference on Human-computer interaction with mobile devices & services. 233--242.

Digital Library

[55]

Sihang Qiu, Ujwal Gadiraju, and Alessandro Bozzon. [n.d.]. Improving Worker Engagement Through Conversational Microtask Crowdsourcing. ([n. d.]).

[56]

Sasank Reddy, Min Mun, Jeff Burke, Deborah Estrin, Mark Hansen, and Mani Srivastava. 2010. Using mobile phones to determine transportation modes. ACM Transactions on Sensor Networks (TOSN) 6, 2 (2010), 1--27.

Digital Library

[57]

Attila Reiss and Didier Stricker. 2012. Creating and benchmarking a new dataset for physical activity monitoring. In Proceedings of the 5th International Conference on PErvasive Technologies Related to Assistive Environments. 1--8.

Digital Library

[58]

Guido Sautter and Klemens Böhm. 2013. High-throughput crowdsourcing mechanisms for complex tasks. Social Network Analysis and Mining 3, 4 (2013), 873--888.

[59]

Burr Settles. 2012. Active Learning. Vol. 18. Morgan & Claypool Publishers.

[60]

Claude E Shannon. 1948. A mathematical theory of communication. Bell system technical journal 27, 3 (1948), 379--423.

[61]

Samuel Sanford Shapiro and Martin B Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52, 3/4 (1965), 591--611.

[62]

Gunnar A Sigurdsson, Gül Varol, Xiaolong Wang, Ali Farhadi, Ivan Laptev, and Abhinav Gupta. 2016. Hollywood in homes: Crowdsourcing data collection for activity understanding. In European Conference on Computer Vision. Springer, 510--526.

[63]

Maja Stikic, Kristof Van Laerhoven, and Bernt Schiele. 2008. Exploring semi-supervised and active learning for activity recognition. In 2008 12th IEEE International Symposium on Wearable Computers. IEEE, 81--88.

Digital Library

[64]

Nadeem Ahmed Syed, Syed Huan, Liu Kah, and Kay Sung. 1999. Incremental learning with support vector machines. (1999).

[65]

Emma L Tonkin, Alison Burrows, Przemys&lstroke;aw R Woznowski, Pawel Laskowski, Kristina Y Yordanova, Niall Twomey, and Ian J Craddock. 2018. Talk, Text, Tag? Understanding Self-Annotation of Smart Home Data from a User's Perspective. Sensors 18, 7 (2018), 2365.

[66]

Niels Van Berkel, Jorge Goncalves, Simo Hosio, and Vassilis Kostakos. 2017. Gamification of mobile experience sampling improves data quality and quantity. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 1--21.

Digital Library

[67]

Kristof Van Laerhoven, David Kilian, and Bernt Schiele. 2008. Using rhythm awareness in long-term activity recognition. In 2008 12th IEEE International Symposium on Wearable Computers. IEEE, 63--66.

Digital Library

[68]

Maja Vukovic, Rajarshi Das, and Soundar Kumara. 2013. From sensing to controlling: the state of the art in ubiquitous crowdsourcing. International Journal of Communication Networks and Distributed Systems 11, 1 (2013), 11--25.

Digital Library

[69]

Yufeng Wang, Xueyu Jia, Qun Jin, and Jianhua Ma. 2016. QuaCentive: a quality-aware incentive mechanism in mobile crowdsourced sensing (MCS). The Journal of Supercomputing 72, 8 (2016), 2924--2941.

Digital Library

[70]

Yu Xiao, Pieter Simoens, Padmanabhan Pillai, Kiryong Ha, and Mahadev Satyanarayanan. 2013. Lowering the barriers to large-scale mobile crowdsensing. In Proceedings of the 14th Workshop on Mobile Computing Systems and Applications. 1--6.

Digital Library

[71]

Mengwei Xu, Feng Qian, Qiaozhu Mei, Kang Huang, and Xuanzhe Liu. 2018. Deeptype: On-device deep learning for input personalization service with minimal privacy concern. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 4 (2018), 1--26.

Digital Library

[72]

Fei Yan, Josef Kittler, David Windridge, William Christmas, Krystian Mikolajczyk, Stephen Cox, and Qiang Huang. 2014. Automatic annotation of tennis games: An integration of audio, vision, and learning. Image and Vision Computing 32, 11 (2014), 896--903.

Digital Library

[73]

Man-Ching Yuen, Irwin King, and Kwong-Sak Leung. 2011. A survey of crowdsourcing systems. In 2011 IEEE third international conference on privacy, security, risk and trust and 2011 IEEE third international conference on social computing. IEEE, 766--773.

[74]

Liyue Zhao, Gita Sukthankar, and Rahul Sukthankar. 2011. Robust active learning using crowdsourced annotations for activity recognition. In Workshops at the Twenty-Fifth AAAI Conference on Artificial Intelligence.

[75]

Chong Zhou and Randy C Paffenroth. 2017. Anomaly detection with robust deep autoencoders. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 665--674.

Digital Library

[76]

Mengdie Zhuang and Ujwal Gadiraju. 2019. In What Mood Are You Today? An Analysis of Crowd Workers' Mood, Performance and Engagement. In Proceedings of the 10th ACM Conference on Web Science. 373--382.

Digital Library

Cited By

Humayun HMalik MGhazali M(2024)Analysis of Motivational Theories in Crowdsourcing Using Long Tail Theory: A Systematic Literature ReviewInternational Journal of Crowd Science10.26599/IJCS.2023.91000108:1(10-27)Online publication date: Feb-2024
https://doi.org/10.26599/IJCS.2023.9100010
Hong ZLi ZZhong SLyu WWang HDing YHe TZhang D(2024)CrossHAR: Generalizing Cross-dataset Human Activity Recognition via Hierarchical Self-Supervised PretrainingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595978:2(1-26)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659597
Chen WHussain WAl-Qudah IAl-Naymat GZhang X(2024)Intersection of machine learning and mobile crowdsourcing: a systematic topic-driven reviewPersonal and Ubiquitous Computing10.1007/s00779-024-01820-wOnline publication date: 10-Jun-2024
https://doi.org/10.1007/s00779-024-01820-w
Show More Cited By

Index Terms

CrowdAct: Achieving High-Quality Crowdsourced Datasets in Mobile Activity Recognition

Recommendations

Annotating smart environment sensor data for activity learning
Smart Environments: Technology to Support Healthcare

The pervasive sensing technologies found in smart homes offer unprecedented opportunities for providing health monitoring and assistance to individuals experiencing difficulties living independently at home. In order to monitor the functional health of ...
Ubiquitous reminders to manage timetable in a smart home
iiWAS '16: Proceedings of the 18th International Conference on Information Integration and Web-based Applications and Services

The aging of the population will bring changes on home care for elders and gerontechnologies provide alternatives for aging in place. This article presents a concept of a smart home that reminds the resident the activities he has planned on an ...
NuActiv: recognizing unseen new activities using semantic attribute-based learning
MobiSys '13: Proceeding of the 11th annual international conference on Mobile systems, applications, and services

We study the problem of how to recognize a new human activity when we have never seen any training example of that activity before. Recognizing human activities is an essential element for user-centric and context-aware applications. Previous studies ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 5, Issue 1

March 2021

1272 pages

EISSN:2474-9567

DOI:10.1145/3459088

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 March 2021

Published in IMWUT Volume 5, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
402
Total Downloads

Downloads (Last 12 months)59
Downloads (Last 6 weeks)8

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Humayun HMalik MGhazali M(2024)Analysis of Motivational Theories in Crowdsourcing Using Long Tail Theory: A Systematic Literature ReviewInternational Journal of Crowd Science10.26599/IJCS.2023.91000108:1(10-27)Online publication date: Feb-2024
https://doi.org/10.26599/IJCS.2023.9100010
Hong ZLi ZZhong SLyu WWang HDing YHe TZhang D(2024)CrossHAR: Generalizing Cross-dataset Human Activity Recognition via Hierarchical Self-Supervised PretrainingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36595978:2(1-26)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659597
Chen WHussain WAl-Qudah IAl-Naymat GZhang X(2024)Intersection of machine learning and mobile crowdsourcing: a systematic topic-driven reviewPersonal and Ubiquitous Computing10.1007/s00779-024-01820-wOnline publication date: 10-Jun-2024
https://doi.org/10.1007/s00779-024-01820-w
Stamatiadou MVryzas NVrysis LDimoulas C(2023)BeeMate the Game: A hunting treasure serious game for raising awareness and audience engagement in air pollution monitoring2023 8th International Conference on Smart and Sustainable Technologies (SpliTech)10.23919/SpliTech58164.2023.10193725(1-4)Online publication date: 20-Jun-2023
https://doi.org/10.23919/SpliTech58164.2023.10193725
Humayun HGhazali MMalik M(2023)Is sustained participation a myth in crowdsourcing? A reviewEuropean Journal of Innovation Management10.1108/EJIM-10-2022-0589Online publication date: 2-Aug-2023
https://doi.org/10.1108/EJIM-10-2022-0589
Tanigaki KTeoh TYoshimura NMaekawa THara T(2022)Predicting Performance Improvement of Human Activity Recognition Model by Additional Data CollectionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503196:3(1-33)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550319
Majethia RGhosh SNolasco HShahid FViswanath VShehu IZhao Y(2022)Stepping Into the Next Decade of Ubiquitous and Pervasive Computing: UbiComp and ISWC 2021IEEE Pervasive Computing10.1109/MPRV.2022.316006321:2(87-99)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1109/MPRV.2022.3160063

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents