short-paper

Open access

Reducing Response Time for Multimedia Event Processing using Domain Adaptation

Authors:

Edward CurryAuthors Info & Claims

ICMR '20: Proceedings of the 2020 International Conference on Multimedia Retrieval

Pages 261 - 265

https://doi.org/10.1145/3372278.3390722

Published: 08 June 2020 Publication History

Abstract

The Internet of Multimedia Things (IoMT) is an emerging concept due to the large amount of multimedia data produced by sensing devices. Existing event-based systems mainly focus on scalar data, and multimedia event-based solutions are domain-specific. Multiple applications may require handling of numerous known/unknown concepts which may belong to the same/different domains with an unbounded vocabulary. Although deep neural network-based techniques are effective for image recognition, the limitation of having to train classifiers for unseen concepts will lead to an increase in the overall response-time for users. Since it is not practical to have all trained classifiers available, it is necessary to address the problem of training of classifiers on demand for unbounded vocabulary. By exploiting transfer learning based techniques, evaluations showed that the proposed framework can answer within ~0.01 min to ~30 min of response-time with accuracy ranges from 95.14% to 98.53%, even when all subscriptions are new/unknown.

References

[1]

Sheharyar Ahmad, Kashif Ahmad, Nasir Ahmad, and Nicola Conci. 2017. Convolutional Neural Networks for Disaster Images Retrieval. In MediaEval.

[2]

Sufyan Almajali, I Dhiah el Diehn, Haythem Bany Salameh, Moussa Ayyash, and Hany Elgala. 2018. A distributed multi-layer MEC-cloud architecture for processing large scale IoT-based multimedia applications. Multimedia Tools and Applications (2018), 1--22.

[3]

Sheeraz A Alvi, Bilal Afzal, Ghalib A Shah, Luigi Atzori, and Waqar Mahmood. 2015. Internet of multimedia things: Vision and challenges. Ad Hoc Networks, Vol. 33 (2015), 87--111.

Digital Library

[4]

Asra Aslam. 2020. Object Detection for Unseen Domains while Reducing Response Time using Knowledge Transfer in Multimedia Event Processing. In Accepted for Proceedings of the 2020 ACM on International Conference on Multimedia Retrieval (ICMR).

Digital Library

[5]

Asra Aslam and Edward Curry. 2018. Towards a Generalized Approach for Deep Neural Network Based Event Processing for the Internet of Multimedia Things. IEEE Access, Vol. 6 (2018), 25573--25587.

[6]

Asra Aslam, Souleiman Hasan, and Edward Curry. 2017. Challenges with image event processing: Poster. In Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems. 347--348.

Digital Library

[7]

Noboru Babaguchi, Yoshihiko Kawai, and Tadahiro Kitahashi. 2002. Event based indexing of broadcasted sports video by intermodal collaboration. IEEE transactions on Multimedia, Vol. 4, 1 (2002), 68--75.

Digital Library

[8]

Oscar Beijbom. 2012. Domain adaptations for computer vision applications. arXiv preprint arXiv:1211.4860 (2012).

[9]

Yoshua Bengio. 2012. Deep learning of representations for unsupervised and transfer learning. In Proceedings of ICML Workshop on Unsupervised and Transfer Learning. 17--36.

[10]

Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool. 2018. Domain adaptive faster r-cnn for object detection in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3339--3348.

[11]

Gabriela Csurka. 2017. Domain adaptation for visual applications: A comprehensive survey. arXiv preprint arXiv:1702.05374 (2017).

Digital Library

[12]

Gianpaolo Cugola and Alessandro Margara. 2012. Processing flows of information: From data stream to complex event processing. ACM Computing Surveys (CSUR), Vol. 44, 3 (2012), 15.

Digital Library

[13]

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 248--255.

[14]

Patrick Th Eugster, Pascal A Felber, Rachid Guerraoui, and Anne-Marie Kermarrec. 2003. The many faces of publish/subscribe. ACM Computing Surveys (CSUR), Vol. 35, 2 (2003), 114--131.

Digital Library

[15]

M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. 2010. The Pascal Visual Object Classes (VOC) Challenge. International Journal of Computer Vision, Vol. 88, 2 (June 2010), 303--338.

Digital Library

[16]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francc ois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, Vol. 17, 1 (2016), 2096--2030.

Digital Library

[17]

Holger Glasl, David Schreiber, Nikolaus Viertl, Stephan Veigl, and Gustavo Fernandez. 2008. Video based traffic congestion prediction on an embedded system. In Intelligent Transportation Systems, 2008. ITSC 2008. 11th International IEEE Conference on. IEEE, 950--955.

[18]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[19]

Judith Hoffman. 2016. Adaptive learning algorithms for transferable visual recognition .University of California, Berkeley.

[20]

Judy Hoffman, Sergio Guadarrama, Eric S Tzeng, Ronghang Hu, Jeff Donahue, Ross Girshick, Trevor Darrell, and Kate Saenko. 2014. LSDA: Large scale detection through adaptation. In Advances in Neural Information Processing Systems. 3536--3544.

[21]

Ling Hu and Qiang Ni. 2017. IoT-driven automated object detection algorithm for urban surveillance systems in smart cities. IEEE Internet of Things Journal, Vol. 5, 2 (2017), 747--754.

[22]

Jermsak Jermsurawong, Mian Umair Ahsan, Abdulhamid Haidar, Haiwei Dong, and Nikolaos Mavridis. 2012. Car parking vacancy detection and its application in 24-hour statistical analysis. In 2012 10th International Conference on Frontiers of Information Technology. IEEE, 84--90.

Digital Library

[23]

Ivan Krasin, Tom Duerig, Neil Alldrin, Vittorio Ferrari, Sami Abu-El-Haija, Alina Kuznetsova, Hassan Rom, Jasper Uijlings, Stefan Popov, Andreas Veit, et al. 2017. Openimages: A public dataset for large-scale multi-label and multi-class image classification. Dataset available from https://github. com/openimages, Vol. 2 (2017), 3.

[24]

Malaram Kumhar, Gaurang Raval, and Vishal Parikh. 2019. Quality Evaluation Model for Multimedia Internet of Things (MIoT) Applications: Challenges and Research Directions. In International Conference on Internet of Things and Connected Technologies. Springer, 330--336.

[25]

Mikolaj E Kundegorski, Samet Akcc ay, Michael Devereux, Andre Mouton, and Toby P Breckon. 2016. On using feature descriptors as visual words for object detection within x-ray baggage security screening. (2016).

[26]

Ching-Hao Lai and Chia-Chen Yu. 2010. An efficient real-time traffic sign recognition system for intelligent vehicles with smart phones. In Technologies and Applications of Artificial Intelligence (TAAI), 2010 International Conference on. IEEE, 195--202.

Digital Library

[27]

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision. 2980--2988.

[28]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 740--755.

[29]

Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. 2016. Ssd: Single shot multibox detector. In European conference on computer vision. Springer, 21--37.

[30]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan. 2015. Learning transferable features with deep adaptation networks. arXiv preprint arXiv:1502.02791 (2015).

[31]

Laura Lopez-Fuentes, Joost van de Weijer, Marc Bolanos, and Harald Skinnemoen. 2017. Multi-modal Deep Learning Approach for Flood Detection. In MediaEval.

[32]

Badri Mohapatra and Prangya Prava Panda. 2019. Machine learning applications to smart city. ACCENTS Transactions on Image Processing and Computer Vision, Vol. 4 (14) (Feb 2019). https://doi.org/10.19101/TIPCV.2018.412004

[33]

Pirkko Mustamo. 2018. Object detection in sports: TensorFlow Object Detection API case study .University of Oulu.

[34]

Ali Nauman, Yazdan Ahmad Qadri, Muhammad Amjad, Yousaf Bin Zikria, Muhammad Khalil Afzal, and Sung Won Kim. 2020. Multimedia Internet of Things: A Comprehensive Survey. IEEE Access, Vol. 8 (2020), 8202--8250.

[35]

Sinno Jialin Pan, Qiang Yang, et al. 2010. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, Vol. 22, 10 (2010), 1345--1359.

Digital Library

[36]

Ted Pedersen, Siddharth Patwardhan, and Jason Michelizzi. 2004. WordNet:: Similarity: measuring the relatedness of concepts. In Demonstration papers at HLT-NAACL 2004. Association for Computational Linguistics, 38--41.

Digital Library

[37]

Joseph Redmon. 2013--2016. Darknet: Open Source Neural Networks in C. http://pjreddie.com/darknet/.

[38]

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 779--788.

[39]

Joseph Redmon and Ali Farhadi. 2016. YOLO9000: Better, Faster, Stronger. arXiv preprint arXiv:1612.08242 (2016).

[40]

Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. 2010. Adapting visual category models to new domains. In European conference on computer vision. Springer, 213--226.

Digital Library

[41]

Juan Carlos San Miguel and José M Mart'inez. 2008. Robust unattended and stolen object detection by fusing simple algorithms. In 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance. IEEE, 18--25.

Digital Library

[42]

Kah Phooi Seng and Li-Minn Ang. 2018. A Big Data Layered Architecture and Functional Units for the Multimedia Internet of Things (MIoT). IEEE Transactions on Multi-Scale Computing Systems (2018).

[43]

Chiao-Fe Shu, Arun Hampapur, Max Lu, Lisa Brown, Jonathan Connell, Andrew Senior, and Yingli Tian. 2005. Ibm smart surveillance system (s3): a open and extensible framework for event based surveillance. In Advanced Video and Signal Based Surveillance, 2005. AVSS 2005. IEEE Conference on. IEEE, 318--323.

[44]

Javier Silvestre-Blanes, V'ictor Sempere-Payá, and Teresa Albero-Albero. 2020. Smart Sensor Architectures for Multimedia Sensing in IoMT. Sensors, Vol. 20, 5 (2020), 1400.

[45]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[46]

Baochen Sun, Jiashi Feng, and Kate Saenko. 2016. Return of frustratingly easy domain adaptation. In AAAI, Vol. 6. 8.

Digital Library

[47]

Yong Tang, Congzhe Zhang, Renshu Gu, Peng Li, and Bin Yang. 2017. Vehicle detection and recognition for intelligent traffic surveillance system. Multimedia tools and applications, Vol. 76, 4 (2017), 5817--5832.

[48]

Limin Wang, Zhe Wang, Yu Qiao, and Luc Van Gool. 2018. Transferring deep object and scene representations for event recognition in still images. International Journal of Computer Vision, Vol. 126, 2--4 (2018), 390--409.

Digital Library

[49]

Mei Wang and Weihong Deng. 2018. Deep visual domain adaptation: A survey. Neurocomputing, Vol. 312 (2018), 135--153.

Digital Library

[50]

Xiu-Shen Wei, Bin-Bin Gao, and Jianxin Wu. 2015. Deep spatial pyramid ensemble for cultural event recognition. In Proceedings of the IEEE international conference on computer vision workshops. 38--44.

Digital Library

[51]

Piyush Yadav and Edward Curry. 2019. VidCEP: Complex Event Processing Framework to Detect Spatiotemporal Patterns in Video Streams. In 2019 IEEE International Conference on Big Data (Big Data). IEEE, 2513--2522.

[52]

Yuhao Zhang and Arun Kumar. 2019. Panorama: a data system for unbounded vocabulary querying over video. Proceedings of the VLDB Endowment, Vol. 13, 4 (2019), 477--491.

Digital Library

Cited By

Ru JTian JXiao CLi JShen H(2024)Imbalanced Open Set Domain Adaptation via Moving-Threshold Estimation and Gradual AlignmentIEEE Transactions on Multimedia10.1109/TMM.2023.329776826(2504-2514)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3297768
Jin YJiang WYang YMu Y(2022)Zero-Shot Video Event Detection With High-Order Semantic Concept Discovery and MatchingIEEE Transactions on Multimedia10.1109/TMM.2021.307362424(1896-1908)Online publication date: 2022
https://doi.org/10.1109/TMM.2021.3073624
Aslam AGurrin CÞór Jónsson BKando NSchoeffmann KChen PO'Connor N(2020)Object Detection for Unseen Domains while Reducing Response Time using Knowledge Transfer in Multimedia Event ProcessingProceedings of the 2020 International Conference on Multimedia Retrieval10.1145/3372278.3391936(373-377)Online publication date: 8-Jun-2020
https://dl.acm.org/doi/10.1145/3372278.3391936
Show More Cited By

Index Terms

Reducing Response Time for Multimedia Event Processing using Domain Adaptation

Recommendations

Object Detection for Unseen Domains while Reducing Response Time using Knowledge Transfer in Multimedia Event Processing
ICMR '20: Proceedings of the 2020 International Conference on Multimedia Retrieval

Event recognition is among one of the popular areas of smart cities that has attracted great attention for researchers. Since Internet of Things (IoT) is mainly focused on scalar data events, research is shifting towards the Internet of Multimedia ...
Investigating response time and accuracy in online classifier learning for multimedia publish-subscribe systems
Abstract
The enormous growth of multimedia content in the field of the Internet of Things (IoT) leads to the challenge of processing multimedia streams in real-time. Event-based systems are constructed to process event streams. They cannot natively consume ...
Feature-level domain adaptation

Domain adaptation is the supervised learning setting in which the training and test data are sampled from different distributions: training data is sampled from a source domain, whilst test data is sampled from a target domain. This paper proposes and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '20: Proceedings of the 2020 International Conference on Multimedia Retrieval

June 2020

605 pages

ISBN:9781450370875

DOI:10.1145/3372278

General Chairs:
Cathal Gurrin
Dublin City University, Ireland
,
Björn Þór Jónsson
IT University of Copenhagen, Denmark
,
Noriko Kando
National Institute of Informatics, Tokyo
,
Program Chairs:
Klaus Schoeffmann
Klagenfurt University, Austria
,
Phoebe Chen
La Trobe University, Australia
,
Noel E. O'Connor
Dublin City University, Ireland

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

ICMR '20

Sponsor:

SIGMM

ICMR '20: International Conference on Multimedia Retrieval

June 8 - 11, 2020

Dublin, Ireland

Acceptance Rates

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
217
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)17

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ru JTian JXiao CLi JShen H(2024)Imbalanced Open Set Domain Adaptation via Moving-Threshold Estimation and Gradual AlignmentIEEE Transactions on Multimedia10.1109/TMM.2023.329776826(2504-2514)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3297768
Jin YJiang WYang YMu Y(2022)Zero-Shot Video Event Detection With High-Order Semantic Concept Discovery and MatchingIEEE Transactions on Multimedia10.1109/TMM.2021.307362424(1896-1908)Online publication date: 2022
https://doi.org/10.1109/TMM.2021.3073624
Aslam AGurrin CÞór Jónsson BKando NSchoeffmann KChen PO'Connor N(2020)Object Detection for Unseen Domains while Reducing Response Time using Knowledge Transfer in Multimedia Event ProcessingProceedings of the 2020 International Conference on Multimedia Retrieval10.1145/3372278.3391936(373-377)Online publication date: 8-Jun-2020
https://dl.acm.org/doi/10.1145/3372278.3391936
Paul A(2020)Recent Advances in Selective Image Encryption and its Indispensability due to COVID-192020 IEEE Recent Advances in Intelligent Computational Systems (RAICS)10.1109/RAICS51191.2020.9332513(201-206)Online publication date: 3-Dec-2020
https://doi.org/10.1109/RAICS51191.2020.9332513

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents