An Automatic Near-Duplicate Video Data Cleaning Method Based on a Consistent Feature Hash Ring
Abstract
:1. Introduction
2. Related Work
2.1. Near-Duplicate Video Detection Methodologies
2.2. Data Cleaning Methodologies
3. The Proposed Method
3.1. High-Dimensional Feature Extraction of Videos
3.2. The Construction of a Consistent Feature Hash Ring
3.3. FD-Means Clustering Cleaning Optimization Algorithm with Fused Mountain Peak Function
4. Experimental Evaluation
4.1. Dataset and Evaluation Criteria
4.2. Experimental Results and Analysis
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Tan, H.-K.; Ngo, C.-W.; Hong, R.; Chua, T.-S. Scalable detection of partial near-duplicate videos by visual-temporal consistency. In Proceedings of the 17th ACM International Conference on Multimedia, Beijing, China, 19–23 October 2009; pp. 145–154. [Google Scholar]
- Basharat, A.; Zhai, Y.; Shah, M. Content based video matching using spatiotemporal volumes. Comput. Vis. Image Underst. 2008, 110, 360–377. [Google Scholar] [CrossRef]
- Zhang, D.-K.; Sun, Z.-H.; Jia, K.-B. Near-duplicate video detection based on temporal and spatial key points. Smart Innov. Syst. Technol. 2020, 180, 129–137. [Google Scholar]
- Nie, X.; Qiao, J.; Liu, J.; Sun, J.; Li, X.; Liu, W. LLE-based video hashing for video identification. In Proceedings of the IEEE 10th International Conference on Signal Processing, Beijing, China, 24–28 October 2010; pp. 1837–1840. [Google Scholar]
- Liu, J.; Huang, Z.; Cai, H.; Shen, H.T.; Ngo, C.W.; Wang, W. Near-duplicate video retrieval: Current research and future trends. ACM Comput. Surv. 2013, 45, 1–23. [Google Scholar] [CrossRef]
- Carvalho, C.; Moreira, R.S.; Jose, J.M. Data quality visual analysis (DQVA) A tool to process and pin spot raw data irregularities. In Proceedings of the 2021 IEEE 11th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA, 27–30 January 2021; pp. 1036–1045. [Google Scholar]
- Phalke, D.-A.; Jahirabadkar, S. A survey on near-duplicate video retrieval using deep learning techniques and framework. In Proceedings of the 2020 IEEE Pune Section International Conference (PuneCon), Pune, India, 16–18 December 2020; pp. 124–128. [Google Scholar]
- Hu, Y.; Lu, X. Learning spatial-temporal features for video copy detection by the combination of CNN and RNN. J. Vis. Commun. Image Represent. 2018, 55, 21–29. [Google Scholar] [CrossRef]
- Abdu, S.A.; Yousef, A.H.; Salem, A. Multimodal Video Sentiment Analysis Using Deep Learning Approaches, a Survey. Inf. Fusion 2021, 76, 204–226. [Google Scholar] [CrossRef]
- Mohiuddin, S.; Malakar, S.; Sarkar, R. Duplicate frame detection in forged videos using sequence matching. In Proceedings of the Communications Computational Information Science, Santiniketan, India, 7–8 January 2021; pp. 29–41. [Google Scholar]
- Shen, L.; Hong, R.; Hao, Y. Advance on large scale near-duplicate video retrieval. Front. Comput. Sci. 2020, 14, 145702. [Google Scholar] [CrossRef]
- Wang, X.-L.; Luo, J.-S.; Xu, T.; Zeng, G.-B. Research on electricity characteristic recognition method of clean heating based on the big data model. In Proceedings of the Lecture Notes Institutional Computational Science Society Informatics Telecommunication Engineering, Binzhou, China, 26–27 September 2020; pp. 25–35. [Google Scholar]
- Zhou, H.-C.; Li, M.-H.; Gu, Z.-Q. Knowledge fusion and spatiotemporal data cleaning: A review. In Proceedings of the 7th IEEE International Conference on Data Science in Cyberspace, Hong Kong, China, 11–13 July 2020; pp. 295–301. [Google Scholar]
- Zheng, Z. Contextual data cleaning with ontology FDs. In Proceedings of the ACM SIGMOD International Conference Management Data, Xi’an, China, 20–25 June 2021; pp. 2911–2913. [Google Scholar]
- Ye, O.; Li, Z.-L.; Zhang, Y. Near-duplicate video cleansing method based on locality sensitive hashing and the sorted neighborhood method. In Proceedings of the EAI International Conference on Robotic Sensor Networks, Kitakyushu, Japan, 17 August 2019; pp. 129–139. [Google Scholar]
- Zhu, Y.; Huang, X.; Huang, Q.; Tian, Q. Large-scale video copy retrieval with temporal-concentration SIFT. Neurocomputing 2016, 187, 83–91. [Google Scholar] [CrossRef]
- Henderson, C.; Lzquierdo, E. Robust feature matching in long-running poor-quality videos. IEEE Trans. Circuits Syst. Video Technol. 2016, 26, 1161–1174. [Google Scholar] [CrossRef]
- Zhang, C.; Liu, D.; Wu, X.; Zhao, G.; Shyu, M.-L.; Peng, Q. Near-duplicate segments-based news web video event mining. Signal Process. 2016, 120, 26–35. [Google Scholar] [CrossRef]
- Chen, Y.; Li, D.; Hua, Y.; He, W. Effective and efficient content redundancy detection of web videos. IEEE Trans. Big Data 2021, 7, 187–198. [Google Scholar] [CrossRef]
- Nie, X.; Jing, W.; Cui, C.; Zhang, C.J.; Zhu, L.; Yin, Y. Joint multi-view hashing for large-scale near-duplicate video retrieval. IEEE Trans. Knowl. Data Eng. 2020, 32, 1951–1965. [Google Scholar] [CrossRef]
- Wang, K.H.; Cheng, C.C.; Chen, Y.L.; Song, Y.; Lai, S.H. Attention-based deep metric learning for near-duplicate video retrieval. In Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy, 10–15 January 2021; pp. 5360–5367. [Google Scholar]
- Li, S.; Chen, Z.; Lu, J.; Li, X.; Zhou, J. Neighborhood preserving hashing for scalable video retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 8211–8220. [Google Scholar]
- Ai, X.; He, Y.; Hu, Y.; Tian, W. Inter-frame relationship graph based near-duplicate video clip detection method. In Proceedings of the 14th Conference on Image and Graphics Technologies and Applications, IGTA 2019, Beijing, China, 19–20 April 2019; pp. 70–79. [Google Scholar]
- Chen, H.; Hu, C.; Lee, F.; Lin, C.; Yao, W.; Chen, L.; Chen, Q. A supervised video hashing method based on a deep 3d convolutional neural network for large-scale video retrieval. Sensors 2021, 21, 3094. [Google Scholar] [CrossRef] [PubMed]
- Hu, Y.-Z.; Mu, Z.-K.; Ai, X.-B. STRNN: End-to-end deep learning framework for video partial copy detection. J. Phys. Conf. Ser. 2019, 1237, 022112. [Google Scholar] [CrossRef]
- Mohamed, Z.H.; Vinila, J.S. A comparative study on data cleaning approaches in sentiment analysis. In Proceedings of the Lecture Notes Electrical Engineering, Thiruvananthapuram, India, 14–17 December 2019; pp. 421–431. [Google Scholar]
- Ying, K.-H.; Hu, W.-Y.; Chen, J.-B.; Li, G.-N. Research on instance-level data cleaning technology. In Proceedings of the 2021 International Conference on Artificial Intelligence, Big Data and Algorithms, Xi’an, China, 28–30 May 2021; pp. 238–242. [Google Scholar]
- Song, S.; Zhang, A.; Wang, J.; Yu, P.S. SCREEN: Stream data cleaning under speed constraints. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, Melbourne, VIC, Australia, 31 May–June 4 2015; pp. 827–841. [Google Scholar]
- Tian, Y.C.; Michiardi, P.; Vukolic, M. Bleach: A distributed stream data cleaning system. In Proceedings of the 2017 IEEE International Congress on Big Data (BigData Congress), Honolulu, HI, USA, 25–30 June 2017; pp. 113–120. [Google Scholar]
- Qiaomin, L.; Anqi, F.; Min, P.; Qiang, X.; Kun, D.; Michael, S. Cleaning RFID data streams based on l-means clustering method. J. China Univ. Post Telecom. 2020, 27, 72–81. [Google Scholar]
- Ranjan, K.G.; Prusty, B.R.; Jena, D. Comparison of two data cleaning methods as applied to volatile time-series. In Proceedings of the 2019 International Conference on Power Electronics Applications and Technology, Mangalore, India, 29–31 August 2019; pp. 1–6. [Google Scholar]
- Zhu, L.; Du, X.; Ma, Q.; Meng, W.; Liu, H. Keyword search with real-time entity resolution in relational databases. In Proceedings of the 2018 10th International Conference on Machine Learning and Computing, Macau, China, 26–28 February 2018; pp. 134–139. [Google Scholar]
- Aissani, A.; Yi, E.T.J.; Thamilyanan, T. End to end real-time data cleansing, standardization and reliability towards a smooth digital oil field deployment. In Proceedings of the Offshore Technology Conference Asia, Kuala Lumpur, Malaysia, 2–6 November 2020; pp. 1–10. [Google Scholar]
- Zheng, Z.; Milani, M.; Chiang, F. CurrentClean: Spatio-temporal cleaning of stale data. In Proceedings of the 2019 IEEE 35th International Conference on Data Engineering (ICDE), Macau, China, 8–11 April 2019; pp. 172–183. [Google Scholar]
- Zhang, Y.; Jin, Z.; Liu, F.; Zhu, W.; Mu, W.; Wang, W. ImageDC: Image data cleaning framework based on deep learning. In Proceedings of the 2020 IEEE International Conference on Artificial Intelligence and Information Systems (ICAIIS), Dalian, China, 20–22 March 2020; pp. 748–752. [Google Scholar]
- Fu, Y.; Han, Z.; Ye, O. FD-means clustering cleaning algorithm for near-duplicate videos. Comput. Eng. Appl. 2022, 58, 197–203. [Google Scholar]
- Jia, W.; Li, L.; Li, Z.; Zhao, S.; Liu, S. Scalable hash from triplet loss feature aggregation for video de-duplication. J. Vis. Commun. Image Represent. 2020, 72, 102908. [Google Scholar] [CrossRef]
- Chen, H.; Li, J.; Frintrop, S.; Hu, X. The MSR-video to text dataset with clean annotations. Comput. Vis. Image Understand. 2022, 225, 103581. [Google Scholar] [CrossRef]
- Cheng, H.-Y.; Yu, C.-C. Automatic data cleaning system for large-scale location image database using a multilevel extractor and multiresolution dissimilarity calculation. IEEE Intell. Syst. 2021, 36, 49–56. [Google Scholar] [CrossRef]
- Ye, O.; Deng, J.; Yu, Z.H.; Liu, T.; Dong, L.H. Abnormal event detection via feature expectation subgraph calibrating classification in video surveillance scenes. IEEE Access 2020, 8, 97564–97575. [Google Scholar] [CrossRef]
- Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. CBAM: Convolution block attention module. In Proceedings of the 15th European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Wang, Z. Research on garbage image classification based on neural network. In Proceedings of the 2022 International Conference on Computer Network, Electronic and Automation (ICCNEA), Xi’an, China, 23–25 September 2022; pp. 214–217. [Google Scholar]
- Kordopatis-Zilos, G.; Papadopoulos, S.; Patras, I.; Kompatsiaris, Y. Near-Duplicate Video Retrieval by Aggregating Intermediate CNN Layers. In Proceedings of the 23rd International Conference on MultiMedia Modeling, Reykjavik, Iceland, 4–6 January 2017; pp. 251–263. [Google Scholar]
The Number of Hidden Layers | Precision | Recall | F1-Score | Accuracy |
---|---|---|---|---|
4 | 0.5577 | 0.611 | 0.583 | 0.7 |
8 | 0.9375 | 0.9375 | 0.9375 | 0.95 |
16 | 0.9375 | 0.944 | 0.941 | 0.975 |
32 | 0.8375 | 0.8 | 0.818 | 0.925 |
Attention Size | Precision | Recall | F1-Score | Accuracy |
---|---|---|---|---|
4 | 0.7944 | 0.86 | 0.826 | 0.9 |
8 | 0.8375 | 0.9 | 0.868 | 0.925 |
16 | 0.8819 | 0.944 | 0.912 | 0.95 |
32 | 0.9375 | 0.978 | 0.941 | 0.975 |
64 | 0.85 | 0.9 | 0.874 | 0.925 |
Models | CC_WEB_VIDEO Dataset | |||
---|---|---|---|---|
Precision | Recall | F1-Score | Accuracy | |
Spatiotemporal Keypoint [3] | 0.61 | 0.96 | 0.75 | 0.64 |
BS-VGG16 [36] | 0.79 | 0.92 | 0.85 | 0.85 |
LBoW [43] | 0.63 | 0.85 | 0.72 | 0.66 |
MLE-MRD [39] | 0.82 | 0.91 | 0.86 | 0.87 |
CBAM-Resnet [42] | 0.77 | 0.92 | 0.84 | 0.88 |
3D-CNN [24] | 0.88 | 0.76 | 0.84 | 0.93 |
RCLA | 0.93 | 0.94 | 0.94 | 0.95 |
Models | The Coal Mine Video Dataset | |||
---|---|---|---|---|
Precision | Recall | F1-Score | Accuracy | |
Spatiotemporal Keypoint [3] | 0.57 | 0.85 | 0.68 | 0.61 |
BS-VGG16 [36] | 0.72 | 0.83 | 0.77 | 0.79 |
LBoW [43] | 0.60 | 0.92 | 0.73 | 0.72 |
MLE-MRD [39] | 0.85 | 0.84 | 0.84 | 0.87 |
CBAM-Resnet [42] | 0.79 | 0.86 | 0.82 | 0.84 |
3D-CNN [24] | 0.91 | 0.89 | 0.90 | 0.90 |
RCLA | 0.93 | 0.94 | 0.93 | 0.92 |
Methods | Cluster Cleaning | CC_WEB_VIDEO Dataset | The Coal Mine Video Dataset | ||||
---|---|---|---|---|---|---|---|
Acc | Rec | F1-Score | Acc | Rec | F1-Score | ||
Spatiotemporal Keypoint [3] | K-Means | 0.4527 | 0.451 | 0.451 | 0.466 | 0.5333 | 0.497 |
FD-Means | 0.4776 | 0.538 | 0.506 | 0.666 | 0.5333 | 0.592 | |
FD-Means fused with the peak function | 0.522 | 0.835 | 0.612 | 0.857 | 0.6 | 0.706 | |
LBoW [43] | K-Means | 0.453 | 0.472 | 0.462 | 0.5 | 0.5333 | 0.516 |
FD-Means | 0.587 | 0.615 | 0.601 | 0.733 | 0.733 | 0.733 | |
FD-Means fused with the peak function | 0.572 | 0.813 | 0.632 | 0.833 | 0.5 | 0.625 | |
BS-VGG16 [36] | K-Means | 0.275 | 0.615 | 0.436 | 0.465 | 0.362 | 0.382 |
FD-Means | 0.650 | 0.929 | 0.76 | 0.667 | 0.833 | 0.74 | |
FD-Means fused with the peak function | 0.49 | 0.967 | 0.633 | 0.5 | 0.4 | 0.444 | |
MLE-MRD [39] | K-Means | 0.53 | 0.62 | 0.57 | 0.57 | 0.65 | 0.61 |
FD-Means | 0.72 | 0.79 | 0.75 | 0.69 | 0.76 | 0.72 | |
FD-Means fused with the peak function | 0.76 | 0.82 | 0.79 | 0.75 | 0.86 | 0.80 | |
CBAM-Resnet [42] | K-Means | 0.423 | 0.56 | 0.481 | 0.5333 | 0.666 | 0.592 |
FD-Means | 0.587 | 0.615 | 0.601 | 0.733 | 0.733 | 0.733 | |
FD-Means fused with the peak function | 0.825 | 0.681 | 0.779 | 0.777 | 0.7 | 0.736 | |
3D-CNN [24] | K-Means | 0.75 | 0.75 | 0.75 | 0.71 | 0.91 | 0.80 |
FD-Means | 0.80 | 0.69 | 0.74 | 0.87 | 0.70 | 0.77 | |
FD-Means fused with the peak function | 0.875 | 0.7 | 0.778 | 0.936 | 0.723 | 0.816 | |
RCLA-HAOPFDMC | K-Means | 0.672 | 0.67 | 0.671 | 0.733 | 0.733 | 0.733 |
FD-Means | 0.901 | 0.802 | 0.848 | 0.864 | 0.9333 | 0.897 | |
FD-Means fused with the peak function | 0.914 | 0.801 | 0.854 | 0.872 | 0.9333 | 0.902 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Qin, Y.; Ye, O.; Fu, Y. An Automatic Near-Duplicate Video Data Cleaning Method Based on a Consistent Feature Hash Ring. Electronics 2024, 13, 1522. https://doi.org/10.3390/electronics13081522
Qin Y, Ye O, Fu Y. An Automatic Near-Duplicate Video Data Cleaning Method Based on a Consistent Feature Hash Ring. Electronics. 2024; 13(8):1522. https://doi.org/10.3390/electronics13081522
Chicago/Turabian StyleQin, Yi, Ou Ye, and Yan Fu. 2024. "An Automatic Near-Duplicate Video Data Cleaning Method Based on a Consistent Feature Hash Ring" Electronics 13, no. 8: 1522. https://doi.org/10.3390/electronics13081522