RainyTrack: Enhancing Object Tracking in Adverse Weather Conditions with Siamese Networks
January 2024
Pages 308 - 315
Abstract
Object tracking is a critical task in the field of computer vision, playing a significant role in many practical applications. However, in complex rainy conditions, existing object tracking methods often perform poorly. This is due to rain introducing noise, blurriness, and occlusion in images, thereby reducing target visibility and tracking accuracy. Furthermore, the lack of large-scale training datasets specifically designed for rainy conditions limits the performance of existing methods in this environment. To address this issue, we propose a novel object tracker focused on achieving efficient and accurate object tracking in rainy conditions. We have enhanced traditional object tracking methods by introducing the "rain removal" concept and modifying feature fusion and attention mechanisms to improve the tracker’s performance in rainy conditions. Additionally, we synthesized rain on three commonly used object tracking datasets, LaSOT, OTB2015, and UAV123, incorporating varying degrees of rainfall and raindrop sizes to simulate real-world rainy conditions. Through extensive experiments on multiple publicly available datasets, we have validated the effectiveness of the proposed method. The experimental results demonstrate that our tracker excels not only on the original datasets but also maintains outstanding tracking performance on the rain-added datasets. This contribution provides new insights and methods for the advancement of the computer vision field.
References
[1]
Chris Holmberg Bahnsen, David Vázquez, Antonio M López, and Thomas B Moeslund. 2019. Learning to remove rain in traffic surveillance by using synthetic data. In 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (Visigrapp 2019). SCITEPRESS Digital Library, 123–130.
[2]
Luca Bertinetto, Jack Valmadre, Joao F Henriques, Andrea Vedaldi, and Philip HS Torr. 2016. Fully-convolutional siamese networks for object tracking. In Computer Vision–ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part II 14. Springer, 850–865.
[3]
Goutam Bhat, Martin Danelljan, Luc Van Gool, and Radu Timofte. 2019. Learning discriminative model prediction for tracking. In Proceedings of the IEEE/CVF international conference on computer vision. 6182–6191.
[4]
Mario Bijelic, Tobias Gruber, Fahim Mannan, Florian Kraus, Werner Ritter, Klaus Dietmayer, and Felix Heide. 2020. Seeing through fog without seeing fog: Deep multimodal sensor fusion in unseen adverse weather. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11682–11692.
[5]
David S Bolme, J Ross Beveridge, Bruce A Draper, and Yui Man Lui. 2010. Visual object tracking using adaptive correlation filters. In 2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, 2544–2550.
[6]
Ziang Cao, Ziyuan Huang, Liang Pan, Shiwei Zhang, Ziwei Liu, and Changhong Fu. 2022. TCTrack: Temporal contexts for aerial tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14798–14808.
[7]
Xin Chen, Bin Yan, Jiawen Zhu, Dong Wang, Xiaoyun Yang, and Huchuan Lu. 2021. Transformer tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8126–8135.
[8]
Zedu Chen, Bineng Zhong, Guorong Li, Shengping Zhang, and Rongrong Ji. 2020. Siamese box adaptive network for visual tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6668–6677.
[9]
Yutao Cui, Cheng Jiang, Limin Wang, and Gangshan Wu. 2021. Target transformed regression for accurate tracking. arXiv preprint arXiv:2104.00403 (2021).
[10]
Yutao Cui, Cheng Jiang, Limin Wang, and Gangshan Wu. 2022. Mixformer: End-to-end tracking with iterative mixed attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13608–13618.
[11]
Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg. 2019. Atom: Accurate tracking by overlap maximization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 4660–4669.
[12]
Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg. 2017. Eco: Efficient convolution operators for tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition. 6638–6646.
[13]
Martin Danelljan, Luc Van Gool, and Radu Timofte. 2020. Probabilistic regression for visual tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 7183–7192.
[14]
Ratnadeep Dey, Debotosh Bhattacharjee, and Mita Nasipuri. 2020. Object detection in rainy condition from video using YOLO based deep learning model. Advanced Computing and Systems for Security: Volume Twelve (2020), 121–131.
[15]
Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, and Haibin Ling. 2019. Lasot: A high-quality benchmark for large-scale single object tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5374–5383.
[16]
Zhihong Fu, Qingjie Liu, Zehua Fu, and Yunhong Wang. 2021. Stmtrack: Template-free visual tracking with space-time memory networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 13774–13783.
[17]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 249–256.
[18]
Dongyan Guo, Yanyan Shao, Ying Cui, Zhenhua Wang, Liyan Zhang, and Chunhua Shen. 2021. Graph attention tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9543–9552.
[19]
Dongyan Guo, Jun Wang, Ying Cui, Zhenhua Wang, and Shengyong Chen. 2020. SiamCAR: Siamese fully convolutional classification and regression for visual tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6269–6277.
[20]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
[21]
Joao F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. 2012. Exploiting the circulant structure of tracking-by-detection with kernels. In Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part IV 12. Springer, 702–715.
[22]
João F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. 2014. High-speed tracking with kernelized correlation filters. IEEE transactions on pattern analysis and machine intelligence 37, 3 (2014), 583–596.
[23]
Lianghua Huang, Xin Zhao, and Kaiqi Huang. 2019. Got-10k: A large high-diversity benchmark for generic object tracking in the wild. IEEE transactions on pattern analysis and machine intelligence 43, 5 (2019), 1562–1577.
[24]
WANG Jiangang, CHEN Simonjian, ZHOU Lubing, 2018. Vehicle detection and width estimation in rain by fusing radar and vision. In Proceed 2018 15th Int’l Conf Contr, Autom, Robot Vision (ICARCV).
[25]
Dirk Langer and Todd Jochem. 1996. Fusing radar and vision for detecting, classifying and avoiding roadway obstacles. In Proceedings of Conference on Intelligent Vehicles. IEEE, 333–338.
[26]
Bo Li, Junjie Yan, Wei Wu, Zheng Zhu, and Xiaolin Hu. 2018. High performance visual tracking with siamese region proposal network. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8971–8980.
[27]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Springer, 740–755.
[28]
Liwei Liu, Junliang Xing, Haizhou Ai, and Xiang Ruan. 2012. Hand posture recognition using finger geometric feature. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012). IEEE, 565–568.
[29]
Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017).
[30]
Alan Lukezic, Tomas Vojir, Luka ˇCehovin Zajc, Jiri Matas, and Matej Kristan. 2017. Discriminative correlation filter with channel and spatial reliability. In Proceedings of the IEEE conference on computer vision and pattern recognition. 6309–6318.
[31]
Christoph Mayer, Martin Danelljan, Goutam Bhat, Matthieu Paul, Danda Pani Paudel, Fisher Yu, and Luc Van Gool. 2022. Transforming model prediction for tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8731–8740.
[32]
Christoph Mayer, Martin Danelljan, Danda Pani Paudel, and Luc Van Gool. 2021. Learning target candidate association to keep track of what not to track. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 13444–13454.
[33]
Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S Ecker, Matthias Bethge, and Wieland Brendel. 2019. Benchmarking robustness in object detection: Autonomous driving when winter is coming. arXiv preprint arXiv:1907.07484 (2019).
[34]
Matthias Mueller, Neil Smith, and Bernard Ghanem. 2016. A benchmark and simulator for uav tracking. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14. Springer, 445–461.
[35]
Matthias Muller, Adel Bibi, Silvio Giancola, Salman Alsubaihi, and Bernard Ghanem. 2018. Trackingnet: A large-scale dataset and benchmark for object tracking in the wild. In Proceedings of the European conference on computer vision (ECCV). 300–317.
[36]
Felix Nobis, Maximilian Geisslinger, Markus Weber, Johannes Betz, and Markus Lienkamp. 2019. A deep learning-based radar and camera sensor fusion architecture for object detection. In 2019 Sensor Data Fusion: Trends, Solutions, Applications (SDF). IEEE, 1–7.
[37]
Jinlong Peng, Zhengkai Jiang, Yueyang Gu, Yang Wu, Yabiao Wang, Ying Tai, Chengjie Wang, and Weiyao Lin. 2021. Siamrcr: Reciprocal classification and regression for visual object tracking. arXiv preprint arXiv:2105.11237 (2021).
[38]
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015).
[39]
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, 2015. Imagenet large scale visual recognition challenge. International journal of computer vision 115 (2015), 211–252.
[40]
Yibing Song, Chao Ma, Xiaohe Wu, Lijun Gong, Linchao Bao, Wangmeng Zuo, Chunhua Shen, Rynson WH Lau, and Ming-Hsuan Yang. 2018. Vital: Visual tracking via adversarial learning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8990–8999.
[41]
Zhi Tian, Chunhua Shen, Hao Chen, and Tong He. 2019. Fcos: Fully convolutional one-stage object detection. In Proceedings of the IEEE/CVF international conference on computer vision. 9627–9636.
[42]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, Illia Polosukhin, 2017. Advances in neural information processing systems. Attention is All you Need (2017).
[43]
Paul Voigtlaender, Jonathon Luiten, Philip HS Torr, and Bastian Leibe. 2020. Siam r-cnn: Visual tracking by re-detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6578–6588.
[44]
Ning Wang, Wengang Zhou, Jie Wang, and Houqiang Li. 2021. Transformer meets tracker: Exploiting temporal context for robust visual tracking. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1571–1580.
[45]
Qiang Wang, Li Zhang, Luca Bertinetto, Weiming Hu, and Philip HS Torr. 2019. Fast online object tracking and segmentation: A unifying approach. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition. 1328–1338.
[46]
Junliang Xing, Haizhou Ai, and Shihong Lao. 2010. Multiple human tracking based on multi-view upper-body detection and discriminative learning. In 2010 20th International Conference on Pattern Recognition. IEEE, 1698–1701.
[47]
Yinda Xu, Zeyu Wang, Zuoxin Li, Ye Yuan, and Gang Yu. 2020. SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines. In AAAI. 12549–12556.
[48]
Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, and Huchuan Lu. 2021. Learning spatio-temporal transformer for visual tracking. In Proceedings of the IEEE/CVF international conference on computer vision. 10448–10457.
[49]
Yi, Wu, Jongwoo, Lim, Ming-Hsuan, and Yang. 2015. Object Tracking Benchmark. IEEE transactions on pattern analysis and machine intelligence 37, 9 (2015), 1834–1848.
[50]
Zhipeng Zhang, Houwen Peng, Jianlong Fu, Bing Li, and Weiming Hu. 2020. Ocean: Object-aware anchor-free tracking. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXI 16. Springer, 771–787.
Index Terms
- RainyTrack: Enhancing Object Tracking in Adverse Weather Conditions with Siamese Networks
Recommendations
Robust object tracking via multi-cue fusion
A long-term object tracking method based on calibrated binocular cameras by fusing information of the two channels and binocular geometry constraints is proposed.The stereo filter which is built based on the epipolar geometry of the binocular cameras is ...
Template Attentional Siamese Network for Object Tracking
ICVIP '18: Proceedings of the 2018 2nd International Conference on Video and Image ProcessingRecent years, visual object tracking has attracted more and more attention as a fundamental topic. Many deep based trackers, especially Siamese Network based trackers, have achieved state-of-the-art performance on multiple benchmarks. However, most of ...
Hierarchical correlation siamese network for real-time object tracking
AbstractUnder the influence of deep learning, many trackers have emerged recently. Among them, Siamese network reaches a pleasant balance between accuracy and speed, but its tracking performance still lags behind other trackers. In this paper, we have ...
Comments
Information & Contributors
Information
Published In
January 2024
480 pages
ISBN:9798400716720
DOI:10.1145/3647649
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 03 May 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICIGP 2024
ICIGP 2024: 2024 the 7th International Conference on Image and Graphics Processing
January 19 - 21, 2024
Beijing, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 5Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)3
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format