Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/3552437.3555699acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

DeepSportradar-v1: Computer Vision Dataset for Sports Understanding with High Quality Annotations

Published: 10 October 2022 Publication History

Abstract

With the recent development of Deep Learning applied to Computer Vision, sport video understanding has gained a lot of attention, providing much richer information for both sport consumers and leagues. This paper introduces DeepSportradar-v1, a suite of computer vision tasks, datasets and benchmarks for automated sport understanding. The main purpose of this framework is to close the gap between academic research and real world settings. To this end, the datasets provide high-resolution raw images, camera parameters and high quality annotations. DeepSportradar currently supports four challenging tasks related to basketball: ball 3D localization, camera calibration, player instance segmentation and player re-identification. For each of the four tasks, a detailed description of the dataset, objective, performance metrics, and the proposed baseline method are provided. To encourage further research on advanced methods for sport understanding, a competition is organized as part of the MMSports workshop from the ACM Multimedia 2022 conference, where participants have to develop state-of-the-art methods to solve the above tasks. The four datasets, development kits and baselines are publicly available.

Supplementary Material

MP4 File (DeepSportradar-v1.mp4)
This video presents our paper Deepsportradar v1 Computer vision dataset for sports understanding with high quality annotations. Our aim is to close the gap between CV models and real applications for sports. Despite recent advancements in CV and DL; DL still depends on the quantity of data but also on the quality of the annotations. For sports the SoccerNet datasets that have received increasing attention for the amount of data provided. However it considers only soccer and the annotations are created on broadcasted videos. This paper introduces: two datasets for basketbal; four CV tasks: Ball 3D localization; Camera calibration; Player instance segmentation; Player re-identification; a toolkit on GitHub containing data, annotations and metrics; and a baseline for each task. The aim of this contribution was to provide an high quality sport dataset framework where images, camera parameters and annotations are available and built close to the actual game recordings.

References

[1]
Md Zahangir Alom et al. 2019. A state-of-the-art survey on deep learning theory and architectures. Electronics, 8, 3, 292.
[2]
Daniel Bolya, Chong Zhou, Fanyi Xiao, and Yong Jae Lee. 2019. Yolact: real-time instance segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, 9157--9166.
[3]
Jianhui Chen and James J Little. 2019. Sports camera calibration via synthetic data. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 0--0.
[4]
Kai Chen et al. 2019. MMDetection: open mmlab detection toolbox and benchmark. arXiv preprint arXiv:1906.07155.
[5]
Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. 2017. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587.
[6]
Bowen Cheng, Maxwell D Collins, Yukun Zhu, Ting Liu, Thomas S Huang, HartwigAdam, and Liang-Chieh Chen. 2020. Panoptic-deeplab: a simple, strong, and fast baseline for bottom-up panoptic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 12475--12485.
[7]
Anthony Cioppa, Adrien Deliege, Floriane Magera, Silvio Giancola, Olivier Barnich, Bernard Ghanem, and Marc Van Droogenbroeck. 2021. Camera calibration and player localization in soccernet-v2 and investigation of their representations for action spotting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4537--4546.
[8]
Anthony Cioppa, Adrien Deliège, Silvio Giancola, Bernard Ghanem, and Marc Van Droogenbroeck. 2022. Scaling up soccernet with multi-view spatial localization and re-identification. Scientific Data, 9, 1, 1--9.
[9]
Anthony Cioppa, Silvio Giancola, Adrien Deliege, Le Kang, Xin Zhou, Zhiyu Cheng, Bernard Ghanem, and Marc Van Droogenbroeck. 2022. Soccernettracking: multiple object tracking dataset and benchmark in soccer videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3491--3502.
[10]
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3213--3223.
[11]
Adrien Deliege, Anthony Cioppa, Silvio Giancola, Meisam J Seikavandi, Jacob V Dueholm, Kamal Nasrollahi, Bernard Ghanem, Thomas B Moeslund, and Marc Van Droogenbroeck. 2021. Soccernet-v2: a dataset and benchmarks for holistic understanding of broadcast soccer videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 4508--4519.
[12]
Dirk Farin, Susanne Krabbe, Wolfgang Effelsberg, et al. 2003. Robust camera calibration for sport videos using court models. In Storage and Retrieval Methods and Applications for Multimedia 2004. Vol. 5307. SPIE, 80--91.
[13]
Dengpan Fu, Dongdong Chen, Jianmin Bao, Hao Yang, Lu Yuan, Lei Zhang, Houqiang Li, and Dong Chen. 2021. Unsupervised pre-training for person reidentification. Proceedings of the IEEE conference on computer vision and pattern recognition.
[14]
Yixiao Ge, Feng Zhu, Dapeng Chen, Rui Zhao, and Hongsheng Li. 2020. Selfpaced contrastive learning with hybrid memory for domain adaptive object re-id. In Advances in Neural Information Processing Systems.
[15]
Silvio Giancola, Mohieddine Amine, Tarek Dghaily, and Bernard Ghanem. 2018. Soccernet: a scalable dataset for action spotting in soccer videos. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 1711--1721.
[16]
Agrim Gupta, Piotr Dollar, and Ross Girshick. 2019. Lvis: a dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 5356--5364.
[17]
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In 2017 IEEE International Conference on Computer Vision (ICCV), 2980--2988.
[18]
Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770--778.
[19]
Lingxiao He, Xingyu Liao, Wu Liu, Xinchen Liu, Peng Cheng, and Tao Mei. 2020. Fastreid: a pytorch toolbox for general instance re-identification. arXiv preprint arXiv:2006.02631.
[20]
Namdar Homayounfar, Sanja Fidler, and Raquel Urtasun. 2017. Sports field localization via deep structured models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5212--5220.
[21]
Peter J. Huber. 1964. Robust Estimation of a Location Parameter. The Annals of Mathematical Statistics, 35, 1, (Mar. 1964), 73--101. 32.
[22]
Paresh R Kamble, Avinash G Keskar, and Kishor M Bhurchandi. 2019. Ball tracking in sports: a survey. Artificial Intelligence Review, 52, 3, 1655--1705.
[23]
Lei Ke, Yu-Wing Tai, and Chi-Keung Tang. 2021. Deep occlusion-aware instance segmentation with overlapping bilayers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4019--4028.
[24]
Neeraj Kumar et al. 2019. A multi-organ nucleus segmentation challenge. IEEE transactions on medical imaging, 39, 5, 1380--1391.
[25]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature, 521, 7553, 436--444.
[26]
Attila Lengyel, Robert-Jan Bruintjes, Marcos Baptista Rios, Osman Semih Kayhan, Davide Zambrano, Nergis Tomen, and Jan van Gemert. 2022. Vipriors 2: visual inductive priors for data-efficient deep learning challenges. arXiv preprint arXiv:2201.08625.
[27]
Wei Li, Rui Zhao, Tong Xiao, and Xiaogang Wang. 2014. Deepreid: deep filter pairing neural network for person re-identification. In CVPR.
[28]
Yulin Li, Jianfeng He, Tianzhu Zhang, Xiang Liu, Yongdong Zhang, and Feng Wu. 2021. Diverse part discovery: occluded person re-identification with partaware transformer. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2897--2906.
[29]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: common objects in context. In European conference on computer vision. Springer, 740-- 755.
[30]
Hao Luo, Youzhi Gu, Xingyu Liao, Shenqi Lai, and Wei Jiang. 2019. Bag of tricks and a strong baseline for deep person re-identification. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 1487--1495.
[31]
Jiaxu Miao, Yu Wu, Ping Liu, Yuhang Ding, and Yi Yang. 2019. Pose-guided feature alignment for occluded person re-identification. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 542--551.
[32]
Massimo Minervini, Andreas Fischbach, Hanno Scharr, and Sotirios A Tsaftaris. 2016. Finely-grained annotated datasets for image-based plant phenotyping. Pattern recognition letters, 81, 80--89.
[33]
Davy Neven, Bert De Brabandere, Marc Proesmans, and Luc Van Gool. 2019. Instance segmentation by jointly optimizing spatial embeddings and clustering bandwidth. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8837--8845.
[34]
Niels Sayez and Christophe De Vleeschouwer. 2022. Accelerating the creation of instance segmentation training sets through bounding box annotation. (2022).
[35]
Long Sha, Jennifer Hobbs, Panna Felsen, Xinyu Wei, Patrick Lucey, and Sujoy Ganguly. 2020. End-to-end camera calibration for broadcast videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 13627--13636.
[36]
Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. International Conference on Learning Representations, ICLR, (Sept. 2015). https://arxi v.org/abs/1409.1556v6 arXiv: 1409.1556.
[37]
Yifan Sun, Liang Zheng, Yi Yang, Qi Tian, and Shengjin Wang. 2018. Beyond part models: person retrieval with refined part pooling. In ECCV.
[38]
Gabriel Van Zandycke. 2021. Deepsport dataset. https://www.kaggle.com/data sets/gabrielvanzandycke/deepsport-dataset. (2021).
[39]
Gabriel Van Zandycke and Christophe De Vleeschouwer. 2022. 3d ball localization from a single calibrated image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3472--3480.
[40]
Gabriel Van Zandycke and Christophe De Vleeschouwer. 2019. Real-time CNNbased segmentation architecture for ball detection in a single view setup. MMSports 2019 - Proceedings of the 2nd International Workshop on Multimedia Content Analysis in Sports, co-located withMM2019, 51--58. isbn: 9781450369114. arXiv: 2007.11876.
[41]
Guanshuo Wang, Yufeng Yuan, Xiong Chen, Jiwei Li, and Xi Zhou. 2018. Learning discriminative features with multiple granularities for person reidentification. Proceedings of the 26th ACM international conference on Multimedia.
[42]
Xiaogang Wang, Gianfranco Doretto, Thomas Sebastian, Jens Rittscher, and Peter Tu. 2007. Shape and appearance context modeling. In Rio de Janeiro, Brazil, 1--8.
[43]
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 1492--1500.
[44]
Qiang Yao, Akira Kubota, Kaoru Kawakita, Keisuke Nonaka, Hiroshi Sankoh, and Sei Naito. 2017. Fast camera self-calibration for synthesizing free viewpoint soccer video. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1612--1616.
[45]
Mang Ye, Jianbing Shen, Gaojie Lin, Tao Xiang, Ling Shao, and Steven C. H. Hoi. 2022. Deep learning for person re-identification: a survey and outlook. 44, 6, (June 2022), 2872--2893.
[46]
Xiaoding Yuan, Adam Kortylewski, Yihong Sun, and Alan Yuille. 2021. Robust instance segmentation through reasoning about multi-object occlusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11141--11150.
[47]
Liang Zheng, Zhi Bie, Yifan Sun, Jingdong Wang, Chi Su, Shengjin Wang, and Qi Tian. 2016. Mars: a video benchmark for large-scale person re-identification. In ECCV.
[48]
Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jingdong Wang, and Qi Tian. 2015. Scalable person re-identification: a benchmark. In Santiago, Chile, (Dec. 2015), 1116--1124.
[49]
Zhedong Zheng, Liang Zheng, and Yi Yang. 2017. Unlabeled samples generated by gan improve the person re-identification baseline in vitro. 2017 IEEE International Conference on Computer Vision (ICCV), 3774--3782.
[50]
Kaiyang Zhou and Tao Xiang. 2019. Torchreid: a library for deep learning person re-identification in pytorch. arXiv preprint arXiv:1910.10093.

Cited By

View all
  • (2024)Mutual Distillation Learning for Person Re-IdentificationIEEE Transactions on Multimedia10.1109/TMM.2024.338467726(8981-8995)Online publication date: 2024
  • (2024)Si-GAIS: Siamese Generalizable-Attention Instance Segmentation for Intersection Perception SystemIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.341164725:11(15759-15774)Online publication date: Dec-2024
  • (2024)POA-Net: Dance Poses and Activity Classification Using Convolutional Neural Networks2024 IEEE Region 10 Symposium (TENSYMP)10.1109/TENSYMP61132.2024.10752281(1-6)Online publication date: 27-Sep-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MMSports '22: Proceedings of the 5th International ACM Workshop on Multimedia Content Analysis in Sports
October 2022
152 pages
ISBN:9781450394888
DOI:10.1145/3552437
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. ball 3d localization
  2. basketball
  3. camera calibration
  4. challenge
  5. competition
  6. computer vision
  7. dataset
  8. deep learning
  9. image understanding
  10. instance segmentation
  11. person re-identification
  12. reid
  13. sports

Qualifiers

  • Research-article

Conference

MM '22
Sponsor:

Acceptance Rates

MMSports '22 Paper Acceptance Rate 17 of 26 submissions, 65%;
Overall Acceptance Rate 29 of 49 submissions, 59%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)97
  • Downloads (Last 6 weeks)6
Reflects downloads up to 28 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Mutual Distillation Learning for Person Re-IdentificationIEEE Transactions on Multimedia10.1109/TMM.2024.338467726(8981-8995)Online publication date: 2024
  • (2024)Si-GAIS: Siamese Generalizable-Attention Instance Segmentation for Intersection Perception SystemIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.341164725:11(15759-15774)Online publication date: Dec-2024
  • (2024)POA-Net: Dance Poses and Activity Classification Using Convolutional Neural Networks2024 IEEE Region 10 Symposium (TENSYMP)10.1109/TENSYMP61132.2024.10752281(1-6)Online publication date: 27-Sep-2024
  • (2024)Investigating Event-Based Cameras for Video Frame Interpolation in Sports2024 IEEE International Workshop on Sport, Technology and Research (STAR)10.1109/STAR62027.2024.10635973(138-143)Online publication date: 8-Jul-2024
  • (2024)Beyond the Premier: Assessing Action Spotting Transfer Capability Across Diverse Domains2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00343(3386-3398)Online publication date: 17-Jun-2024
  • (2024)A Universal Protocol to Benchmark Camera Calibration for Sports2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00338(3335-3346)Online publication date: 17-Jun-2024
  • (2024)SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00334(3293-3305)Online publication date: 17-Jun-2024
  • (2024)SoccerNet-Depth: a Scalable Dataset for Monocular Depth Estimation in Sports Videos2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00333(3280-3282)Online publication date: 17-Jun-2024
  • (2024)X-VARS: Introducing Explainability in Football Refereeing with Multi-Modal Large Language Models2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW63382.2024.00332(3267-3279)Online publication date: 17-Jun-2024
  • (2024)Towards Learning Monocular 3D Object Localization from 2D Labels Using the Physical Laws of Motion2024 International Conference on 3D Vision (3DV)10.1109/3DV62453.2024.00155(1564-1573)Online publication date: 18-Mar-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media