research-article

Two-branch Objectness-centric Open World Detection

Authors:

Xianglong LiuAuthors Info & Claims

HCMA '22: Proceedings of the 3rd International Workshop on Human-Centric Multimedia Analysis

Pages 35 - 40

https://doi.org/10.1145/3552458.3556453

Published: 10 October 2022 Publication History

Abstract

In recent years, with the development of deep learning, object detection has made great progress and has been widely used in many tasks. However, the previous models are all performed on closed sets, while there are many unknown categories in the real open world. Directly applying a model trained on known categories to the unknown classes will lead to misclassification. In this paper, we propose a two-branch objectness-centric open world object detection framework consisting of the bias-guided detector and the objectness-centric calibrator to effectively capture the objectness of both known and unknown instances and make the accurate prediction for known classes. The bias-guided detector trained with the known labels can predict the classes and boxes for known classes accurately. While the objectness-centric calibrator can localize the instances of any class, and does not affect the classification and regression of known classes. In the inference stage, we use the objectness-centric affirmation to confirm the results for known classes and predict the unknown instances. Comprehensive experiments conducted on the open world object detection benchmark validate the effectiveness of our method compared to state-of-the-art open world object detection approaches.

Supplementary Material

MP4 File (HCMA22-09.mp4)

Open World Object Detection (OWOD), simulating the real dynamic world where knowledge grows continuously, attempts to detect both known and unknown classes and incrementally learn the identified unknown ones. In this work, we propose a two-branch objectness-centric open world detection framework to fully explore the generalized objectness of both unknown and known classes and ensure open world recognition performance.

Download
10.52 MB

References

[1]

Abhijit Bendale and Terrance Boult. 2015. Towards open world recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1893--1902.

[2]

Abhijit Bendale and Terrance E Boult. 2016. Towards open set deep networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1563--1572.

[3]

Jiale Cao, Hisham Cholakkal, Rao Muhammad Anwer, Fahad Shahbaz Khan, Yanwei Pang, and Ling Shao. 2020. D2det: Towards high quality object detection and instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11485--11494.

[4]

Akshay Dhamija, Manuel Gunther, Jonathan Ventura, and Terrance Boult. 2020. The overlooked elephant of object detection: Open set. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 1021--1030.

[5]

Akshay Raj Dhamija, Manuel Günther, and Terrance E Boult. 2018. Reducing network agnostophobia. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 9175--9186.

[6]

Yunhao Ge, Yao Xiao, Zhi Xu, Meng Zheng, Srikrishna Karanam, Terrence Chen, Laurent Itti, and Ziyan Wu. 2021. A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2195--2204.

[7]

Zongyuan Ge, Sergey Demyanov, Zetao Chen, and Rahil Garnavi. 2017. Generative OpenMax for multi-class open set classification. In British Machine Vision Conference 2017. British Machine Vision Association and Society for Pattern Recognition.

[8]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[9]

Xin Hong, Yanyan Lan, Liang Pang, Jiafeng Guo, and Xueqi Cheng. 2021. Transformation Driven Visual Reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6903--6912.

[10]

Lalit P Jain, Walter J Scheirer, and Terrance E Boult. 2014. Multi-class open set recognition using probability of inclusion. In European Conference on Computer Vision. Springer, 393--409.

[11]

KJ Joseph, Salman Khan, Fahad Shahbaz Khan, and Vineeth N Balasubramanian. 2021. Towards open world object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5830--5840.

[12]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, Vol. 25 (2012).

Digital Library

[13]

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision. 2980--2988.

[14]

Lawrence Neal, Matthew Olson, Xiaoli Fern, Weng-Keen Wong, and Fuxin Li. 2018. Open set learning with counterfactual images. In Proceedings of the European Conference on Computer Vision (ECCV). 613--628.

Digital Library

[15]

Cody Reading, Ali Harakeh, Julia Chae, and Steven L Waslander. 2021. Categorical depth distribution network for monocular 3d object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8555--8564.

[16]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems, Vol. 28 (2015), 91--99.

[17]

Walter J Scheirer, Anderson de Rezende Rocha, Archana Sapkota, and Terrance E Boult. 2012. Toward open set recognition. IEEE transactions on pattern analysis and machine intelligence, Vol. 35, 7 (2012), 1757--1772.

[18]

Walter J Scheirer, Lalit P Jain, and Terrance E Boult. 2014. Probability models for open set recognition. IEEE transactions on pattern analysis and machine intelligence, Vol. 36, 11 (2014), 2317--2324.

[19]

Shuo Yang, Peize Sun, Yi Jiang, Xiaobo Xia, Ruiheng Zhang, Zehuan Yuan, Changhu Wang, Ping Luo, and Min Xu. 2022. Objects in Semantic Topology. In International Conference on Learning Representations.

[20]

Ryota Yoshihashi, Wen Shao, Rei Kawakami, Shaodi You, Makoto Iida, and Takeshi Naemura. 2019. Classification-reconstruction learning for open-set recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4016--4025.

[21]

Xiaowei Zhao, Xianglong Liu, Yifan Shen, Yuqing Ma, Yixuan Qiao, and Duorui Wang. 2022. Revisiting open world object detection. arXiv preprint arXiv:2201.00471 (2022).

[22]

Xingyi Zhou, Dequan Wang, and Philipp Kr"ahenbühl. 2019. Objects as points. arXiv preprint arXiv:1904.07850 (2019).

[23]

Chenchen Zhu, Fangyi Chen, Uzair Ahmed, Zhiqiang Shen, and Marios Savvides. 2021. Semantic relation reasoning for shot-stable few-shot object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8782--8791.

Cited By

He YChen WWang SLiu TWang M(2025)Recalling Unknowns Without Losing Precision: An Effective Solution to Large Model-Guided Open World Object DetectionIEEE Transactions on Image Processing10.1109/TIP.2024.345958934(729-742)Online publication date: 2025
https://doi.org/10.1109/TIP.2024.3459589
Wang XXu D(2024)Text-Guided Unknown Pseudo-Labeling for Open-World Object DetectionElectronics10.3390/electronics1322452813:22(4528)Online publication date: 18-Nov-2024
https://doi.org/10.3390/electronics13224528
Xi XHuang YLin JLuo RLarson K(2024)KTCNProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/162(1462-1470)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/162
Show More Cited By

Index Terms

Two-branch Objectness-centric Open World Detection
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object detection

Recommendations

Rethinking Open-World Object Detection in Autonomous Driving Scenarios
MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Existing object detection models have been demonstrated to successfully discriminate and localize the predefined object categories under the seen or similar situations. However, the open-world object detection as required by autonomous driving ...
UC-OWOD: Unknown-Classified Open World Object Detection
Computer Vision – ECCV 2022
Abstract
Open World Object Detection (OWOD) is a challenging computer vision problem that requires detecting unknown objects and gradually learning the identified unknown classes. However, it cannot distinguish unknown instances as multiple unknown ...
LVMUM: Toward Open-World Object Detection with Large Vision Models and Unsupervised Modeling
Advanced Intelligent Computing Technology and Applications
Abstract
Open-world object detection (OWOD), as an emerging and challenging task in object detection, requires the model to have the ability to detect known and unknown objects in dynamic environments. Furthermore, it should have the capability to perform ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HCMA '22: Proceedings of the 3rd International Workshop on Human-Centric Multimedia Analysis

October 2022

106 pages

ISBN:9781450394925

DOI:10.1145/3552458

Program Chairs:
Dingwen Zhang
Northwestern Polytechnical University, Xi'an, China
,
Chaowei Fang
Xidian University, Xi'an, China
,
Wu Liu
JD AI Research, Beijing, China
,
Xinchen Liu
JD AI Research, Beijing, China
,
Jingkuan Song
University of Electronic Science & Technology of China, Chengdu, China
,
Hongyuan Zhu
Agency for Science, Technology, and Research (A*STAR), Singapore
,
Wenbing Huang
Tsinghua University, Beijing, China
,
John Smith
IBM Research, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Beijing Nova Program of Science and Technology
National Natural Science Foundation of China

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

HCMA '22 Paper Acceptance Rate 12 of 21 submissions, 57%;

Overall Acceptance Rate 12 of 21 submissions, 57%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
278
Total Downloads

Downloads (Last 12 months)87
Downloads (Last 6 weeks)8

Reflects downloads up to 31 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

He YChen WWang SLiu TWang M(2025)Recalling Unknowns Without Losing Precision: An Effective Solution to Large Model-Guided Open World Object DetectionIEEE Transactions on Image Processing10.1109/TIP.2024.345958934(729-742)Online publication date: 2025
https://doi.org/10.1109/TIP.2024.3459589
Wang XXu D(2024)Text-Guided Unknown Pseudo-Labeling for Open-World Object DetectionElectronics10.3390/electronics1322452813:22(4528)Online publication date: 18-Nov-2024
https://doi.org/10.3390/electronics13224528
Xi XHuang YLin JLuo RLarson K(2024)KTCNProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/162(1462-1470)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/162
Yuan ZXiong HHou SLi WFan Q(2024)EAGLE Network: A Novel Incremental Learning Framework for Detecting Unknown Logos in Open-World EnvironmentsProceedings of the 1st on Continual Learning meets Multimodal Foundation Models: Fundamentals and Advances10.1145/3688859.3690081(23-30)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3688859.3690081
Sarkar HChudasama VOnoe NWasnik PBalasubramanian V(2024)Open-Set Object Detection By Aligning Known Class Representations2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00029(218-227)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00029
Inkawhich MInkawhich NYang HZhang JLinderman RChen Y(2024)OSR-ViT: A Simple and Modular Framework for Open-Set Object Detection and Discovery2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10826036(928-937)Online publication date: 15-Dec-2024
https://doi.org/10.1109/BigData62323.2024.10826036
Fan JZhang EWei YWang YXia JLiu JLiu XMa S(2024)DDOWODPattern Recognition Letters10.1016/j.patrec.2024.10.002186:C(170-177)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1016/j.patrec.2024.10.002
Zhao RWang JChen YZheng ZCui KSu J(2024)Class-Agnostic Detection of Unknown Objects from Foreground Improves Robust Open World Object DetectionPattern Recognition and Computer Vision10.1007/978-981-97-8858-3_6(78-92)Online publication date: 3-Nov-2024
https://doi.org/10.1007/978-981-97-8858-3_6
Yavuz MGüney F(2024)O1O: Grouping of Known Classes to Identify Unknown Objects as Odd-One-OutComputer Vision – ACCV 202410.1007/978-981-96-0972-7_23(394-410)Online publication date: 10-Dec-2024
https://doi.org/10.1007/978-981-96-0972-7_23
Ma SWang YWei YFan JLi TLiu HLv F(2023)CAT: LoCalization and IdentificAtion Cascade Detection Transformer for Open-World Object Detection2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01885(19681-19690)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01885
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten