research-article

Orthogonal Uncertainty Representation of Data Manifold for Robust Long-Tailed Learning

Authors:

Lingling LiAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 4848 - 4857

https://doi.org/10.1145/3581783.3611698

Published: 27 October 2023 Publication History

Abstract

In scenarios with long-tailed distributions, the model's ability to identify tail classes is limited due to the under-representation of tail samples. Class rebalancing, information augmentation, and other techniques have been proposed to facilitate models to learn the potential distribution of tail classes. The disadvantage is that these methods generally pursue models with balanced class accuracy on the data manifold, while ignoring the ability of the model to resist interference. By constructing noisy data manifold, we found that the robustness of models trained on unbalanced data has a long-tail phenomenon. That is, even if the class accuracy is balanced on the data domain, it still has bias on the noisy data manifold. However, existing methods cannot effectively mitigate the above phenomenon, which makes the model vulnerable in long-tailed scenarios. In this work, we propose an Orthogonal Uncertainty Representation (hOUR) of feature embedding and an end-to-end training strategy to improve the long-tail phenomenon of model robustness. As a general enhancement tool, OUR has excellent compatibility with other methods and does not require additional data generation, ensuring fast and efficient training. Comprehensive evaluations on long-tailed datasets show that our method significantly improves the long-tail phenomenon of robustness, bringing consistent performance gains to other long-tailed learning methods.

References

[1]

Sumyeong Ahn, Jongwoo Ko, and Se-Young Yun. 2023. CUDA: Curriculum of Data Augmentation for Long-tailed Recognition. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=RgUPdudkWlN

[2]

Shaden Alshammari, Yu-Xiong Wang, Deva Ramanan, and Shu Kong. 2022. Long-tailed recognition via weight balancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6897--6907.

[3]

Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. 2019. Learning imbalanced datasets with label-distribution-aware margin loss. Advances in neural information processing systems, Vol. 32 (2019).

[4]

Peng Chu, Xiao Bian, Shaopeng Liu, and Haibin Ling. 2020. Feature space augmentation for long-tailed data. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXIX 16. Springer, 694--710.

Digital Library

[5]

Jiequan Cui, Shu Liu, Zhuotao Tian, Zhisheng Zhong, and Jiaya Jia. 2022. Reslt: Residual learning for long-tailed recognition. IEEE transactions on pattern analysis and machine intelligence, Vol. 45, 3 (2022), 3695--3706.

[6]

Jiequan Cui, Zhisheng Zhong, Shu Liu, Bei Yu, and Jiaya Jia. 2021. Parametric contrastive learning. In Proceedings of the IEEE/CVF international conference on computer vision. 715--724.

[7]

Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie. 2019. Class-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9268--9277.

[8]

Yin Cui, Yang Song, Chen Sun, Andrew Howard, and Serge Belongie. 2018. Large scale fine-grained categorization and domain-specific transfer learning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4109--4118.

[9]

Andrew Estabrooks, Taeho Jo, and Nathalie Japkowicz. 2004. A multiple resampling method for learning from imbalanced data sets. Computational intelligence, Vol. 20, 1 (2004), 18--36.

[10]

Luca Grementieri and Rita Fioresi. 2022. Model-centric data manifold: the data through the eyes of the model. SIAM Journal on Imaging Sciences, Vol. 15, 3 (2022), 1140--1156.

Digital Library

[11]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[12]

Yin-Yin He, Jianxin Wu, and Xiu-Shen Wei. 2021. Distilling virtual examples for long-tailed recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 235--244.

[13]

Feng Hong, Jiangchao Yao, Zhihan Zhou, Ya Zhang, and Yanfeng Wang. 2023. Long-tailed partial label learning via dynamic rebalancing. arXiv preprint arXiv:2302.05080 (2023).

[14]

Youngkyu Hong, Seungju Han, Kwanghee Choi, Seokjun Seo, Beomsu Kim, and Buru Chang. 2021. Disentangling label distribution for long-tailed visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6626--6636.

[15]

Jaehyung Kim, Jongheon Jeong, and Jinwoo Shin. 2020. M2m: Imbalanced classification via major-to-minor translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 13896--13905.

[16]

Na Lei, Dongsheng An, Yang Guo, Kehua Su, Shixia Liu, Zhongxuan Luo, Shing-Tung Yau, and Xianfeng Gu. 2020. A geometric understanding of deep learning. Engineering, Vol. 6, 3 (2020), 361--374.

[17]

Mengke Li, Yiu-Ming Cheung, and Zhikai Hu. 2022a. Key point sensitive loss for long-tailed visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 4 (2022), 4812--4825.

[18]

Mengke Li, Yiu-ming Cheung, and Yang Lu. 2022b. Long-tailed visual recognition via gaussian clouded logit adjustment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6929--6938.

[19]

Shuang Li, Kaixiong Gong, Chi Harold Liu, Yulin Wang, Feng Qiao, and Xinjing Cheng. 2021. Metasaug: Meta semantic augmentation for long-tailed visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5212--5221.

[20]

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision. 2980--2988.

[21]

Bo Liu, Haoxiang Li, Hao Kang, Gang Hua, and Nuno Vasconcelos. 2021. Gistnet: a geometric structure transfer network for long-tailed recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8209--8218.

[22]

Jialun Liu, Yifan Sun, Chuchu Han, Zhaopeng Dou, and Wenhui Li. 2020. Deep representation learning on long-tailed data: A learnable embedding augmentation perspective. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2970--2979.

[23]

Yanbiao Ma, Licheng Jiao, Fang Liu, Yuxin Li, Shuyuan Yang, and Xu Liu. 2023. Delving into Semantic Scale Imbalance. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=07tc5kKRIo

[24]

Sarah Parisot, Pedro M Esperancc a, Steven McDonagh, Tamas J Madarasz, Yongxin Yang, and Zhenguo Li. 2022. Long-tail recognition via compositional knowledge transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6939--6948.

[25]

Seulki Park, Youngkyu Hong, Byeongho Heo, Sangdoo Yun, and Jin Young Choi. 2022. The majority can help the minority: Context-rich minority oversampling for long-tailed classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6887--6896.

[26]

Jiawei Ren, Cunjun Yu, Xiao Ma, Haiyu Zhao, Shuai Yi, et al. 2020. Balanced meta-softmax for long-tailed visual recognition. Advances in neural information processing systems, Vol. 33 (2020), 4175--4186.

[27]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. Imagenet large scale visual recognition challenge. International journal of computer vision, Vol. 115 (2015), 211--252.

[28]

Saptarshi Sinha, Hiroki Ohashi, and Katsuyuki Nakamura. 2022. Class-difficulty based methods for long-tailed visual recognition. International Journal of Computer Vision, Vol. 130, 10 (2022), 2517--2531.

Digital Library

[29]

Kaihua Tang, Jianqiang Huang, and Hanwang Zhang. 2020. Long-tailed classification by keeping the good and removing the bad momentum causal effect. Advances in Neural Information Processing Systems, Vol. 33 (2020), 1513--1524.

[30]

Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. 2018. The inaturalist species classification and detection dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition. 8769--8778.

[31]

Chaozheng Wang, Shuzheng Gao, Pengyun Wang, Cuiyun Gao, Wenjie Pei, Lujia Pan, and Zenglin Xu. 2022. Label-aware distribution calibration for long-tailed classification. IEEE Transactions on Neural Networks and Learning Systems 99 (2022), 1--13.

[32]

Jianfeng Wang, Thomas Lukasiewicz, Xiaolin Hu, Jianfei Cai, and Zhenghua Xu. 2021. Rsg: A simple but effective module for learning imbalanced datasets. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3784--3793.

[33]

Tao Wang, Yu Li, Bingyi Kang, Junnan Li, Junhao Liew, Sheng Tang, Steven Hoi, and Jiashi Feng. 2020a. The devil is in classification: A simple framework for long-tail instance segmentation. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIV 16. Springer, 728--744.

Digital Library

[34]

Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, and Stella X Yu. 2020b. Long-tailed recognition by routing diverse distribution-aware experts. arXiv preprint arXiv:2010.01809 (2020).

[35]

Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1492--1500.

[36]

Zhengzhuo Xu, Zenghao Chai, and Chun Yuan. 2021. Towards calibrated model for long-tailed visual recognition from prior perspective. Advances in Neural Information Processing Systems, Vol. 34 (2021), 7139--7152.

[37]

Jing-Hao Xue and Peter Hall. 2014. Why does rebalancing class-unbalanced data improve AUC for linear discriminant analysis? IEEE transactions on pattern analysis and machine intelligence, Vol. 37, 5 (2014), 1109--1112.

[38]

Shiran Zada, Itay Benou, and Michal Irani. 2022. Pure noise to the rescue of insufficient data: Improving imbalanced classification by training on random noise images. In International Conference on Machine Learning. PMLR, 25817--25833.

[39]

Yuhang Zang, Chen Huang, and Chen Change Loy. 2021. Fasa: Feature augmentation and sampling adaptation for long-tailed instance segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 3457--3466.

[40]

Yifan Zhang, Bingyi Kang, Bryan Hooi, Shuicheng Yan, and Jiashi Feng. 2023. Deep long-tailed learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).

[41]

Zizhao Zhang and Tomas Pfister. 2021. Learning fast sample re-weighting without reward data. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 725--734.

[42]

Peilin Zhao, Yifan Zhang, Min Wu, Steven CH Hoi, Mingkui Tan, and Junzhou Huang. 2018. Adaptive cost-sensitive online classification. IEEE Transactions on Knowledge and Data Engineering, Vol. 31, 2 (2018), 214--228.

Digital Library

[43]

Zhisheng Zhong, Jiequan Cui, Shu Liu, and Jiaya Jia. 2021. Improving calibration for long-tailed recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 16489--16498.

[44]

Boyan Zhou, Quan Cui, Xiu-Shen Wei, and Zhao-Min Chen. 2020. Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9719--9728.

[45]

Zhi-Hua Zhou and Xu-Ying Liu. 2005. Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Transactions on knowledge and data engineering, Vol. 18, 1 (2005), 63--77.

Digital Library

Cited By

He PJiao LLi LLiu XLiu FMa WYang SShang RCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Domain Generalization-Aware Uncertainty Introspective Learning for 3D Point Clouds SegmentationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681574(651-660)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681574
Ru LGuo XYu LZhang YLao JWang JChen JLi YYang MCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Parameter-Efficient Complementary Expert Learning for Long-Tailed Visual RecognitionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680799(5393-5402)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680799

Index Terms

Orthogonal Uncertainty Representation of Data Manifold for Robust Long-Tailed Learning
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations
  2. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification

Recommendations

Imbalance fault diagnosis under long-tailed distribution: Challenges, solutions and prospects
Abstract
Intelligent fault diagnosis based on deep learning has yielded remarkable progress for its strong feature representation capability in recent years. Nevertheless, in engineering scenarios, machines often operate under normal condition ...
Highlights
- Define the class long-tailed distribution in intelligent fault diagnosis for the first time.
Decoupled Contrastive Learning for Long-Tailed Distribution
Pattern Recognition and Computer Vision
Abstract
Self-supervised contrastive learning is popularly used to obtain powerful representation models. However, unlabeled data in the real world naturally exhibits a long-tailed distribution, making the traditional instance-wise contrastive learning ...
Long-Tailed Instance Segmentation Using Gumbel Optimized Loss
Computer Vision – ECCV 2022
Abstract
Major advancements have been made in the field of object detection and segmentation recently. However, when it comes to rare categories, the state-of-the-art methods fail to detect them, resulting in a significant performance gap between rare and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

the National Natural Science Foundation of China
the State Key Program and the Foundation for Innovative Research Groups of the National Natural Science Foundation of China
the 111 Project
the Program for Cheung Kong Scholars and Innovative Research Team in University
the Key Research and Development Program in Shaanxi Province of China
the National Science Basic Research Plan in Shaanxi Province of China
the National Natural Science Foundation of China
the ST Innovation Project from the Chinese Ministry of Education
the China Postdoctoral fund
the Key Scientific Technological Innovation Research Project by Ministry of Education
the Scientific Research Project of Education Department In Shaanxi Province of China

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
115
Total Downloads

Downloads (Last 12 months)73
Downloads (Last 6 weeks)7

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

He PJiao LLi LLiu XLiu FMa WYang SShang RCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Domain Generalization-Aware Uncertainty Introspective Learning for 3D Point Clouds SegmentationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681574(651-660)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681574
Ru LGuo XYu LZhang YLao JWang JChen JLi YYang MCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Parameter-Efficient Complementary Expert Learning for Long-Tailed Visual RecognitionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680799(5393-5402)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680799

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents