Abstract
In multi-label text classification, the numbers of instances in different categories are usually extremely imbalanced. How to learn good models from imbalanced data is a challenging task. Some existing works tackle it through class re-balancing strategies or imbalanced loss objectives, but their performance remains limited in the cases of imbalanced distributed data. In this work, we propose a model, which combined Siamese Network and Bilateral-Branch Network to deal with both representation learning and classifier learning simultaneously. In the siamese network component, we propose a category-specific similarity strategy to improve the representation learning and adapt a novelty dynamic learning mechanism to make the model end-to-end trainable, and in the bilateral-branch network, we adopt the cumulative learning strategy to shift the learning focus from universal pattern to tail learning. In general, we adopt a multi-task architecture to ensure that both the head categories and the tail categories are adequately trained. The experiments on two benchmark datasets show that our method can improve the performance on the entire and tail categories, and achieves competitive performance compared with existing approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., Shah, R.: Signature verification using a Siamese time delay neural network. Adv. Neural Inf. Process. Syst. 6 (1993)
Chu, P., Bian, X., Liu, S., Ling, H.: Feature space augmentation for long-tailed data. In: European Conference on Computer Vision, pp. 694–710 (2020)
Cui, Y., Jia, M., Lin, T.Y., Song, Y., Belongie, S.: Class-balanced loss based on effective number of samples. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9268–9277 (2019)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 4171–4186 (2019)
Guo, H., Wang, S.: Long-tailed multi-label visual recognition by collaborative training on uniform and re-balanced samplings. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15089–15098, June 2021
Hu, T., Li, J., Fukumoto, F., Zhou, R.: A multi-task based bilateral-branch network for imbalanced citation intent classification. In: 16th International Conference on Ubiquitous Information Management and Communication, pp. 1–8 (2022)
Huang, Y., Giledereli, B., Köksal, A., Özgür, A., Ozkirimli, E.: Balancing methods for multi-label text classification with long-tailed class distribution. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8153–8161 (2021)
Lewis, D.D., Yang, Y., Russell-Rose, T., Li, F.: RCV1: a new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361–397 (2004)
Li, J., Fukumoto, F.: Multi-task neural shared structure search: a study based on text mining. In: International Conference on Database Systems for Advanced Applications, pp. 202–218 (2021)
Li, J., Sato, A., Shimura, K., Fukumoto, F.: Multi-task peer-review score prediction. In: Proceedings of the First Workshop on Scholarly Document Processing, pp. 121–126 (2020)
Li, Y., Wang, T., Kang, B., Tang, S., Wang, C., Li, J., Feng, J.: Overcoming classifier imbalance for long-tail object detection with balanced group softmax. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10991–11000 (2020)
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)
Liu, J., Sun, Y., Han, C., Dou, Z., Li, W.: Deep representation learning on long-tailed data: a learnable embedding augmentation perspective. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2970–2979 (2020)
Liu, J., Chang, W.C., Wu, Y., Yang, Y.: Deep learning for extreme multi-label text classification. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 115–124 (2017)
More, A.: Survey of resampling techniques for improving classification performance in unbalanced datasets. arXiv preprint arXiv:1608.06048 (2016)
Muangkammuen, P., Xu, S., Fukumoto, F., Saikaew, K.R., Li, J.: A neural local coherence analysis model for clarity text scoring. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 2138–2143 (2020)
Shimura, K., Li, J., Fukumoto, F.: HFT-CNN: learning hierarchical category structure for multi-label short text categorization. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 811–816 (2018)
Shimura, K., Li, J., Fukumoto, F.: Text categorization by learning predominant sense of words as auxiliary task. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1109–1119 (2019)
Su, C., Fukumoto, F., Huang, X., Li, J., Wang, R., Chen, Z.: DeepMet: a reading comprehension paradigm for token-level metaphor detection. In: Proceedings of the Second Workshop on Figurative Language Processing, pp. 30–39 (2020)
Wallace, B.C., Small, K., Brodley, C.E., Trikalinos, T.A.: Class imbalance, redux. In: 2011 IEEE 11th International Conference on Data Mining, pp. 754–763 (2011)
Wu, T., Huang, Q., Liu, Z., Wang, Y., Lin, D.: Distribution-balanced loss for multi-label classification in long-tailed datasets. In: European Conference on Computer Vision, pp. 162–178 (2020)
Xiao, L., Huang, X., Chen, B., Jing, L.: Label-specific document representation for multi-label text classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 466–475 (2019)
Yang, P., Sun, X., Li, W., Ma, S., Wu, W., Wang, H.: SGM: sequence generation model for multi-label classification. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 3915–3926 (2018)
Yang, W., Li, J., Fukumoto, F., Ye, Y.: HSCNN: a Hybrid-Siamese convolutional neural network for extremely imbalanced multi-label text classification. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pp. 6716–6722 (2020)
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016)
Zhang, Z., Li, J., Fukumoto, F., Ye, Y.: Abstract, rationale, stance: a joint model for scientific claim verification. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 3580–3586 (2021)
Zhou, B., Cui, Q., Wei, X.S., Chen, Z.M.: BBN: bilateral-branch network with cumulative learning for long-tailed visual recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9719–9728 (2020)
Acknowledgements
This work was partially supported by JKA and by Research Grant for Young Scholars funded by Yamanashi Prefecture.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, J., Li, J., Fukumoto, F. (2023). BBSN: Bilateral-Branch Siamese Network for Imbalanced Multi-label Text Classification. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Lecture Notes in Computer Science, vol 13625. Springer, Cham. https://doi.org/10.1007/978-3-031-30111-7_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-30111-7_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30110-0
Online ISBN: 978-3-031-30111-7
eBook Packages: Computer ScienceComputer Science (R0)