Abstract
Active learning (AL) is a label-efficient technique for training deep models when only a limited labeled set is available and the manual annotation is expensive. Implicit semantic data augmentation (ISDA) effectively extends the limited amount of labeled samples and increases the diversity of labeled sets without introducing a noticeable extra computational cost. The scarcity of labeled instances and the huge annotation cost of unlabelled samples encourage us to ponder on the combination of AL and ISDA. A nature direction is a pipelined integration, which selects the unlabeled samples via acquisition function in AL for labeling and generates virtual samples by changing the selected samples to semantic transformation directions within ISDA. However, this pipelined combination would not guarantee the diversity of virtual samples. This paper proposes diversity-aware semantic transformation active learning, or DAST-AL framework, that looks ahead the effect of ISDA in the selection of unlabeled samples. Specifically, DAST-AL exploits expected partial model change maximization (EPMCM) to consider selected samples’ potential contribution of the diversity to the labeled set by leveraging the semantic transformation within ISDA when selecting the unlabeled samples. After that, DAST-AL can confidently and efficiently augment the labeled set by implicitly generating more diverse samples. The empirical results on both image classification and semantic segmentation tasks show that the proposed DAST-AL can slightly outperform the state-of-the-art AL approaches. Under the same condition, the proposed method takes less than 3 min for the first cycle of active labeling while the existing agreement discrepancy selection incurs more than 40 min.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abraham, I., Murphey, T.D.: Active learning of dynamics for data-driven control using Koopman operators. IEEE Trans. Rob. 35(5), 1071–1083 (2019)
Beluch, W.H., Genewein, T., Nürnberger, A., Köhler, J.M.: The power of ensembles for active learning in image classification. In: CVPR, pp. 9368–9377 (2018)
Bottou, L.: Stochastic gradient descent tricks. In: Neural Networks: Tricks of the Trade, pp. 421–436 (2012)
Cai, W., Zhang, M., Zhang, Y.: Batch mode active learning for regression with expected model change. IEEE Trans. Neural Netw. Learn. Syst. 28(7), 1668–1681 (2016)
Cai, W., Zhang, Y., Zhou, J.: Maximizing expected model change for active learning in regression. In: IEEE International Conference on Data Mining, pp. 51–60 (2013)
Caron, M., Bojanowski, P., Joulin, A., Douze, M.: Deep clustering for unsupervised learning of visual features. In: ECCV, pp. 132–149 (2018)
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR, pp. 3213–3223 (2016)
Curtiss, J.H.: A note on the theory of moment generating functions. Ann. Math. Stat. 13(4), 430–433 (1942)
Dasgupta, S., Hsu, D.: Hierarchical sampling for active learning. In: International Conference on Machine Learning, pp. 208–215 (2008)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: CVPR, pp. 248–255 (2009)
Doersch, C., Gupta, A., Efros, A.A.: Unsupervised visual representation learning by context prediction. In: CVPR, pp. 1422–1430 (2015)
Ebrahimi, S., Rohrbach, A., Darrell, T.: Gradient-free policy architecture search and adaptation. In: Conference on Robot Learning, pp. 505–514 (2017)
Fu, M., Yuan, T., Wan, F., Xu, S., Ye, Q.: Agreement-discrepancy-selection: active learning with progressive distribution alignment. In: AAAI, pp. 7466–7473 (2021)
Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059 (2016)
Gal, Y., Islam, R., Ghahramani, Z.: Deep bayesian active learning with image data. In: International Conference on Machine Learning, pp. 1183–1192 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)
Jiang, B., Zhang, Z., Lin, D., Tang, J., Luo, B.: Semi-supervised learning with graph learning-convolutional networks. In: CVPR, pp. 11313–11320 (2019)
Joshi, A.J., Porikli, F., Papanikolopoulos, N.: Multi-class active learning for image classification. In: CVPR, pp. 2372–2379 (2009)
Kim, K., Park, D., Kim, K.I., Chun, S.Y.: Task-aware variational adversarial active learning. In: CVPR, pp. 8166–8175 (2021)
Kingma, D.P., Mohamed, S., Rezende, D.J., Welling, M.: Semi-supervised learning with deep generative models. In: NeurIPS, pp. 3581–3589 (2014)
Kuo, W., Häne, C., Yuh, E., Mukherjee, P., Malik, J.: Cost-sensitive active learning for intracranial hemorrhage detection. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 715–723. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_82
Lewis, D.D., Gale, W.A.: A sequential algorithm for training text classifiers, pp. 3–12 (1994)
Li, J., Chen, Z., Chen, J., Lin, Q.: Diversity-sensitive generative adversarial network for terrain mapping under limited human intervention. IEEE Trans. Cybern. 51, 6029–6040 (2020)
Mahajan, D., et al.: Exploring the limits of weakly supervised pretraining. In: ECCV, pp. 181–196 (2018)
Making, M.O.D.: Synthesis lectures on artificial intelligence and machine learning (2012)
Mayer, C., Timofte, R.: Adversarial sampling for active learning. In: IEEE Winter Conference on Applications of Computer Vision, pp. 3071–3079 (2020)
Noroozi, M., Pirsiavash, H., Favaro, P.: Representation learning by learning to count. In: ICCV, pp. 5898–5906 (2017)
Peyre, J., Sivic, J., Laptev, I., Schmid, C.: Weakly-supervised learning of visual relations. In: CVPR, pp. 5179–5188 (2017)
Saito, S., Yang, J., Ma, Q., Black, M.J.: Scanimate: weakly supervised learning of skinned clothed avatar networks. In: CVPR, pp. 2886–2897 (2021)
Sener, O., Savarese, S.: Active learning for convolutional neural networks: a core-set approach. In: ICLR (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)
Sinha, S., Ebrahimi, S., Darrell, T.: Variational adversarial active learning. In: ICCV, pp. 5972–5981 (2019)
Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. NeurIPS 28, 3483–3491 (2015)
Wang, D., Zhang, Y., Zhang, K., Wang, L.: Focalmix: semi-supervised learning for 3D medical image detection. In: CVPR, pp. 3951–3960 (2020)
Wang, K., Zhang, D., Li, Y., Zhang, R., Lin, L.: Cost-effective active learning for deep image classification. IEEE TCSVT 27(12), 2591–2600 (2016)
Wang, Y., Huang, G., Song, S., Pan, X., Xia, Y., Wu, C.: Regularizing deep networks with semantic data augmentation. IEEE TPAMI 44, 3733–3748 (2021)
Yoo, D., Kweon, I.S.: Learning loss for active learning. In: CVPR, pp. 93–102 (2019)
Yu, F., Koltun, V., Funkhouser, T.: Dilated residual networks. In: CVPR, pp. 472–480 (2017)
Yuan, T., et al.: Multiple instance active learning for object detection. In: CVPR, pp. 5330–5339 (2021)
Zhai, X., Oliver, A., Kolesnikov, A., Beyer, L.: S4l: self-supervised semi-supervised learning. In: ICCV, pp. 1476–1485 (2019)
Zhang, B., Li, L., Yang, S., Wang, S., Zha, Z.J., Huang, Q.: State-relabeling adversarial active learning. In: CVPR, pp. 8756–8765 (2020)
Zhu, J.J., Bento, J.: Generative adversarial active learning. arXiv preprint (2017)
Zhuang, C., Zhai, A.L., Yamins, D.: Local aggregation for unsupervised learning of visual embeddings. In: ICCV, pp. 6002–6012 (2019)
Zhukov, D., Alayrac, J.B., Cinbis, R.G., Fouhey, D., Laptev, I., Sivic, J.: Cross-task weakly supervised learning from instructional videos. In: CVPR, pp. 3537–3545 (2019)
Acknowledgements
This work was supported by the National Nature Science Foundation of China under Grants U2013201, 62073225, 62072315, 61836005 and 62006157, the Natural Science Foundation of Guangdong Province-Outstanding Youth Program under Grant 2019B151502018, the Guangdong “Pearl River Talent Recruitment Program” under Grant 2019ZT08X603, the Guangdong"Pearl River Talent Plan" under Grant 2019JC01X235, and the Shenzhen Science and Technology Innovation Commission R2020A045.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, Z., Zhang, J., Wang, P., Chen, J., Li, J. (2022). When Active Learning Meets Implicit Semantic Data Augmentation. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13685. Springer, Cham. https://doi.org/10.1007/978-3-031-19806-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-19806-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19805-2
Online ISBN: 978-3-031-19806-9
eBook Packages: Computer ScienceComputer Science (R0)