Cross-scale fusion and domain adversarial network for generalizable rail surface defect segmentation on unseen datasets
Pages 367 - 386
Abstract
Surface quality control is a crucial part of rail manufacturing. Deep neural networks have shown impressive accuracy in rail surface defect segmentation under the assumption that the test images have the same distribution as the training images. However, in practice detection, the rail images exhibit variations in appearance and scale for different rail types and production conditions. Directly deploying the deep neural network on unseen images shows a performance degradation due to the distribution discrepancies of training images. To this end, we propose a cross-scale fusion and domain adversarial network (CFDANet) to improve the generalization ability of deep neural networks on unseen datasets. To alleviate the domain shift caused by defect scale differences, we design a dual-encoder to extract multi-scale features from images of different resolutions. Then, those features are adaptively fused through a cross-scale fusion module. For the domain shift caused by inconsistent rail appearance, we introduce transferable-aware domain adversarial learning to extract domain invariant features from different datasets. Moreover, we further propose a transferable curriculum to suppress the negative impact of images with low transferability. Experimental results show that our CFDANet can accurately segment defects in unseen datasets and surpass other state-of-the-art domain generalization methods in all five target domain settings. The source code is released at https://github.com/dotaball/railseg_dg.
References
[1]
Badmos O, Kopp A, Bernthaler T, and Schneider G Image-based defect detection in lithium-ion battery electrode using convolutional neural networks Journal of Intelligent Manufacturing 2020 31 4 885-897
[2]
Ben Gharsallah M and Ben Braiek E Computer aided manufacturing method for surface silicon steel inspection based on an efficient anisotropic diffusion algorithm Journal of Intelligent Manufacturing 2021 32 4 1025-1041
[3]
Ben-David S, Blitzer J, Crammer K, and Pereira F Analysis of representations for domain adaptation Advances in Neural Information Processing Systems 2006 19 137-144
[4]
Bengio Y, Louradour J, Collobert R, and Weston J Curriculum learning Proceedings of the 26th Annual International Conference on Machine Learning 2009
[5]
Chen C, Bai W, Davies RH, Bhuva AN, Manisty CH, Augusto JB, Moon JC, Aung N, Lee AM, Sanghvi MM, Fung K, Miguel Paiva J, Petersen SE, Lukaschuk E, Piechnik SK, Neubauer S, and Rueckert D Improving the generalizability of convolutional neural network-based segmentation on CMR images Frontiers in Cardiovascular Medicine 2020 7 105
[6]
Gan J, Li Q, Wang J, and Yu H A hierarchical extractor-based visual rail surface inspection system IEEE Sensors Journal 2017 17 23 7935-7944
[7]
Ganin Y and Lempitsky V Unsupervised domain adaptation by backpropagation International Conference on Machine Learning 2015 PMLR 1180-1189
[8]
Gao, S., Zhou, H., Gao, Y., & Zhuang, X. (2022). Joint modeling of image and label statistics for enhancing model generalizability of medical image segmentation. arXiv preprint arXiv:2206.04336.
[9]
Hao R, Lu B, Cheng Y, Li X, and Huang B A steel surface defect inspection approach towards smart industrial monitoring Journal of Intelligent Manufacturing 2021 32 7 1833-1843
[10]
Hendrycks, D., & Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations.
[11]
Jain S, Seth G, Paruthi A, Soni U, and Kumar G Synthetic data augmentation for surface defect detection and classification using deep learning Journal of Intelligent Manufacturing 2020 33 4 1007-1020
[12]
Khandelwal P and Yushkevich P Domain generalizer: A few-shot meta learning framework for domain generalization in medical imaging Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning 2020 Springer 73-84
[13]
Li D, Yang Y, Song YZ, and Hospedales T Learning to generalize: Meta-learning for domain generalization Proceedings of the AAAI Conference on Artificial Intelligence 2018
[14]
Liu Q, Dou Q, and Heng PA Shape-aware meta-learning for generalizing prostate MRI segmentation to unseen domains International Conference on Medical Image Computing and Computer-Assisted Intervention 2020 Springer 475-485
[15]
Luo Y, Zheng L, Guan T, Yu J, and Yang Y Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2019
[16]
Ni X, Ma Z, Liu J, Shi B, and Liu H Attention network for rail surface defect detection via consistency of intersection-over-union (IoU)-guided center-point estimation IEEE Transactions on Industrial Informatics 2021 18 3 1694-1705
[17]
Nieniewski M Morphological detection and extraction of rail surface defects IEEE Transactions on Instrumentation and Measurement 2020 69 9 6870-6879
[18]
Niu M, Song K, Huang L, Wang Q, Yan Y, and Meng Q Unsupervised saliency detection of rail surface defects using stereoscopic images IEEE Transactions on Industrial Informatics 2020 17 3 2271-2281
[19]
Niu M, Wang Y, Song K, Wang Q, Zhao Y, and Yan Y An adaptive pyramid graph and variation residual-based anomaly detection network for rail surface defects IEEE Transactions on Instrumentation and Measurement 2021 70 1-13
[20]
Pan F, Shin I, Rameau F, Lee S, and Kweon IS Unsupervised intra-domain adaptation for semantic segmentation through self-supervision Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020
[21]
Saenko K, Kulis B, Fritz M, and Darrell T Adapting visual category models to new domains European conference on computer vision 2010 Springer 213-226
[22]
Singh SA and Desai KA Automated surface defect detection framework using machine vision and convolutional neural networks Journal of Intelligent Manufacturing 2022
[23]
Song G, Song K, and Yan Y Saliency detection for strip steel surface defects using multiple constraints and improved texture features Optics and Lasers in Engineering 2020 128
[24]
Song K, Wang J, Bao Y, Huang L, and Yan Y A novel visible-depth-thermal image dataset of salient object detection for robotic visual perception IEEE/ASME Transactions on Mechatronics 2022
[25]
Song Y, Liu Z, Wang J, Tang R, Duan G, and Tan J Multiscale adversarial and weighted gradient domain adaptive network for data scarcity surface defect detection IEEE Transactions on Instrumentation and Measurement 2021 70 1-10
[26]
Tao, A., Sapra, K., & Catanzaro, B. (2020). Hierarchical multi-scale attention for semantic segmentation. arXiv preprint arXiv:2005.10821.
[27]
Tsai YH, Hung WC, Schulter S, Sohn K, Yang MH, and Chandraker M Learning to adapt structured output space for semantic segmentation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018
[28]
Van der Maaten L and Hinton G Visualizing data using t-SNE Journal of Machine Learning Research 2008 9 11 2579-2605
[29]
Vu TH, Jain H, Bucher M, Cord M, and Pérez P Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2019
[30]
Wang J, Li Q, Gan J, Yu H, and Yang X Surface defect detection via entity sparsity pursuit with intrinsic priors IEEE Transactions on Industrial Informatics 2019 16 1 141-150
[31]
Woo S, Park J, Lee JY, and Kweon IS CBAM Convolutional block attention module European on Computer Vision 2018 Springer 3-19
[32]
Xie E, Wang W, Yu Z, Anandkumar A, Alvarez JM, and Luo P SegFormer: Simple and efficient design for semantic segmentation with transformers Advances in Neural Information Processing Systems 2021 34 12077-12090
[33]
Yu H, Li Q, Tan Y, Gan J, Wang J, Geng YA, and Jia L A coarse-to-fine model for rail surface defect detection IEEE Transactions on Instrumentation and Measurement 2018 68 3 656-666
[34]
Zhang, L., Wang, X., Yang, D., Sanford, T., Harmon, S., Turkbey, B., Roth, H., Myronenko, A., Xu, D., Xu Z., (2019). When unseen domain generalization is unnecessary? Rethinking data augmentation. arXiv preprint arXiv:1906.03347.
[35]
Zhang D, Song K, Wang Q, He Y, Wen X, and Yan Y Two deep learning networks for rail surface defect inspection of limited samples with line-level label IEEE Transactions on Industrial Informatics 2020 17 10 6731-6741
[36]
Zhang H, Song Y, Chen Y, Zhong H, Liu L, Wang Y, Akilan T, and Jonathan Wu QM MRSDI-CNN: Multi-model rail surface defect inspection system based on convolutional neural networks IEEE Transactions on Intelligent Transportation Systems 2021
[37]
Zhang S, Zhang Q, Gu J, Su L, Li K, and Pecht M Visual inspection of steel surface defects based on domain adaptation and adaptive convolutional neural network Mechanical Systems and Signal Processing 2021 153
[38]
Zhao X, Sicilia A, Minhas DS, O’Connor EE, Aizenstein HJ, Klunk WE, Tudorascu DL, and Hwang SJ Robust white matter hyperintensity segmentation on unseen domain 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) 2021
[39]
Zhou, K., Yang, Y., Qiao, Y., & Xiang, T. (2021). Mixstyle neural networks for domain generalization and adaptation. arXiv preprint arXiv:2107.02053.
Index Terms
- Cross-scale fusion and domain adversarial network for generalizable rail surface defect segmentation on unseen datasets
Index terms have been assigned to the content through auto-classification.
Recommendations
Adversarial Feature Augmentation for Cross-domain Few-Shot Classification
Computer Vision – ECCV 2022AbstractFew-shot classification is a promising approach to solving the problem of classifying novel classes with only limited annotated data for training. Existing methods based on meta-learning predict novel-class labels for (target domain) testing tasks ...
SiamDoGe: Domain Generalizable Semantic Segmentation Using Siamese Network
Computer Vision – ECCV 2022AbstractDeep learning-based approaches usually suffer from performance drop on out-of-distribution samples, therefore domain generalization is often introduced to improve the robustness of deep models. Domain randomization (DR) is a common strategy to ...
Discriminative adversarial domain generalization with meta-learning based cross-domain validation
AbstractThe generalization capability of machine learning models, which refers to generalizing the knowledge for an “unseen” domain via learning from one or multiple seen domain(s), is of great importance to develop and deploy machine learning ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 27 November 2022
Accepted: 29 October 2022
Received: 26 June 2022
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 11 Aug 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in