PreAugNet: improve data augmentation for industrial defect classification with small-scale training data
Authors: 
Isack Farady, Chih-Yang Lin, and Ming-Ching ChangAuthors Info & Claims
Isack Farady, Chih-Yang Lin, and Ming-Ching ChangAuthors Info & ClaimsPages 1233 - 1246
Abstract
With the prevalence of deep learning and convolutional neural network (CNN), data augmentation is widely used for enriching training samples to gain model training improvement. Data augmentation is important when training samples are scarce. This work focuses on improving data augmentation for training an industrial steel surface defect classification network, where the performance is largely depending on the availability of high-quality training samples. It is very difficult to find a sufficiently large dataset for this application in real-world settings. When it comes to synthetic data augmentation, the performance is often degraded by incorrect class labels, and a large effort is required to generate high-quality samples. This paper introduces a novel off-line pre-augmentation network (PreAugNet) which acts as a class boundary classifier that can effectively screen the quality of the augmented samples and improve image augmentation. This PreAugNet can generate augmented samples and update decision boundaries via an independent support vector machine (SVM) classifier. New samples are automatically distributed and combined with the original data for training the target network. The experiments show that these new augmentation samples can improve classification without changing the target network architecture. The proposed method for steel surface defect inspection is evaluated on three real-world datasets: AOI steel defect dataset, MT, and NEU datasets. PreAugNet significantly increases the accuracy by 3.3% (AOI dataset), 6.25% (MT dataset) and 2.1% (NEU dataset), respectively.
References
[1]
Abiodun OI, Jantan A, Omolara AE, Dada KV, Mohamed NA, and Arshad H State-of-the-art in artificial neural network applications: a survey Heliyon 2018 4 11 e00938
[2]
Abu M, Amir A, Lean Y, Zahri N, and Azemi S The performance analysis of transfer learning for steel defect detection by using deep learning Journal of Physics: Conference Series 2021 1755 1 012041
[3]
Bowles, C., Chen, L., Guerrero, R., Bentley, P., Gunn, R., Hammers, A., Dickie, D. A., Hernández, M. V., Wardlaw, J., & Rueckert, D. (2018). Gan augmentation: Augmenting training data using generative adversarial networks. Preprint retrieved from https://arxiv.org/abs/1810.10863.
[4]
Burrascano P Learning vector quantization for the probabilistic neural network IEEE Transactions on Neural Networks 1991 2 4 458-461
[5]
Buslaev A, Iglovikov VI, Khvedchenya E, Parinov A, Druzhinin M, and Kalinin AA Albumentations: Fast and flexible image augmentations Information 2020 11 2 125
[6]
Chen H, Pang Y, Hu Q, and Liu K Solar cell surface defect inspection based on multispectral convolutional neural network Journal of Intelligent Manufacturing 2020 31 453-468
[7]
Cheon S, Lee H, Kim CO, and Lee SH Convolutional neural network for wafer surface defect classification and the detection of unknown defect class IEEE Transactions on Semiconductor Manufacturing 2019 32 2 163-170
[8]
Choe J, Lee S, and Shim H Attention-based dropout layer for weakly supervised single object localization and semantic segmentation IEEE Transactions on Pattern Analysis and Machine Intelligence 2020 43 12 4256-4271
[9]
Czimmermann T, Ciuti G, Milazzo M, Chiurazzi M, Roccella S, Oddo CM, and Dario P Visual-based defect detection and classification approaches for industrial applications—A survey Sensors 2020 20 5 1459
[10]
DeVries, T., & Taylor, G. W. (2017). Improved regularization of convolutional neural networks with cutout. Preprint retrived from https://arxiv.org/abs/1708.04552.
[11]
Elleuch M, Maalej R, and Kherallah M A new design based-SVM of the CNN classifier architecture with dropout for offline Arabic handwritten recognition Procedia Computer Science 2016 80 1712-1723
[12]
Farady I, Lin CY, Akhyar F, Roshini R, and Alex JSR Evaluation of data augmentation on surface defect detection 2021 IEEE international conference on consumer electronics-Taiwan (ICCE-TW) 2021 IEEE
[13]
Farady I, Sarkar MD, Chang WT, and Lin CY Evaluation of additional augmented images for steel surface defect detection 2022 IEEE international conference on consumer electronics-Taiwan 2022 IEEE 1-2
[14]
Frid-Adar M, Diamant I, Klang E, Amitai M, Goldberger J, and Greenspan H GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification Neurocomputing 2018 321 321-331
[15]
Ghiasi G, Lin TY, and Le QV Dropblock: A regularization method for convolutional networks Advances in Neural Information Processing Systems 2018 31 1-10
[16]
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, and Bengio Y Generative adversarial networks Communications of the ACM 2020 63 11 139-144
[17]
He K, Gkioxari G, Dollár P, and Girshick R Mask r-cnn Proceedings of the IEEE international conference on computer vision 2017 IEEE 2961-2969
[18]
He K, Zhang X, Ren S, and Sun J Deep residual learning for image recognition Proceeding of the IEEE conference on computer vision and pattern recognition 2016 IEEE 770-778
[19]
He Y, Song K, Dong H, and Yan Y Semi-supervised defect classification of steel surface based on multi-training and generative adversarial network Optics and Lasers in Engineering 2019 122 294-302
[20]
He Y, Song K, Meng Q, and Yan Y An end-to-end steel surface defect detection approach via fusing multiple hierarchical features IEEE Transactions on Instrumentation and Measurement 2019 69 4 1493-1504
[21]
Hernández-García, A., & König, P. (2018). Data augmentation instead of explicit regularization. Preprint retrieved from https://arxiv.org/abs/1806.03852.
[22]
Huang Y, Qiu C, Wang X, Wang S, and Yuan K A compact convolutional neural network for surface defect inspection Sensors 2020 20 7 1974
[23]
Huang Y, Qiu C, and Yuan K Surface defect saliency of magnetic tile The Visual Computer 2020 36 85-96
[24]
Jain S, Seth G, Paruthi A, Soni U, and Kumar G Synthetic data augmentation for surface defect detection and classification using deep learning Journal of Intelligent Manufacturing 2022 2022 1-14
[25]
Joshi KD, Chauhan V, and Surgenor B A flexible machine vision system for small part inspection based on a hybrid SVM/ANN approach Journal of Intelligent Manufacturing 2020 31 103-125
[26]
Kang J, Park YJ, Lee J, Wang SH, and Eom DS Novel leakage detection by ensemble CNN-SVM and graph-based localization in water distribution systems IEEE Transactions on Industrial Electronics 2017 65 5 4279-4289
[27]
Krizhevsky A, Sutskever I, and Hinton GE Imagenet classification with deep convolutional neural networks Communications of the ACM 2017 60 6 84-90
[28]
Kumar A Computer-vision-based fabric defect detection: a survey IEEE Transactions on Industrial Electronics 2008 55 1 348-363
[29]
LeCun Y, Bottou L, Bengio Y, and Haffner P Gradient-based learning applied to document recognition Proceedings of the IEEE 1998 86 11 2278-2324
[30]
Li C, Xu T, Zhu J, and Zhang B Triple generative adversarial nets Advances in Neural Information Processing Systems 2017 30 1
[31]
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016). Ssd: Single shot multibox detector. Computer Vision–ECCV 2016: 14th European Conference, The Netherlands, October 11–14, 2016.
[32]
Liu MY and Tuzel O Coupled generative adversarial networks Advances in Neural Information Processing Systems 2016 29 1
[33]
Luo Q, Fang X, Liu L, Yang C, and Sun Y Automated visual defect detection for flat steel surface: A survey IEEE Transactions on Instrumentation and Measurement 2020 69 3 626-644
[34]
Mao KZ, Tan KC, and Ser W Probabilistic neural-network structure determination for pattern classification IEEE Transactions on Neural Networks 2000 11 4 1009-1016
[35]
Marino S, Beauseroy P, and Smolarz A Unsupervised adversarial deep domain adaptation method for potato defects classification Computers and Electronics in Agriculture 2020 174 105501
[36]
Meireles MR, Almeida PE, and Simões MG A comprehensive review for industrial applicability of artificial neural networks IEEE Transactions on Industrial Electronics 2003 50 3 585-601
[37]
Ngan HY, Pang GK, and Yung NH Automated fabric defect detection—a review Image and Vision Computing 2011 29 7 442-458
[38]
Niu XX and Suen CY A novel hybrid CNN–SVM classifier for recognizing handwritten digits Pattern Recognition 2012 45 4 1318-1325
[39]
Pan H, Pang Z, Wang Y, Wang Y, and Chen L A new image recognition and classification method combining transfer learning algorithm and mobilenet model for welding defects Ieee Access 2020 8 119951-119960
[40]
Park JK, Kwon BK, Park JH, and Kang DJ Machine learning-based imaging system for surface defect inspection International Journal of Precision Engineering and Manufacturing-Green Technology 2016 3 303-310
[41]
Redmon J, Divvala S, Girshick R, and Farhadi A You only look once: unified, real-time object detection Proceedings of the EEE conference on computer vision and pattern recognition 2016 IEEE 779-788
[42]
Ren S, He K, Girshick R, and Sun J Faster r-cnn: Towards real-time object detection with region proposal networks Advances in Neural Information Processing Systems 2015 28 1
[43]
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 234-241
[44]
Saito K, Kim D, Sclaroff S, and Saenko K Universal domain adaptation through self supervision Advances in Neural Information Processing Systems 2020 33 16282-16292
[45]
Saritas MM and Yasar A Performance analysis of ANN and Naive Bayes classification algorithm for data classification International Journal of Intelligent Systems and Applications in Engineering 2019 7 2 88-91
[46]
Shorten C and Khoshgoftaar TM A survey on image data augmentation for deep learning Journal of Big Data 2019 6 1 1-48
[47]
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. Preprint retrived from https://arxiv.org/abs/1409.1556.
[48]
Singh KK and Lee YJ Hide-and-seek: Forcing a network to be meticulous for weakly-supervised object and action localization IEEE International Conference on Computer Vision (ICCV) 2017 2017 3544-3553
[49]
Sun X, Liu L, Li C, Yin J, Zhao J, and Si W Classification for remote sensing data with improved CNN-SVM method Ieee Access 2019 7 164507-164516
[50]
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, and Wojna Z Rethinking the inception architecture for computer vision Proceedings of the IEEE conference on computer vision and pattern recognition 2016 IEEE 2818-2826
[51]
Tan M and Le Q Efficientnet: rethinking model scaling for convolutional neural networks International Conference on Machine Learning 2019 2019 6105-6144
[52]
Vapnik VN An overview of statistical learning theory IEEE Transactions on Neural Networks 1999 10 5 988-999
[53]
Wong SC, Gatt A, Stamatescu V, and McDonnell MD Understanding data augmentation for classification: when to warp? 2016 international conference on digital image computing: techniques and applications (DICTA) 2016 IEEE 1-6
[54]
Xue DX, Zhang R, Feng H, and Wang YL CNN-SVM for microvascular morphological type recognition with data augmentation Journal of Medical and Biological Engineering 2016 36 755-764
[55]
Yang S, Wang Y, Van De Weijer J, Herranz L, and Jui S Generalized source-free domain adaptation Proceedings of the IEEE/CVF International Conference on Computer Vision 2021 IEEE 8978-8987
[56]
Yun JP, Shin WC, Koo G, Kim MS, Lee C, and Lee SJ Automated defect inspection system for metal surfaces based on deep learning and data augmentation Journal of Manufacturing Systems 2020 55 317-324
[57]
Yun S, Han D, Oh SJ, Chun S, Choe J, and Yoo Y Cutmix: Regularization strategy to train strong classifiers with localizable features Proceeding of the IEEE/CVF international conference on computer vision 2019 IEEE 6023-6032
[58]
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017). mixup: Beyond empirical risk minimization. Preprint retrived from https://arxiv.org/abs/1710.09412.
[59]
Zhang S, Zhang Q, Gu J, Su L, Li K, and Pecht M Visual inspection of steel surface defects based on domain adaptation and adaptive convolutional neural network Mechanical Systems and Signal Processing 2021 153 107541
[60]
Zhang W, Li X, Ma H, Luo Z, and Li X Transfer learning using deep representation regularization in remaining useful life prediction across operating conditions Reliability Engineering & System Safety 2021 211 107556
[61]
Zhang W, Li X, Ma H, Luo Z, and Li X Universal domain adaptation in fault diagnostics with hybrid weighted deep adversarial learning IEEE Transactions on Industrial Informatics 2021 17 12 7957-7967
[62]
Zhang Y, Wang Y, Jiang Z, Zheng L, Chen J, and Lu J Tire Defect Detection by Dual-Domain Adaptation-Based Transfer Learning Strategy IEEE Sensors Journal 2022 22 19 18804-18814
[63]
Zhong, J., Liu, X., & Hsieh, C.J. (2020a). Improving the speed and quality of gan by adversarial training. Preprint retrieved from https://arxiv.org/abs/2008.03364.
[64]
Zhong Z, Zheng L, Kang G, Li S, and Yang Y Random erasing data augmentation Proceedings of the AAAI Conference on Artificial Intelligence 2020 13 7 13001-13008
[65]
Zhu JY, Park T, Isola P, and Efros AA Unpaired image-to-image translation using cycle-consistent adversarial networks Proceedings of the IEEE international conference on computer vision 2017 IEEE 2223-2232
Recommendations
Improved Surface Defect Classification from a Simple Convolutional Neural Network by Image Preprocessing and Data Augmentation
Bioinspired Systems for Translational Applications: From Robotics to Social EngineeringAbstractConvolutional neural networks (CNNs) play an important role in an increasing number of image processing tasks. There is an obvious demand to improve their classification performance and efficiency. Current research in this area tends to focus on ...
Training data augmentation: An empirical study using generative adversarial net-based approach with normalizing flow models for materials informatics
AbstractWe address the issue of small data size for training models for regression problems, which is a significant issue in materials science. Many density estimators that use generative models based on deep neural networks have been ...
Highlights- We address training data augmentation issue on regression problems.
- A GAN-based ...
CycleGAN Based Data Augmentation For Melanoma images Classification
AIPR '20: Proceedings of the 2020 3rd International Conference on Artificial Intelligence and Pattern RecognitionIt is widely-known that melanoma is one of the deadliest skin cancers with a very high mortality rate, while it is curable with early identification. Therefore, early detection of melanoma is extremely necessary for the treatment of this disease. In ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 28 March 2023
Accepted: 07 March 2023
Received: 23 July 2022
Author Tags
Qualifiers
- Research-article
Funding Sources
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in