Cited By
View all- Fraccaroli MBizzarri ACasellati PLamma E(2024)Exploiting CNN’s visual explanations to drive anomaly detectionApplied Intelligence10.1007/s10489-023-05177-054:1(414-427)Online publication date: 1-Jan-2024
GRD-Net: : Generative-Reconstructive-Discriminative Anomaly Detection with Region of Interest Attention Module
Authors: 
Niccolò Ferrari, Michele Fraccaroli, Evelina Lamma Academic Editor: Mohammad R. KhosraviAuthors Info & Claims
Niccolò Ferrari, Michele Fraccaroli, Evelina Lamma Academic Editor: Mohammad R. KhosraviAuthors Info & ClaimsAbstract
Anomaly detection is nowadays increasingly used in industrial applications and processes. One of the main fields of the appliance is the visual inspection for surface anomaly detection, which aims to spot regions that deviate from regularity and consequently identify abnormal products. Defect localization is a key task that is usually achieved using a basic comparison between generated image and the original one, implementing some blob analysis or image-editing algorithms in the postprocessing step, which is very biased towards the source dataset, and they are unable to generalize. Furthermore, in industrial applications, the totality of the image is not always interesting but could be one or some regions of interest (ROIs), where only in those areas there are relevant anomalies to be spotted. For these reasons, we propose a new architecture composed by two blocks. The first block is a generative adversarial network (GAN), based on a residual autoencoder (ResAE), to perform reconstruction and denoising processes, while the second block produces image segmentation, spotting defects. This method learns from a dataset composed of good products and generated synthetic defects. The discriminative network is trained using a ROI for each image contained in the training dataset. The network will learn in which area anomalies are relevant. This approach guarantees the reduction of using preprocessing algorithms, formerly developed with blob analysis and image-editing procedures. To test our model, we used challenging MVTec anomaly detection datasets and an industrial large dataset of pharmaceutical BFS strips of vials. This set constitutes a more realistic use case of the aforementioned network.
References
[1]
D. Bank, N. Koenigstein, and R. Giryes, “Autoencoders,” 2020, https://arxiv.org/abs/2003.05991.
[2]
U. Michelucci, “An introduction to autoencoders,” 2022, https://arxiv.org/abs/2201.03898.
[3]
D. P. Kingma and M. Welling, “An introduction to variational autoencoders,” 2019, https://arxiv.org/abs/1906.02691.
[4]
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” 2013, https://arxiv.org/abs/1312.6114.
[5]
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” Communications of the ACM, vol. 63, no. 11, pp. 139–144, 2020.
[6]
V. Zavrtanik, M. Kristan, and D. Skočaj, “Draem-a discriminatively trained reconstruction embedding for surface anomaly detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8330–8339, Montreal, BC, Canada, October 2021.
[7]
S. Akcay, A. Atapour-Abarghouei, and T. P. Breckon, “Ganomaly: semi-supervised anomaly detection via adversarial training,” in Asian Conference on Computer Vision, pp. 622–637, Springer, Berlin, Germany, 2018.
[8]
C. Wickramasinghe, D. Marino, and M. Manic, “resnet autoencoders for unsupervised feature learning from high-dimensional data: deep models resistant to performance degradation,” IEEE Access, vol. 9, p. 1, 2021.
[9]
S. Akçay, A. Atapour-Abarghouei, and T. P. Breckon, “Skip-ganomaly: skip connected and adversarially trained encoder-decoder anomaly detection,” in Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8, IEEE, Budapest, Hungary, July 2019.
[10]
B. Paul, S. Löwe, M. Fauser, D. Sattlegger, and C. Steger, “Improving unsupervised defect segmentation by applying structural similarity to autoencoders,” 2018, https://arxiv.org/abs/1807.02011.
[11]
D. Gong, L. Liu, V. Le, B. Saha, M. R. Mansour, S. Venkatesh, and H. Anton van den, “Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1705–1714, Montreal, BC, Canada, June 2019.
[12]
S. Venkataramanan, K.-C. Peng, R. V. Singh, and A. Mahalanobis, “Attention guided anomaly localization in images,” in European Conference on Computer Vision, pp. 485–503, Springer, Berlin, Germany, 2020.
[13]
S. Pidhorskyi, R. Almohsen, and G. Doretto, “Generative probabilistic novelty detection with adversarial autoencoders,” Advances in Neural Information Processing Systems, vol. 31, 2018.
[14]
M. Sabokrou, M. Khalooei, M. Fathy, and E. Adeli, “Adversarially learned one-class classifier for novelty detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3379–3388, Vancouver, BC, Canada, June 2018.
[15]
X. Xia, X. Pan, N. Li, X. He, L. Ma, X. Zhang, and N. Ding, “Gan-based anomaly detection: a review,” Neurocomputing, vol. 493, pp. 497–535, 2022.
[16]
B. Paul, M. Fauser, D. Sattlegger, and C. Steger, “Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9592–9600, Long Beach, CA, USA, June 2019.
[17]
X. Xie, Y. Huang, W. Ning, D. Wu, Z. Li, and H. Yang, “Rdad: a reconstructive and discriminative anomaly detection model based on transformer,” International Journal of Intelligent Systems, vol. 37, no. 11, pp. 8928–8946, 2022.
[18]
P. Perera, N. Ramesh, and B. Xiang, “Ocgan: one-class novelty detection using gans with constrained latent representations,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2898–2906, Baltimore, MD, USA, June 2019.
[19]
O. Rippel, P. Mertens, and D. Merhof, “Modeling the distribution of normal data in pre-trained deep features for anomaly detection,” in Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), pp. 6726–6733, IEEE, Milan, Italy, January 2021.
[20]
L. Bergman, N. Cohen, and Y. Hoshen, “Deep nearest neighbor anomaly detection,” 2020, https://arxiv.org/abs/2002.10445.
[21]
P. Napoletano, F. Piccoli, and R. Schettini, “Anomaly detection in nanofibrous materials by cnn-based self-similarity,” Sensors, vol. 18, no. 2, p. 209, 2018.
[22]
N. Cohen and Y. Hoshen, “Sub-image anomaly detection with deep pyramid correspondences,” 2020, https://arxiv.org/abs/2005.02357.
[23]
T. Defard, A. Setkov, A. Loesch, and R. Audigier, “Padim: a patch distribution modeling framework for anomaly detection and localization,” in Proceedings of the International Conference on Pattern Recognition, pp. 475–489, Springer, Berlin, Germany, March 2021.
[24]
K. Roth, L. Pemula, J. Zepeda, B. Schölkopf, T. Brox, and P. Gehler, “Towards total recall in industrial anomaly detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14318–14328, New Orleans, LA, USA, June 2022.
[25]
L. Dinh, J. Sohl-Dickstein, and S. Bengio, “Density estimation using real nvp,” 2016, https://arxiv.org/abs/1605.08803.
[26]
Y. Shi, J. Yang, and Z. Qi, “Unsupervised anomaly segmentation via deep feature reconstruction,” Neurocomputing, vol. 424, pp. 9–22, 2021.
[27]
T. Schlegl, P. Seeböck, S. M. Waldstein, U. Schmidt-Erfurth, and G. Langs, “Unsupervised anomaly detection with generative adversarial networks to guide marker discovery,” in International conference on information processing in medical imaging Lecture Notes in Computer Science, vol. 10265, M. Niethammer, Ed., pp. 146–157, Springer, Berlin, Germany, 2017.
[28]
T. Schlegl, P. Seeböck, S. M. Waldstein, G. Langs, and U. Schmidt-Erfurth, “f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks,” Medical Image Analysis, vol. 54, pp. 30–44, 2019.
[29]
M. Rudolph, B. Wandt, and B. Rosenhahn, “Same same but differnet: semi-supervised defect detection with normalizing flows,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 1907–1916, Hanover, Germany, Augest 2021.
[30]
D. Gudovskiy, S. Ishizaka, and K. Kozuka, “Cflow-ad: real-time unsupervised anomaly detection with localization via conditional normalizing flows,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 98–107, Tokyo, Japan, July 2022.
[31]
J. Bae, J.-H. Lee, and S. Kim, “Pni: industrial anomaly detection using position and neighborhood information,” 2023, https://arxiv.org/abs/2211.12634.
[32]
B. Paul, M. Fauser, D. Sattlegger, and C. Steger, “Uninformed students: student-teacher anomaly detection with discriminative latent embeddings,” 2019, https://arxiv.org/abs/1911.02357.
[33]
P. Bergmann, K. Batzner, M. Fauser, D. Sattlegger, and C. Steger, “Beyond dents and scratches: logical constraints in unsupervised anomaly detection and localization,” International Journal of Computer Vision, vol. 130, no. 4, pp. 947–969, 2022.
[34]
K. Batzner, L. Heckler, and R. König, “Efficientad: accurate visual anomaly detection at millisecond-level latencies,” 2023, https://arxiv.org/abs/2303.14535.
[35]
D. Li, D. Chen, L. Shi, B. Jin, J. Goh, and S.-K. Ng, “Mad-gan: multivariate anomaly detection for time series data with generative adversarial networks,” 2019, https://arxiv.org/abs/1901.04997.
[36]
G. Zhu, H. Zhao, H. Liu, and H. Sun, “A novel lstm-gan algorithm for time series anomaly detection,” in 2019 Prognostics and System Health Management Conference (PHM-Qingdao), pp. 1–6, Qingdao, China, October 2019.
[37]
Z. Niu, K. Yu, and X. Wu, “Lstm-based vae-gan for time-series anomaly detection,” Sensors, vol. 20, no. 13, p. 3738, 2020.
[38]
W. Jiang, Y. Hong, B. Zhou, X. He, and C. Cheng, “A gan-based anomaly detection approach for imbalanced industrial time series,” IEEE Access, vol. 7, pp. 143608–143619, 2019.
[39]
P. Radoglou Grammatikis, P. Sarigiannidis, G. Efstathopoulos, and E. Panaousis, “Aries: a novel multivariate intrusion detection system for smart grid,” Sensors, vol. 20, no. 18, p. 5305, 2020.
[40]
I. Siniosoglou, P. Radoglou-Grammatikis, G. Efstathopoulos, P. Fouliras, and P. Sarigiannidis, “A unified deep learning anomaly detection and classification approach for smart grid environments,” IEEE Transactions on Network and Service Management, vol. 18, no. 2, pp. 1137–1151, 2021.
[41]
D. Chen, L. Yue, X. Chang, M. Xu, and T. Jia, “Nm-gan: noise-modulated generative adversarial network for video anomaly detection,” Pattern Recognition, vol. 116, 2021.
[42]
K. Perlin, “An image synthesizer,” ACM Siggraph Computer Graphics, vol. 19, no. 3, pp. 287–296, 1985.
[43]
Z. Wang, A. C. Bovik, H. R. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004.
[44]
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, Cambridge, MA, USA, 2016.
[45]
A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and A. A. Bharath, “Generative adversarial networks: an overview,” IEEE Signal Processing Magazine, vol. 35, no. 1, pp. 53–65, 2018.
[46]
A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey, “Adversarial autoencoders,” 2015, https://arxiv.org/abs/1511.05644.
[47]
M. Mirza and O. Simon, “Conditional generative adversarial nets,” 2014, https://arxiv.org/abs/1411.1784.
[48]
T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 2980–2988, 2017.
[49]
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” 2015, https://arxiv.org/abs/1512.03385.
[50]
M. Fraccaroli, A. Bizzarri, P. Casellati, and E. Lamma, “Exploiting cnn’s visual explanations to drive anomaly detection,” Submitted to Applied Intelligence, Springer, Berlin, Germany, 2021.
[51]
M. Fraccaroli, A. Bizzarri, P. Casellati, and E. Lamma, “Cross entropy overlap distance,” 2022, https://ml.unife.it/wp-content/uploads/Papers/FraBizCasLam-ITAL_IA22.pdf.
Recommendations
RADDA-Net: Residual attention-based dual discriminator adversarial network for surface defect detection
AbstractSurface defect detection is a critical yet challenging task in the product production process, mainly because of the complex background texture on the material surface, the large intra-class defect differences, and the weak texture defect ...
USE-Net: Incorporating Squeeze-and-Excitation blocks into U-Net for prostate zonal segmentation of multi-institutional MRI datasets
Highlights- We propose USE-Net that incorporates Squeeze-and-Excitation blocks into U-Net.
- ...
Graphical abstractDisplay Omitted
AbstractProstate cancer is the most common malignant tumors in men but prostate Magnetic Resonance Imaging (MRI) analysis remains challenging. Besides whole prostate gland segmentation, the capability to differentiate between the blurry ...
OLR-Net: Object Label Retrieval Network for principal diagnosis extraction
Abstract Background:Extracting principal diagnosis from patient discharge summaries is an essential task for the meaningful use of medical data. The extraction process, usually by medical staff, is laborious and time-consuming. Although automatic models ...
Graphical abstractDisplay Omitted
Highlights- OLR-Net has been introduced to extract principle diagnosis from discharge summaries.
- Region proposals can locate diagnoses of interest from long text of medical record.
- Hierarchical self-attention can make use of hidden ...
Comments
Information & Contributors
Information
Published In
Copyright © 2023 Niccolò Ferrari et al.
This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Publisher
John Wiley and Sons Ltd.
United Kingdom
Publication History
Published: 01 January 2023
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
Cited By
View all- Fraccaroli MBizzarri ACasellati PLamma E(2024)Exploiting CNN’s visual explanations to drive anomaly detectionApplied Intelligence10.1007/s10489-023-05177-054:1(414-427)Online publication date: 1-Jan-2024