Image Hash Layer Triggered CNN Framework for Wafer Map Failure Pattern Retrieval and Classification
Article No.: 75, Pages 1 - 26
Abstract
Recently, deep learning methods are often used in wafer map failure pattern classification. CNN requires less feature engineering but still needs preprocessing, e.g., denoising and resizing. Denoising is used to improve the quality of the input data, and resizing is used to transform the input into an identical size when the input data sizes are various. However, denoising and resizing may distort the original data information. Nevertheless, CNN-based applications are focusing on studying different feature map architectures and the input data manipulation is less attractive. In this study, we proposed an image hash layer triggered CNN framework for wafer map failure pattern retrieval and classification. The motivation and novelty are to design a CNN layer that can play as a resizing, information retrieval-preservation method in one step. The experiments proved that the proposed hash layer can retrieve the failure pattern information while maintaining the classification performance even though the input data size is decreased significantly. In the meantime, it can prevent overfitting, false negatives, and false positives, and save computing costs to a certain extent.
Appendix
A Evaluation Metric
A confusion matrix is a table that is often used to evaluate the performance of a classification. The data used for the evaluation should have labeled targets. The Table 5 shows the confusion matrix, and in the Table 5:
True Positive (TP): TP is the correctly predicted positive value which means that the value of the actual class positive is predicted as positive.
True Negative (TN): TN is the correctly predicted negative value which means that the value of the actual class negative is predicted as negative.
False Positive (FP): FP is the incorrectly predicted positive value which means that the value of the actual class negative is predicted as positive.
False Negative (FN): FN is the incorrectly predicted negative value which means that the value of the actual class positive is predicted as negative.
We can calculate Accuracy, Precision, and Recall based on these four parameters.
\begin{equation} Accuracy=\frac{TP+TN}{TP+TN+FP+FN}=\frac{Correctly \; Predicted}{All \; Positives \; and \; Negatives} \end{equation}
(6)
\begin{equation} Precision=\frac{TP}{TP+FP}=\frac{True \; Positive}{Total \; Predicted \; Positives} \end{equation}
(7)
\begin{equation} Recall=\frac{TP}{TP+FN}=\frac{True \; Positive}{Total \; Actual \; Positives} \end{equation}
(8)
B More Details of Experimental Results
B.1 More Details of Machine Learning based Hash Evaluation
More details of the Table 2 and Table 3 are shown in Appendix Table 6, Table 7, and Table 8. The evaluation is performed on five data sets, respectively.
B.2 More Details of Hash-CNN vs. 2D-CNN on Different Resized Images
The Figure 12 shows the average accuracy comparison of 2D-CNN and Hash-CNN. 2D-CNN is trained on resized images of 16 × 16, 32 × 32, and 64 × 64, respectively. Figure 13 shows the precision and recall comparison between Hash-CNN and 2D-CNN on training data.
B.3 More Details of Hash-CNN vs. 2D-CNN on Padded Image
The Figure 14 shows the max and min training accuracy of 2D-CNN and Hash-CNN during 100 epochs, and the Figure 15 shows the max and min test accuracy of 2D-CNN and Hash-CNN during 100 epochs. We can see that 2D-CNN outperforms Hash-CNN on training data. However, on the test data, as shown in the Figure 15, the performance of Hash-CNN becomes closer to 2D-CNN.
Fig. 14.
Fig. 15.
The Table 9 shows the comparison between 2D-CNN and Hash-CNN on the training and test data. We can see that the average precision of both models is higher than the accuracy. That means that false positives are a smaller part of our study. The average recall of 2D-CNN on training is lower than precision, while the average recall of Hash-CNN is higher than the other two metrics on training and test data. We are concerned with reducing false negatives and false positives, so we can say that Hash-CNN is better than 2D-CNN.
Source Code Availability Statement
All models, or codes that support the findings of this study are proprietary and may only be provided upon reasonable request to [email protected].
References
[1]
M. Abd. Al Rahman, Sebelan Danishvar, and Alireza Mousavi. 2021. An improved capsule network (WaferCaps) for wafer bin map classification based on DCGAN data upsampling. IEEE Transactions on Semiconductor Manufacturing 35, 1 (2021), 50–59.
[2]
Fatima Adly, Omar Alhussein, Paul D. Yoo, Yousof Al-Hammadi, Kamal Taha, Sami Muhaidat, Young-Seon Jeong, Uihyoung Lee, and Mohammed Ismail. 2015. Simplified subspaced regression network for identification of defect patterns in semiconductor wafer maps. IEEE Transactions on Industrial Informatics 11, 6 (2015), 1267–1276.
[3]
Fatima Adly, Paul D. Yoo, Sami Muhaidat, Yousof Al-Hammadi, Uihyoung Lee, and Mohammed Ismail. 2015. Randomized general regression network for identification of defect patterns in semiconductor wafer maps. IEEE Transactions on Semiconductor Manufacturing 28, 2 (2015), 145–152.
[4]
Ramy Baly and Hazem Hajj. 2012. Wafer classification using support vector machines. IEEE Transactions on Semiconductor Manufacturing 25, 3 (2012), 373–383.
[5]
Mikhail Belkin and Partha Niyogi. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation 15, 6 (2003), 1373–1396.
[6]
Jaegyeong Cha and Jongpil Jeong. 2022. Improved U-Net with residual attention block for mixed-defect wafer maps. Applied Sciences 12, 4 (2022), 2209.
[7]
Fei-Long Chen and Shu-Fan Liu. 2000. A neural-network approach to recognize defect spatial pattern in semiconductor fabrication. IEEE Transactions on Semiconductor Manufacturing 13, 3 (2000), 366–373.
[8]
Shouhong Chen, Yuxuan Zhang, Mulan Yi, Yuling Shang, and Ping Yang. 2021. AI classification of wafer map defect patterns by using dual-channel convolutional neural network. Engineering Failure Analysis 130 (2021), 105756.
[9]
Chen-Fu Chien, Wen-Chih Wang, and Jen-Chieh Cheng. 2007. Data mining for yield enhancement in semiconductor manufacturing and an empirical study. Expert Systems with Applications 33, 1 (2007), 192–198.
[10]
Jong-Chih Chien, Ming-Tao Wu, and Jiann-Der Lee. 2020. Inspection and classification of semiconductor wafer surface defects using CNN deep learning networks. Applied Sciences 10, 15 (2020), 5340.
[11]
Gyunghyun Choi, Sung-Hee Kim, Chunghun Ha, and Suk Joo Bae. 2012. Multi-step ART1 algorithm for recognition of defect patterns on semiconductor wafers. International Journal of Production Research 50, 12 (2012), 3274–3287.
[12]
Mengying Fan, Qin Wang, and Ben van der Waal. 2016. Wafer defect patterns recognition based on OPTICS and multi-label classification. In 2016 IEEE Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC). IEEE, 912–915.
[13]
Ricardo Fitas, Bernardo Rocha, Valter Costa, and Armando Sousa. 2021. Design and comparison of image hashing methods: A case study on cork stopper unique identification. Journal of Imaging 7, 3 (2021), 48.
[14]
Chenn-Jung Huang. 2007. Clustered defect detection of high quality chips using self-supervised multilayer perceptron. Expert Systems with Applications 33, 4 (2007), 996–1003.
[15]
Cheng Hao Jin, Hyun-Jin Kim, Yongjun Piao, Meijing Li, and Minghao Piao. 2020. Wafer map defect pattern classification based on convolutional neural network features and error-correcting output codes. Journal of Intelligent Manufacturing (2020), 1–15.
[16]
Cheng Hao Jin, Hyuk Jun Na, Minghao Piao, Gouchol Pok, and Keun Ho Ryu. 2019. A novel DBSCAN-based defect pattern detection and classification framework for wafer bin map. IEEE Transactions on Semiconductor Manufacturing (2019).
[17]
Lu Jin, Xiangbo Shu, Kai Li, Zechao Li, Guo-Jun Qi, and Jinhui Tang. 2018. Deep ordinal hashing with spatial attention. IEEE Transactions on Image Processing 28, 5 (2018), 2173–2186.
[18]
Byunghoon Kim, Young-Seon Jeong, Seung Hoon Tong, In-Kap Chang, and Myong-Kee Jeongyoung. 2015. A regularized singular value decomposition-based approach for failure pattern classification on fail bit map in a DRAM wafer. IEEE Transactions on Semiconductor Manufacturing 28, 1 (2015), 41–49.
[19]
Neal Krawetz. 2023. Difference Hash. https://www.hackerfactor.com/blog/?/archives/529-Kind-of-Like-That.html
[20]
MIR Lab. 2023. LSWMD Data. http://mirlab.org/dataSet/public/
[21]
Hanjiang Lai, Pan Yan, Xiangbo Shu, Yunchao Wei, and Shuicheng Yan. 2016. Instance-aware hashing for multi-label image retrieval. IEEE Transactions on Image Processing 25, 6 (2016), 2469–2479.
[22]
Xu Lu, Li Liu, Liqiang Nie, Xiaojun Chang, and Huaxiang Zhang. 2020. Semantic-driven interpretable deep multi-modal hashing for large-scale multimedia retrieval. IEEE Transactions on Multimedia (2020).
[23]
P. Mohanaiah, P. Sathyanarayana, and L. GuruKumar. 2013. Image texture feature extraction using GLCM approach. International Journal of Scientific and Research Publications 3, 5 (2013), 1.
[24]
Melanie Po-Leen Ooi, Hong Kuan Sok, Ye Chow Kuang, Serge Demidenko, and Chris Chan. 2013. Defect cluster recognition system for fabricated semiconductor wafers. Engineering Applications of Artificial Intelligence 26, 3 (2013), 1029–1043.
[25]
Dmitry Petrov. 2022. Wavelet Image Hash in Python. https://fullstackml.com/wavelet-image-hash-in-python-3504fdd282b5
[26]
Minghao Piao, Cheng Hao Jin, Jong Yun Lee, and Jeong-Yong Byun. 2018. Decision tree ensemble-based wafer map failure pattern recognition based on radon transform-based features. IEEE Transactions on Semiconductor Manufacturing 31, 2 (2018), 250–257.
[27]
Qibing Qin, Zhiqiang Wei, Lei Huang, Kezhen Xie, and Wenfeng Zhang. 2021. Deep top similarity hashing with class-wise loss for multi-label image retrieval. Neurocomputing 439 (2021), 302–315.
[28]
Sam T. Roweis and Lawrence K. Saul. 2000. Nonlinear dimensionality reduction by locally linear embedding. Science 290, 5500 (2000), 2323–2326.
[29]
Muhammad Saqlain, Qasim Abbas, and Jong Yun Lee. 2020. A deep convolutional neural network for wafer defect identification on an imbalanced dataset in semiconductor manufacturing processes. IEEE Transactions on Semiconductor Manufacturing (2020).
[30]
Prashant P. Shinde, Priyadarshini P. Pai, and Shashishekar P. Adiga. 2022. Wafer defect localization and classification using deep learning techniques. IEEE Access 10 (2022), 39969–39974.
[31]
Masashi Sugiyama. 2007. Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis. Journal of Machine Learning Research 8, May (2007), 1027–1061.
[32]
P. N. Tan, M. Steinbach, and V. Kumar. 2006. Ensemble methods. In Introduction to Data Mining. Vol. 5. Pearson Educ.
[33]
Shing Chiang Tan, Junzo Watada, Zuwairie Ibrahim, and Marzuki Khalid. 2014. Evolutionary fuzzy ARTMAP neural networks for classification of semiconductor defects. IEEE Transactions on Neural Networks and Learning Systems 26, 5 (2014), 933–950.
[34]
Joshua B. Tenenbaum, Vin De Silva, and John C. Langford. 2000. A global geometric framework for nonlinear dimensionality reduction. Science 290, 5500 (2000), 2319–2323.
[35]
Laurens Van Der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9 (2008), 2579–2605.
[36]
C.-H. Wang, S.-J. Wang, and W.-D. Lee. 2006. Automatic identification of spatial defect patterns for semiconductor manufacturing. International Journal of Production Research 44, 23 (2006), 5169–5185.
[37]
Fu-Kwun Wang, Jia-Hong Chou, and Zemenu Endalamaw Amogne. 2022. A deep convolutional neural network with residual blocks for wafer map defect pattern recognition. Quality and Reliability Engineering International 38, 1 (2022), 343–357.
[38]
Rui Wang and Nan Chen. 2020. Defect pattern recognition on wafers using convolutional neural networks. Quality and Reliability Engineering International 36, 4 (2020), 1245–1257.
[39]
Ming-Ju Wu, Jyh-Shing R. Jang, and Jui-Long Chen. 2014. Wafer map failure pattern recognition and similarity ranking for large-scale data sets. IEEE Transactions on Semiconductor Manufacturing 28, 1 (2014), 1–12.
[40]
Qiao Xu, Naigong Yu, and Firdaous Essaf. 2022. Improved wafer map inspection using attention mechanism and cosine normalization. Machines 10, 2 (2022), 146.
[41]
Chenggang Yan, Biao Gong, Yuxuan Wei, and Yue Gao. 2020. Deep multi-view enhancement hashing for image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 4 (2020), 1445–1451.
[42]
Bian Yang, Fan Gu, and Xiamu Niu. 2006. Block mean value based image perceptual hashing. In 2006 International Conference on Intelligent Information Hiding and Multimedia. IEEE, 167–172.
[43]
Suhee Yoon and Seokho Kang. 2022. Semi-automatic wafer map pattern classification with convolutional neural networks. Computers & Industrial Engineering 166 (2022), 107977.
[44]
Jianbo Yu. 2011. Bearing performance degradation assessment using locality preserving projections and Gaussian mixture models. Mechanical Systems and Signal Processing 25, 7 (2011), 2573–2588.
[45]
Jianbo Yu. 2011. Fault detection using principal components-based Gaussian mixture model for semiconductor manufacturing processes. IEEE Transactions on Semiconductor Manufacturing 24, 3 (2011), 432–444.
[46]
Jianbo Yu. 2012. Semiconductor manufacturing process monitoring using Gaussian mixture model and Bayesian method with local and nonlocal information. IEEE Transactions on Semiconductor Manufacturing 25, 3 (2012), 480–493.
[47]
Jianbo Yu and Jiatong Liu. 2020. Two-dimensional principal component analysis-based convolutional autoencoder for wafer map defect detection. IEEE Transactions on Industrial Electronics (2020).
[48]
Jianbo Yu and Xiaolei Lu. 2015. Wafer map defect detection and recognition using joint local and nonlocal linear discriminant analysis. IEEE Transactions on Semiconductor Manufacturing 29, 1 (2015), 33–43.
[49]
Jianbo Yu, Zongli Shen, and Shijin Wang. 2021. Wafer map defect recognition based on deep transfer learning-based densely connected convolutional network and deep forest. Engineering Applications of Artificial Intelligence 105 (2021), 104387.
[50]
Tao Yuan, Way Kuo, and Suk Joo Bae. 2011. Detection of spatial defect patterns generated in semiconductor fabrication processes. IEEE Transactions on Semiconductor Manufacturing 24, 3 (2011), 392–403.
[51]
Christoph Zauner. 2010. Implementation and benchmarking of perceptual image hash functions. Master’s thesis, Upper Austria University of Applied Sciences, Austria (2010).
[52]
Qing Zhang, Yuhang Zhang, Jizuo Li, and Yongfu Li. 2022. WDP-BNN: Efficient wafer defect pattern classification via binarized neural network. Integration 85 (2022), 76–86.
[53]
Huilin Zheng, Syed Waseem Abbas Sherazi, Sang Hyeok Son, and Jong Yun Lee. 2021. A deep convolutional neural network-based multi-class image classification for automatic wafer map failure recognition in semiconductor manufacturing. Applied Sciences 11, 20 (2021), 9769.
[54]
Lei Zhu, Zi Huang, Zhihui Li, Liang Xie, and Heng Tao Shen. 2018. Exploring auxiliary context: Discrete semantic transfer hashing for scalable image retrieval. IEEE Transactions on Neural Networks and Learning Systems 29, 11 (2018), 5264–5276.
[55]
Lei Zhu, Chaoqun Zheng, Xu Lu, Zhiyong Cheng, Liqiang Nie, and Huaxiang Zhang. 2021. Efficient multi-modal hashing with online query adaption for multimedia retrieval. ACM Transactions on Information Systems (TOIS) 40, 2 (2021), 1–36.
[56]
Xiaofeng Zhu, Zi Huang, Hong Cheng, Jiangtao Cui, and Heng Tao Shen. 2013. Sparse hashing for fast multimedia search. ACM Transactions on Information Systems (TOIS) 31, 2 (2013), 1–24.
Index Terms
- Image Hash Layer Triggered CNN Framework for Wafer Map Failure Pattern Retrieval and Classification
Recommendations
CNN and ensemble learning based wafer map failure pattern recognition based on local property based features
AbstractThe combination of feature extraction and classification methods is an often used approach for wafer map failure pattern recognition. Recently, the convolutional neural network (CNN) method is used to raw wafer map data without feature extraction. ...
Ore Image Classification Based on Improved CNN
AbstractThe identification of ore deposits is an important technical task in mining and excavation. However, conventional techniques are time-consuming and tedious. Therefore, data augmentation and transfer learning were used in this topic to ...
A novel proposed CNN–SVM architecture for ECG scalograms classification
AbstractNowadays, the number of sudden deaths due to heart disease is increasing with the coronavirus pandemic. Therefore, automatic classification of electrocardiogram (ECG) signals is crucial for diagnosis and treatment. Thanks to deep learning ...
Comments
Information & Contributors
Information
Published In
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 13 February 2024
Online AM: 19 December 2023
Accepted: 15 December 2023
Revised: 13 April 2023
Received: 22 July 2022
Published in TKDD Volume 18, Issue 4
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 157Total Downloads
- Downloads (Last 12 months)157
- Downloads (Last 6 weeks)23
Reflects downloads up to 11 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in