research-article

Real-time defect detection of TFT-LCD displays using a lightweight network architecture

Authors:

Zhongping Chen,

Songlin Chen, and

Xiang MoAuthors Info & Claims

Journal of Intelligent Manufacturing, Volume 35, Issue 3

Pages 1337 - 1352

https://doi.org/10.1007/s10845-023-02110-7

Published: 05 April 2023 Publication History

Abstract

The mura defects of thin film transistor-liquid crystal display (TFT-LCD) panels have low contrast and random locations, which makes it impossible for us to correctly evaluate the number and type of mura defects on the image in the field inspection. In response to the above problems, this paper proposes a lightweight YOLO-ADPAM detection method based on an attention mechanism. First, we designed a K-means-ciou++ clustering algorithm using the Complete-Intersection-Over-Union loss function to cluster the anchor box size of the display defect dataset, making the bounding box regression more accurate and stable and improving the recognition and positioning accuracy of the algorithm. Second, we design a parallel attention module, combining the advantages of the channel and spatial attention mechanisms to effectively extract helpful information from feature maps. The channel attention branch can compensate for the defect information lost by global average pooling to a certain extent, and selecting a larger convolution kernel in the spatial attention branch is beneficial to retain crucial spatial information. Third, using atrous spatial pyramid pooling and depthwise separable convolution in the Neck network can further improve the receptive field of the feature map and improve the detection accuracy of the network. The experimental results show that the mAP of our proposed YOLO-ADPAM algorithm in TFT-LCD defect detection reaches 98.20%, and the detection speed reaches 83.23 FPS, which meets the detection accuracy and real-time requirements of TFT-LCD defect detection tasks.

References

[1]

Bochkovskiy, A., Wang, C. Y., & Liao, H. Y. M. (2020). Yolov4: Optimal speed and accuracy of object detection. arXiv:2004.10934

[2]

Çelik A, Küçükmanisa A, Sümer A, et al. A real-time defective pixel detection system for LCDS using deep learning based object detectors Journal of Intelligent Manufacturing 2020

Digital Library

[3]

Chen M, Chen P, Wang S, et al. TFT-LCD mura defect visual inspection method in multiple backgrounds Journal of the Society for Information Display 2022

[4]

Chen LC, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFS IEEE Transactions on Pattern Analysis and Machine Intelligence 2017 40 4 834-848

[5]

Cui Y, Wang S, Wu H, et al. Liquid crystal display defects in multiple backgrounds with visual real-time detection Journal of the Society for Information Display 2021 29 7 547-560

[6]

Deng Y, Pan X, Wang X, et al. Vison-based 3D shape measurement system for transparent microdefect characterization IEEE Access 2019 7 105721-105733

[7]

Dong H, Song K, He Y, et al. Pga-net: Pyramid feature fusion and global context attention network for automated surface defect detection IEEE Transactions on Industrial Informatics 2019 16 12 7448-7458

[8]

Dong X, Taylor CJ, and Cootes TF A random forest-based automatic inspection system for aerospace welds in X-ray images IEEE Transactions on Automation Science and Engineering 2020 18 4 2128-2141

[9]

Ge, Z., Liu, S., Wang, F., et al. (2021) Yolox: Exceeding yolo series in 2021. arXiv:2107.08430.

[10]

He K, Zhang X, Ren S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition IEEE Transactions on Pattern Analysis and Machine Intelligence 2015 37 9 1904-1916

Digital Library

[11]

Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, (pp 7132–7141)

[12]

Kim M, Lee M, An M, et al. Effective automatic defect classification process based on CNN with stacking ensemble model for TFT-LCD panel Journal of Intelligent Manufacturing 2020 31 5 1165-1174

Digital Library

[13]

Kwak J, Lee KB, Jang J, et al. Automatic inspection of salt-and-pepper defects in OLED panels using image processing and control chart techniques Journal of Intelligent Manufacturing 2019 30 3 1047-1055

Digital Library

[14]

Le NT, Wang JW, Shih MH, et al. Novel framework for optical film defect detection and classification IEEE Access 2020 8 60,964-60,978

[15]

Lin, T. Y., Goyal, P., Girshick, R., et al. (2017). Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, (pp. 2980–2988)

[16]

Luo R, Chen R, Jia F, et al. RBD-NET: Robust breakage detection algorithm for industrial leather Journal of Intelligent Manufacturing 2022

Digital Library

[17]

Mei S, Cheng J, He X, et al. A novel weakly supervised ensemble learning framework for automated pixel-wise industry anomaly detection IEEE Sensors Journal 2021 22 2 1560-1570

[18]

Ming W, Zhang S, Liu X, et al. Survey of mura defect detection in liquid crystal displays based on machine vision Crystals 2021 11 12 1444

[19]

Pan Y, Lu R, and Zhang T FPGA-accelerated textured surface defect segmentation based on complete period Fourier reconstruction Journal of Real-Time Image Processing 2020 17 5 1659-1673

Digital Library

[20]

Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv:1804.02767

[21]

Ren S, He K, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks Advances in neural information processing systems 2015

Digital Library

[22]

Schlosser T, Friedrich M, Beuth F, et al. Improving automated visual fault inspection for semiconductor manufacturing using a hybrid multistage system of deep neural networks Journal of Intelligent Manufacturing 2022 33 4 1099-1123

Digital Library

[23]

Sun Y, Li X, and Xiao J A cascaded mura defect detection method based on mean shift and level set algorithm for active-matrix OLED display panel Journal of the Society for Information Display 2019 27 1 13-20

[24]

Tan, M., Pang, R., & Le, Q. V. (2020). Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10781–10790).

[25]

Wang, C.Y., Bochkovskiy, A., & Liao, H. Y. M. (2021). Scaled-yolov4: Scaling cross stage partial network. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition (pp. 13029–13038).

[26]

Wang, C. Y., Liao, H. Y. M., Wu, Y. H., et al. (2020a). CSPNET: A new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (pp 390–391).

[27]

Wang, Q., Wu, B., Zhu, P., et al. (2020b). Supplementary material for ECA-NET: Efficient channel attention for deep convolutional neural networks. In Proceedings of the 2020 IEEE/CVF conference on computer vision and pattern recognition, IEEE, Seattle, WA, USA (pp. 13–19)

[28]

Woo, S., Park, J., Lee, J. Y., et al. (2018). CBAM: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV) (pp. 3–19)

Digital Library

[29]

Yang H, Chen Y, Song K, et al. Multiscale feature-clustering-based fully convolutional autoencoder for fast accurate visual inspection of texture surface defects IEEE Transactions on Automation Science and Engineering 2019 16 3 1450-1467

[30]

Yang, H., Zhou, Q., Song, K., et al. (2020). An anomaly feature-editing-based adversarial network for texture defect visual inspection. IEEE Transactions on Industrial Informatics,17(3), 2220–2230.

[31]

Zheng, Z., Wang, P., Liu W., et al. (2020). Distance-IOU loss: Faster and better learning for bounding box regression. In: Proceedings of the AAAI conference on artificial intelligence (pp. 12993–13000)

[32]

Zhi, Z., Jiang, H., Yang, D., et al. (2022). An end-to-end welding defect detection approach based on titanium alloy time-of-flight diffraction images. Journal of Intelligent Manufacturing.

Digital Library

[33]

Zhou, X., Wang, D., & Krähenbühl, P. (2019). Objects as points. arXiv:1904.07850

[34]

Zhu, .X, Lyu, S., Wang, X., et al. (2021). Tph-yolov5: Improved yolov5 based on transformer prediction head for object detection on drone-captured scenarios. In: Proceedings of the IEEE/CVF international conference on computer vision (pp. 2778–2788)

Cited By

Liu QLiu MJonathan QShen W(2024)A real-time anchor-free defect detector with global and local feature enhancement for surface defect detectionExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123199246:COnline publication date: 15-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123199

Recommendations

The Defect Detection Using Human Visual System and Wavelet Transform in TFT-LCD Image
FBIT '07: Proceedings of the 2007 Frontiers in the Convergence of Bioscience and Information Technologies

The thin film transistor liquid crystal display (TFT-LCD) image has non-uniform brightness, which is the major difficulty in finding the defect region called Mura. To facilitate Mura segmentation, globally widely varying background signal has to be ...
Read More
Image enhancement for backlight-scaled TFT-LCD displays

One common way to extend the battery life of a portable device is to reduce the LCD backlight intensity. In contrast to previous approaches that minimize the power consumption by adjusting the backlight intensity frame by frame to reach a specified ...
Read More
Automatic detection of Mura defect in TFT-LCD based on regression diagnostics

This paper proposes a computationally efficient Mura defect detection method that is constructed based on regression diagnostics using the prediction error sum of squares (PRESS) residuals and an image estimation procedure for automatic Mura inspection ...
Read More

Comments

Information & Contributors

Information

Published In

cover image Journal of Intelligent Manufacturing

Journal of Intelligent Manufacturing Volume 35, Issue 3

Mar 2024

458 pages

ISSN:0956-5515

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 05 April 2023

Accepted: 07 March 2023

Received: 03 November 2022

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of China
Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Liu QLiu MJonathan QShen W(2024)A real-time anchor-free defect detector with global and local feature enhancement for surface defect detectionExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123199246:COnline publication date: 15-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123199

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents