Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Sparse one-dimensional convolutional neural network-based feature learning for fault detection and diagnosis in multivariable manufacturing processes

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Those fault detection and diagnosis (FDD) models can identify various faulty signals in industrial processes by extracting features from process data with high nonlinearity and correlations. However, the diagnostic performance of those models mainly depends primarily on the validity of the features extracted from the process data. In this paper, a novel deep neural network (DNN) model, sparse one-dimensional convolutional neural network (S1-DCNN), is proposed to learn features from process signals and improve the performance of FDD in industrial processes. S1-DCNN not only extracts discriminative features from complex process signals, but selects effective features based on a sparsity regularization in the convolution layers. Thus, an S1-DCNN-based representation learning method is developed for FDD in industrial processes. Tennessee Eastman process and fed-batch fermentation penicillin process are employed to validate effectiveness of S1-DCNN for FDD. The experimental results illustrate that S1-DCNN extracted and selected effective representative features for process FDD.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Butte VK, Tang LC (2010) Multivariate charting techniques: a review and a line-column approach. Qual Reliab Eng Int 26:443–451

    Google Scholar 

  2. Yu J, Zhang C, Wang S (2021) Multichannel one-dimensional convolutional neural network-based feature learning for fault diagnosis of industrial processes. Neural Comput Appl 33:3085–3104

    Google Scholar 

  3. Peres FAP, Fogliatto FS (2018) Variable selection methods in multivariate statistical process control: a systematic literature review. Comput Ind Eng 115:603–619

    Google Scholar 

  4. Jiang Q, Huang B (2016) Distributed monitoring for large-scale processes based on multivariate statistical analysis and Bayesian method. J Process Control 46:75–83

    Google Scholar 

  5. de Lázaro JMB, Moreno AP, Santiago OL, da Silva Neto AJ (2015) Optimizing kernel methods to reduce dimensionality in fault diagnosis of industrial systems. Comput Ind Eng 87:140–149

    Google Scholar 

  6. Wang B, Yan X, Jiang Q (2016) Independent component analysis model utilizing de-mixing information for improved non-Gaussian process monitoring. Comput Ind Eng 94:188–200

    Google Scholar 

  7. Luo L, Peng X, Tong C (2021) A multigroup framework for fault detection and diagnosis in large-scale multivariate systems. J Process Control 100:65–79

    Google Scholar 

  8. Bounoua W, Bakdi A (2021) Fault detection and diagnosis of nonlinear dynamical processes through correlation dimension and fractal analysis based dynamic kernel PCA. Chem Eng Sci 229:116099

    Google Scholar 

  9. Lee JM, Yoo CK, Lee IB (2004) Statistical process monitoring with independent component analysis. J Process Control 14(5):467–485

    Google Scholar 

  10. Jia Q, Zhang Y (2016) Quality-related fault detection approach based on dynamic kernel partial least squares. Chem Eng Res Des 106:242–252

    Google Scholar 

  11. Taqvi SAA, Zabiri H, Tufa LD, Uddin F, Fatima SA, Maulud AS (2021) A Review on data-driven learning approaches for fault detection and diagnosis in chemical processes, ChemBioEng Reviews

  12. Sun RX, Tsung F (2003) A kernel-distance-based multivariate control chart using support vector methods. Int J Prod Res 41(13):2975–2989

    MATH  Google Scholar 

  13. Hsu CC, Chen MC, Chen LS (2010) Intelligent ICA-SVM fault detector for non-Gaussian multivariate process monitoring. Expert Syst Appl 37(4):3264–3273

    Google Scholar 

  14. Chiang LH, Kotanchek ME, Kordon AK (2004) Fault diagnosis based on Fisher discriminant analysis and support vector machines. Comput Chem Eng 28(8):1389–1401

    Google Scholar 

  15. Yu J (2011) Localized Fisher discriminant analysis based complex chemical process monitoring. AIChE J 57(7):1817–1828

    Google Scholar 

  16. Gauri SK, Chakraborty S (2009) Recognition of control chart patterns using improved selection of features. Comput Ind Eng 56(4):1577–1588

    Google Scholar 

  17. Huda S, Abdollahian M, Mammadov M, Yearwood J, Ahmed S, Sultan I (2014) A hybrid wrapper-filter approach to detect the source(s) of out-of-control signals in multivariate manufacturing process. Eur J Oper Res 237(3):857–870

    Google Scholar 

  18. Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828

    Google Scholar 

  19. Wang J, Ma Y, Zhang L, Gao RX, Wu D (2018) Deep learning for smart manufacturing: methods and applications. J Manuf Syst 48:144–156

    Google Scholar 

  20. Yu J, Liu H, Zheng X (2020) Two-dimensional joint local and nonlocal discriminant analysis-based 2D image feature extraction for deep learning. Neural Comput Appl 32:6009–6024

    Google Scholar 

  21. Zhang Z, Jiang T, Li S, Yang Y (2018) Automated feature learning for nonlinear process monitoring-An approach using stacked denoising autoencoder and k-nearest neighbor rule. J Process Control 64:49–61

    Google Scholar 

  22. Ren H, Chai Y, Qu JF, Zhang K, Tang Q (2018) An intelligent fault detection method based on sparse auto-encoder for industrial process systems: a case study on tennessee eastman process chemical system. In: International conference on intelligent human-machine systems and cybernetics, pp 190–193

  23. Chen S, Yu J, Wang S (2020) Monitoring of complex profiles based on deep stacked denoising autoencoders. Comput Ind Eng, p 106402

  24. Yu J, Zheng X (2019) Stacked convolutional sparse denoising auto-encoder for identification of defect patterns in semiconductor wafer map. Comput Ind 109:121–133

    MathSciNet  Google Scholar 

  25. Liu X, Jianbo Yu, Ye L (2021) Residual attention convolutional auto-encoder for feature learning and fault detection in nonlinear industrial processes. Neural Comput Appl 33(8):3085–3104

    Google Scholar 

  26. Yu JB, Yan X (2018) Layer-by-Layer enhancement strategy of favorable features of the deep belief network for industrial process monitoring. Ind Eng Chem Res 57(45):15479–15490

    MathSciNet  Google Scholar 

  27. Wang Y, Pan Z, Yuan X, Yang C, Gui W (2019) A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network. ISA Trans. https://doi.org/10.1016/j.isatra.2019.07.001

    Article  Google Scholar 

  28. Hu G, Li H, Xia Y, Luo L (2018) A deep Boltzmann machine and multi-grained scanning forest ensemble collaborative method and its application to industrial fault diagnosis. Comput Ind 100:287–296

    Google Scholar 

  29. LeCun Y, Bengio Y (1995) Convolutional networks for images, speech, and time series, Handb Brain Theory Neural Netw, 3361(10): 1995

  30. Ciresan DC, Meier U, Masci J, Gambardella LM, Schmidhuber J (2011) Flexible, high performance convolutional neural networks for image classification. In: Twenty-second international joint conference on artificial intelligence. pp 1237–1242.

  31. Hershey S, Chaudhuri S, Ellis DPW, Gemmeke JF, Jansen A, Moore RC, Plakal M, Platt D, Saurous RA, Seybold B, Slaney M, Weiss RJ, Wilson K (2017) CNN architectures for large-scale audio classification. In: 2017 IEEE international conference on acoustics, speech and signal processing (icassp). IEEE pp 131–135

  32. Lee KB, Cheon S, Kim CO (2017) A convolutional neural network for fault classification and diagnosis in semiconductor manufacturing processes. IEEE Trans Semicond Manuf 30(2):135–142

    Google Scholar 

  33. Ge X, Wang B, Yang X, Pan Y, Liu B, Liu B (2021) Fault detection and diagnosis for reactive distillation based on convolutional neural network. Comput Chem Eng 145:107172

    Google Scholar 

  34. Wu H, Zhao J (2018) Deep convolutional neural network model based chemical process fault diagnosis. Comput Chem Eng 115:185–197

    Google Scholar 

  35. Hsu CY, Liu WC (2021) Multiple time-series convolutional neural network for fault detection and diagnosis and empirical study in semiconductor manufacturing. J Intell Manuf 32:823–836

    Google Scholar 

  36. Kiranyaz S, Ince T, Gabbouj M (2015) Real-time patient-specific ECG classification by 1-D convolutional neural networks. IEEE Trans Biomed Eng 63(3):664–675

    Google Scholar 

  37. Chen S, Yu J, Wang S (2021) One-dimensional convolutional neural network-based active feature extraction for fault detection and diagnosis of industrial processes and its understanding via visualization, ISA transactions

  38. Ng A (2011) Sparse autoencoder. CS294A Lecture notes. 72:1–19

    Google Scholar 

  39. Tao C, Pan H, Li Y, Zou Z (2015) Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification. IEEE Geosci Remote Sens Lett 12(12):2438–2442

    Google Scholar 

  40. Zhang L, Ma W, Zhang D (2016) Stacked sparse autoencoder in PolSAR data classification using local spatial information. IEEE Geosci Remote Sens Lett 13(9):1359–1363

    Google Scholar 

  41. Agarwal S, Roth D (2002) Learning a sparse representation for object detection. European conference on computer vision. Springer, Berlin, pp 113–127

    Google Scholar 

  42. Boureau Y, Cun YL (2008) Sparse feature learning for deep belief networks. Adv Neural Inf Process Syst, pp 1185–1192

  43. Peng Y, Chen X, Ye Q, Jiao J (2014) Fault detection and classification in chemical processes using NMFSC and structural SVMs. Can J Chem Eng 92(6):1016–1023

    Google Scholar 

  44. Liu B, Wang M, Foroosh H, Hassan F, Marshall T, Marianna P (2015) Sparse convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 806–814

  45. Parashar A, Rhu M, Mukkara A, Puglielli A, Venkatesan R, Khailany B, Emer J, Keckler SW, Dally WJ (2017) An accelerator for compressed-sparse convolutional neural networks. In: 2017 ACM/IEEE 44th annual international symposium on computer architecture (ISCA). IEEE. pp 27–40

  46. Poultney C, Chopra S, Cun YL (2007) Efficient learning of sparse representations with an energy-based mode. In: Adv Neural Inf Process Syst. pp 1137–1144.

  47. Hotelling H (1931) The economics of exhaustible resources. J Polit Econ 39(2):137–175

    MATH  Google Scholar 

  48. Jackson JE (1985) Multivariate quality control. Commun Statistics-Theory Methods 14(11):2657–2688

    MATH  Google Scholar 

  49. MacGregor JF, Kourti T (1995) Statistical process control of multivariate processes. Control Eng Pract 3(3):403–414

    Google Scholar 

  50. Montgomery DC (2012) Statistical quality control, Wiley Global Education

  51. Downs JJ, Vogel EF (1993) A plant-wide industrial process control problem. Comput Chem Eng 17(3):245–255

    Google Scholar 

  52. McAvoy TJ (1998) A methodology for screening level control structures in plantwide control systems. Comput Chem Eng 22(11):1543–1552

    Google Scholar 

  53. Yu J (2011) Localized fisher discriminant analysis based complex process monitoring. AIChE J 57(7):1817–1828

    Google Scholar 

  54. Vincent P, Larochelle H, Lajoie I (2010) Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J Mach Learn Res 11:3371–3408

    MathSciNet  MATH  Google Scholar 

  55. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    MathSciNet  MATH  Google Scholar 

  56. Werbos PJ (1990) Backpropagation through time: what it does and how to do it. Proc IEEE 78(10):1550–1560

    Google Scholar 

  57. Drucker H, Burges CJC, Kaufman L, Smola A, Vapnik V (1997) Support vector regression machines. Adv Neural Inf Process Syst, pp 155–161

  58. Chen SM, Yu JB, Wang SJ (2020) One-dimensional Convolutional auto-encoder-based feature learning for fault diagnosis of multivariate processes. J Process Control 87:54–67

    Google Scholar 

  59. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 7132–7141

  60. Iandola F, Moskewicz M, Karayev S, Girshick R, Darrell T, Keutzer K (2014) Implementing efficient convnet descriptor pyramids, arXiv preprint arXiv:1404.1869

  61. Hoyer PO (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5(11):1457–1469

    MathSciNet  MATH  Google Scholar 

  62. Yang C, Hou J (2016) Fed-batch fermentation penicillin process fault diagnosis and detection based on support vector machine. Neurocomputing 190:117–123

    Google Scholar 

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (No. 71771173) and Action Plan for Scientific and Technological Innovation" of Shanghai Science and Technology Commission (No. 21SQBS01402).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Tongji University, Shanghai, 200084, People’s Republic of China

    Jianbo Yu & Chengyi Zhang

  2. School of Economics and Management, Tongji University, Shanghai, 200082, People’s Republic of China

    Shijin Wang

Authors
  1. Jianbo Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengyi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shijin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianbo Yu.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, J., Zhang, C. & Wang, S. Sparse one-dimensional convolutional neural network-based feature learning for fault detection and diagnosis in multivariable manufacturing processes. Neural Comput & Applic 34, 4343–4366 (2022). https://doi.org/10.1007/s00521-021-06575-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06575-6

Keywords

Search

Navigation