research-article

Sparse one-dimensional convolutional neural network-based feature learning for fault detection and diagnosis in multivariable manufacturing processes

Authors:

Shijin WangAuthors Info & Claims

Neural Computing and Applications, Volume 34, Issue 6

Pages 4343 - 4366

https://doi.org/10.1007/s00521-021-06575-6

Published: 01 March 2022 Publication History

Abstract

Those fault detection and diagnosis (FDD) models can identify various faulty signals in industrial processes by extracting features from process data with high nonlinearity and correlations. However, the diagnostic performance of those models mainly depends primarily on the validity of the features extracted from the process data. In this paper, a novel deep neural network (DNN) model, sparse one-dimensional convolutional neural network (S1-DCNN), is proposed to learn features from process signals and improve the performance of FDD in industrial processes. S1-DCNN not only extracts discriminative features from complex process signals, but selects effective features based on a sparsity regularization in the convolution layers. Thus, an S1-DCNN-based representation learning method is developed for FDD in industrial processes. Tennessee Eastman process and fed-batch fermentation penicillin process are employed to validate effectiveness of S1-DCNN for FDD. The experimental results illustrate that S1-DCNN extracted and selected effective representative features for process FDD.

References

[1]

Butte VK and Tang LC Multivariate charting techniques: a review and a line-column approach Qual Reliab Eng Int 2010 26 443-451

[2]

Yu J, Zhang C, and Wang S Multichannel one-dimensional convolutional neural network-based feature learning for fault diagnosis of industrial processes Neural Comput Appl 2021 33 3085-3104

[3]

Peres FAP and Fogliatto FS Variable selection methods in multivariate statistical process control: a systematic literature review Comput Ind Eng 2018 115 603-619

[4]

Jiang Q and Huang B Distributed monitoring for large-scale processes based on multivariate statistical analysis and Bayesian method J Process Control 2016 46 75-83

[5]

de Lázaro JMB, Moreno AP, Santiago OL, and da Silva Neto AJ Optimizing kernel methods to reduce dimensionality in fault diagnosis of industrial systems Comput Ind Eng 2015 87 140-149

[6]

Wang B, Yan X, and Jiang Q Independent component analysis model utilizing de-mixing information for improved non-Gaussian process monitoring Comput Ind Eng 2016 94 188-200

[7]

Luo L, Peng X, and Tong C A multigroup framework for fault detection and diagnosis in large-scale multivariate systems J Process Control 2021 100 65-79

[8]

Bounoua W and Bakdi A Fault detection and diagnosis of nonlinear dynamical processes through correlation dimension and fractal analysis based dynamic kernel PCA Chem Eng Sci 2021 229 116099

[9]

Lee JM, Yoo CK, and Lee IB Statistical process monitoring with independent component analysis J Process Control 2004 14 5 467-485

[10]

Jia Q and Zhang Y Quality-related fault detection approach based on dynamic kernel partial least squares Chem Eng Res Des 2016 106 242-252

[11]

Taqvi SAA, Zabiri H, Tufa LD, Uddin F, Fatima SA, Maulud AS (2021) A Review on data-driven learning approaches for fault detection and diagnosis in chemical processes, ChemBioEng Reviews

[12]

Sun RX and Tsung F A kernel-distance-based multivariate control chart using support vector methods Int J Prod Res 2003 41 13 2975-2989

[13]

Hsu CC, Chen MC, and Chen LS Intelligent ICA-SVM fault detector for non-Gaussian multivariate process monitoring Expert Syst Appl 2010 37 4 3264-3273

[14]

Chiang LH, Kotanchek ME, and Kordon AK Fault diagnosis based on Fisher discriminant analysis and support vector machines Comput Chem Eng 2004 28 8 1389-1401

[15]

Yu J Localized Fisher discriminant analysis based complex chemical process monitoring AIChE J 2011 57 7 1817-1828

[16]

Gauri SK and Chakraborty S Recognition of control chart patterns using improved selection of features Comput Ind Eng 2009 56 4 1577-1588

[17]

Huda S, Abdollahian M, Mammadov M, Yearwood J, Ahmed S, and Sultan I A hybrid wrapper-filter approach to detect the source(s) of out-of-control signals in multivariate manufacturing process Eur J Oper Res 2014 237 3 857-870

[18]

Bengio Y, Courville A, and Vincent P Representation learning: a review and new perspectives IEEE Trans Pattern Anal Mach Intell 2013 35 8 1798-1828

[19]

Wang J, Ma Y, Zhang L, Gao RX, and Wu D Deep learning for smart manufacturing: methods and applications J Manuf Syst 2018 48 144-156

[20]

Yu J, Liu H, and Zheng X Two-dimensional joint local and nonlocal discriminant analysis-based 2D image feature extraction for deep learning Neural Comput Appl 2020 32 6009-6024

[21]

Zhang Z, Jiang T, Li S, and Yang Y Automated feature learning for nonlinear process monitoring-An approach using stacked denoising autoencoder and k-nearest neighbor rule J Process Control 2018 64 49-61

[22]

Ren H, Chai Y, Qu JF, Zhang K, Tang Q (2018) An intelligent fault detection method based on sparse auto-encoder for industrial process systems: a case study on tennessee eastman process chemical system. In: International conference on intelligent human-machine systems and cybernetics, pp 190–193

[23]

Chen S, Yu J, Wang S (2020) Monitoring of complex profiles based on deep stacked denoising autoencoders. Comput Ind Eng, p 106402

[24]

Yu J and Zheng X Stacked convolutional sparse denoising auto-encoder for identification of defect patterns in semiconductor wafer map Comput Ind 2019 109 121-133

[25]

Liu X, Jianbo Yu, and Ye L Residual attention convolutional auto-encoder for feature learning and fault detection in nonlinear industrial processes Neural Comput Appl 2021 33 8 3085-3104

[26]

Yu JB and Yan X Layer-by-Layer enhancement strategy of favorable features of the deep belief network for industrial process monitoring Ind Eng Chem Res 2018 57 45 15479-15490

[27]

Wang Y, Pan Z, Yuan X, Yang C, and Gui W A novel deep learning based fault diagnosis approach for chemical process with extended deep belief network ISA Trans 2019

[28]

Hu G, Li H, Xia Y, and Luo L A deep Boltzmann machine and multi-grained scanning forest ensemble collaborative method and its application to industrial fault diagnosis Comput Ind 2018 100 287-296

[29]

LeCun Y, Bengio Y (1995) Convolutional networks for images, speech, and time series, Handb Brain Theory Neural Netw, 3361(10): 1995

[30]

Ciresan DC, Meier U, Masci J, Gambardella LM, Schmidhuber J (2011) Flexible, high performance convolutional neural networks for image classification. In: Twenty-second international joint conference on artificial intelligence. pp 1237–1242.

[31]

Hershey S, Chaudhuri S, Ellis DPW, Gemmeke JF, Jansen A, Moore RC, Plakal M, Platt D, Saurous RA, Seybold B, Slaney M, Weiss RJ, Wilson K (2017) CNN architectures for large-scale audio classification. In: 2017 IEEE international conference on acoustics, speech and signal processing (icassp). IEEE pp 131–135

[32]

Lee KB, Cheon S, and Kim CO A convolutional neural network for fault classification and diagnosis in semiconductor manufacturing processes IEEE Trans Semicond Manuf 2017 30 2 135-142

[33]

Ge X, Wang B, Yang X, Pan Y, Liu B, and Liu B Fault detection and diagnosis for reactive distillation based on convolutional neural network Comput Chem Eng 2021 145 107172

[34]

Wu H and Zhao J Deep convolutional neural network model based chemical process fault diagnosis Comput Chem Eng 2018 115 185-197

[35]

Hsu CY and Liu WC Multiple time-series convolutional neural network for fault detection and diagnosis and empirical study in semiconductor manufacturing J Intell Manuf 2021 32 823-836

[36]

Kiranyaz S, Ince T, and Gabbouj M Real-time patient-specific ECG classification by 1-D convolutional neural networks IEEE Trans Biomed Eng 2015 63 3 664-675

[37]

Chen S, Yu J, Wang S (2021) One-dimensional convolutional neural network-based active feature extraction for fault detection and diagnosis of industrial processes and its understanding via visualization, ISA transactions

[38]

Ng A Sparse autoencoder CS294A Lecture notes. 2011 72 1-19

[39]

Tao C, Pan H, Li Y, and Zou Z Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification IEEE Geosci Remote Sens Lett 2015 12 12 2438-2442

[40]

Zhang L, Ma W, and Zhang D Stacked sparse autoencoder in PolSAR data classification using local spatial information IEEE Geosci Remote Sens Lett 2016 13 9 1359-1363

[41]

Agarwal S and Roth D Learning a sparse representation for object detection European conference on computer vision 2002 Berlin Springer 113-127

[42]

Boureau Y, Cun YL (2008) Sparse feature learning for deep belief networks. Adv Neural Inf Process Syst, pp 1185–1192

[43]

Peng Y, Chen X, Ye Q, and Jiao J Fault detection and classification in chemical processes using NMFSC and structural SVMs Can J Chem Eng 2014 92 6 1016-1023

[44]

Liu B, Wang M, Foroosh H, Hassan F, Marshall T, Marianna P (2015) Sparse convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 806–814

[45]

Parashar A, Rhu M, Mukkara A, Puglielli A, Venkatesan R, Khailany B, Emer J, Keckler SW, Dally WJ (2017) An accelerator for compressed-sparse convolutional neural networks. In: 2017 ACM/IEEE 44th annual international symposium on computer architecture (ISCA). IEEE. pp 27–40

[46]

Poultney C, Chopra S, Cun YL (2007) Efficient learning of sparse representations with an energy-based mode. In: Adv Neural Inf Process Syst. pp 1137–1144.

[47]

Hotelling H The economics of exhaustible resources J Polit Econ 1931 39 2 137-175

[48]

Jackson JE Multivariate quality control Commun Statistics-Theory Methods 1985 14 11 2657-2688

[49]

MacGregor JF and Kourti T Statistical process control of multivariate processes Control Eng Pract 1995 3 3 403-414

[50]

Montgomery DC (2012) Statistical quality control, Wiley Global Education

[51]

Downs JJ and Vogel EF A plant-wide industrial process control problem Comput Chem Eng 1993 17 3 245-255

[52]

McAvoy TJ A methodology for screening level control structures in plantwide control systems Comput Chem Eng 1998 22 11 1543-1552

[53]

Yu J Localized fisher discriminant analysis based complex process monitoring AIChE J 2011 57 7 1817-1828

[54]

Vincent P, Larochelle H, and Lajoie I Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion J Mach Learn Res 2010 11 3371-3408

[55]

Hinton GE and Salakhutdinov RR Reducing the dimensionality of data with neural networks Science 2006 313 5786 504-507

[56]

Werbos PJ Backpropagation through time: what it does and how to do it Proc IEEE 1990 78 10 1550-1560

[57]

Drucker H, Burges CJC, Kaufman L, Smola A, Vapnik V (1997) Support vector regression machines. Adv Neural Inf Process Syst, pp 155–161

[58]

Chen SM, Yu JB, and Wang SJ One-dimensional Convolutional auto-encoder-based feature learning for fault diagnosis of multivariate processes J Process Control 2020 87 54-67

[59]

Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 7132–7141

[60]

Iandola F, Moskewicz M, Karayev S, Girshick R, Darrell T, Keutzer K (2014) Implementing efficient convnet descriptor pyramids, arXiv preprint arXiv:1404.1869

[61]

Hoyer PO Non-negative matrix factorization with sparseness constraints J Mach Learn Res 2004 5 11 1457-1469

[62]

Yang C and Hou J Fed-batch fermentation penicillin process fault diagnosis and detection based on support vector machine Neurocomputing 2016 190 117-123

Cited By

Qi LRen YFang YZhou J(2023)Two-view LSTM variational auto-encoder for fault detection and diagnosis in multivariable manufacturing processesNeural Computing and Applications10.1007/s00521-023-08949-435:29(22007-22026)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1007/s00521-023-08949-4

Recommendations

Multichannel one-dimensional convolutional neural network-based feature learning for fault diagnosis of industrial processes
Abstract
In industrial processes, the noise and high dimension of process signals usually affect the performance of those methods in fault detection and diagnosis. A predominant property of a fault diagnosis model is to extract effective features from ...
Intelligent fault diagnosis of rotating machinery based on one-dimensional convolutional neural network
Highlights
- A one-dimensional convolutional neural network model is proposed to Diagnostic fault with the original vibration signal.
Abstract
Fault diagnosis of rotating machinery plays a significant role in the reliability and safety of modern industrial systems. The traditional fault diagnosis methods usually need manually extracting the features from raw sensor data ...
Multi-level features fusion network-based feature learning for machinery fault diagnosis
Abstract
Bearings are one of the most critical components in rotating machinery. Since the failures of bearings will cause unexpected machine damages, it is significant to timely and accurately recognize the defects in bearings. However, due to ...
Highlights
- A new DNN model, MLFNet is proposed for feature learning from vibration signal.

Comments

Information & Contributors

Information

Published In

cover image Neural Computing and Applications

Neural Computing and Applications Volume 34, Issue 6

Mar 2022

907 pages

ISSN:0941-0643

EISSN:1433-3058

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2021.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 March 2022

Accepted: 21 September 2021

Received: 12 April 2021

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Qi LRen YFang YZhou J(2023)Two-view LSTM variational auto-encoder for fault detection and diagnosis in multivariable manufacturing processesNeural Computing and Applications10.1007/s00521-023-08949-435:29(22007-22026)Online publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1007/s00521-023-08949-4

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents