A mixture of shallow neural networks for virtual sensing: : Could perform better than deep neural networks

Owing to outstanding representation abilities, deep neural networks (DNNs) have recently been extensively studied and attracted increasing attention in virtual sensing of key industrial variables. Although various learning algorithms have been developed to train the DNNs, the complexities of industrial data, such as scarcity of labeled samples, inevitable measurement noise, and multi-modality, may prevent the DNNs from achieving the best performance. This paper aims to deal with the limitations of the DNNs in an alternative way, by fully exploiting the potentials of shallow NNs. Specifically, a Bayesian model structure of a semi-supervised mixture of multiple single-hidden layer NNs (BSsMSLNN), each of which serves as an expert for a local region, is first proposed. Then, a variational inference- and gradient ascent-based training algorithm is developed, which allows parameter learning and mode identification to be completed in a unified framework. The performance of the BSsMSLNN is evaluated using both synthetic and real industrial cases for virtual sensing, demonstrating the effectiveness and superiority of the proposed schemes.