A Spatio-Temporal Deep Learning Network for the Short-Term Energy Consumption Prediction of Multiple Nodes in Manufacturing Systems

Abstract

Short-term energy prediction plays an important role in green manufacturing in the industrial internet environment and has become the basis of energy wastage identification, energy-saving plans and energy-saving control. However, the short-term energy prediction of multiple nodes in manufacturing systems is still a challenging issue owing to the fuzzy material flow (spatial relationship) and the dynamic production rhythm (temporal relationship). To obtain the complex spatial and temporal relationships, a spatio-temporal deep learning network (STDLN) method is presented for the short-term energy consumption prediction of multiple nodes in manufacturing systems. The method combines a graph convolutional network (GCN) and a gated recurrent unit (GRU) and predicts the future energy consumption of multiple nodes based on prior knowledge of material flow and the historical energy consumption time series. The GCN is aimed at capturing spatial relationships, with the material flow represented by a topology model, and the GRU is aimed at capturing dynamic rhythm from the energy consumption time series. To evaluate the method presented, several experiments were performed on the power consumption dataset of an aluminum profile plant. The results show that the method presented can predict the energy consumption of multiple nodes simultaneously and achieve a higher performance than methods based on the GRU, GCN, support vector regression (SVR), etc.

1. Introduction

In the context of global warming and fossil fuel depletion, green manufacturing has drawn more and more attention. Energy saving in manufacturing processes is one of the core goals of green manufacturing, and it has been well supported by industrial internets, which enable real-time collection of energy consumption from smart energy meters. Short-term (i.e., hourly or minute-by-minute) energy prediction plays an important role in green manufacturing and has become the basis of energy wastage identification, energy-saving plans and energy-saving control [1,2,3,4]. However, short-term energy prediction in manufacturing systems is still a challenging issue owing to its complex spatial and temporal relationships between production nodes.

The spatial relationship means the topological structure composed of the material flow. From the perspective of machines, the production status at the upstream production node is transferred to the downstream production node [5]. From the perspective of management, the production node is a hierarchical concept composed of a plant, workshop, line, machine group or individual machine [2,6,7], and the energy consumption at the upper production node is transferred to the lower production node. Moreover, these transfers are very fuzzy in many flexible manufacturing systems [3,8]. In general, the output from one upstream/upper production node may randomly be transferred to a few downstream/lower nodes, and one faulty production node must be frequently replaced by another parallel node.

The temporal relationship means the dynamic production rhythm that reflects production periodicity and trend. The energy consumption time series at different granularities can reflect different production laws. The minute-by-minute series of machine nodes generally reflect process law, and the hourly series of workshop nodes generally reflect worker law [9]. These production laws are easily affected by machine failure, lack of material, changeovers, etc.

According to input data, the energy consumption prediction methods for manufacturing systems can be categorized into the parametric method and the sequential method. The former generally inputs impact parameters such as yield parameters, technological parameters, weather parameters, etc., and tries to capture the relationship between input and output through a building mechanism model or a machine learning model [1,10,11]. Generally, the relationship between parameters and energy consumption can be regarded as a kind of spatial relationship between the machine and its surroundings. The authors [1,12] presented a parametric method for energy consumption prediction of tire vulcanization, in which a few technological parameters were inputted and which was then used to detect heat loss anomalies during the process. However, we find that such methods generally require rich prior technical knowledge and complex multi-data fusion, and hence they are hard to implement in many real cases. Moreover, their prediction precision may be unsatisfactory owing to ignoring of the temporal relationship.

The sequential methods depend on the historical time series of energy consumption, and they are still prevalent owing to high flexibility and availability. The classical sequential methods include the autoregressive integrated moving average (ARIMA) model [13], the XGBoost model [14], the Kalman filtering model [15], the support vector regression (SVR) model [16,17,18] and the back-propagation (BP) neural network model [19]. However, the above methods cannot fit well with capturing complex features from big data. In recent years, deep learning networks have become very prevalent owing to their strong feature extraction capabilities from big data.

A recurrent neural network (RNN) [20] is a typical deep learning network for feature capturing from time series. In this network, hidden unit patterns are fed back to themselves and previous states are memorized. However, it is not able to keep long-term memory information since the gradients tend to either vanish or explode. To this end, two improved RNNs, long short-term memory (LSTM) [21] and gated recurrent unit (GRU) [22,23], were proposed. In LSTM, three gates, i.e., input gate, output gate and forget gate, are added to serve as three switches. They determine what is to be memorized or forgotten and facilitate the keeping, utilizing, or destroying of a previous state in an appropriate time. The GRU can be regarded as an updated version of LSTM with a simpler structure. It merges the input gate and the forget gate into the update gate while the memory units and hidden units are also combined, and modulates the information flow inside the units without separated memory cells [24].

In recent years, LSTM, GRU and their improvements have been widely applied in energy consumption prediction of time series and other industrial data predictions. Some methods focus on the prediction of a single node considering only the temporal relationship. Wang et al. [25] implemented a day-ahead photovoltaic power prediction through integrating LSTM with time correlation principles. He and Tsang [26] achieved short-term load prediction of colleges and universities through integrating LSTM with the periodic pattern decomposition of time series. Similar methods based on LSTM and pattern decomposition can be seen in the energy prediction of solar-assisted water heating systems [27], in regional natural gas consumption prediction [28], in the power prediction of universities [26], in the heating energy prediction of non-residential buildings [29], etc. Wang, Yan, Li, Gao and Zhao [24] integrated local feature knowledge into a deep heterogeneous GRU model and implemented tool wear prediction in manufacturing. The above methods can achieve high prediction precision for time series through extracting local features and then reducing local noise, but they did not consider spatial relationships between multiple nodes and have not been applied in the simultaneous prediction of multiple nodes. For a manufacturing system with hundreds or thousands of nodes, it is too time-consuming to build a prediction model for each node.

Some methods consider the spatial relationship in the time-series prediction of multiple nodes. Liu et al. [30] proposed a dual-stage two-phase attention-based RNN for long-term and multivariate time-series prediction, in which spatial correlations are captured through adding attention to the node dimensions. The method was employed to predict the consumption energy of multiple appliances in a low-energy building. A few methods integrated convolutional neural network (CNN) with GRU (or LSTM) to predict the energy consumption time series of multiple nodes, and they have been applied in regional integrated energy systems [31,32,33,34] and building energy management systems [35]. In these methods, the spatial relationship is extracted through a convolutional operation on high-dimensional time series. However, the CNN is commonly used for Euclidean data such as images, regular grids and so on, and cannot work well in the context of a manufacturing system with a complex topological structure. Thus, the CNN is inapposite for describing spatial relationships. On the other hand, an attention mechanism is commonly used for sequence data with clear contexts such as nature language processes and cannot work well in the context of manufacturing systems without describing topological structure.

Therefore, short-term energy consumption prediction of multiple nodes in manufacturing systems is still a challenging issue. The main challenge of the issue lies in the extraction of spatial relationships and in spatio-temporal collaborative learning. Graph convolutional networks (GCNs) [36] inspired the solution for spatial-relationship learning issues. The GCN presented a scalable approach for learning on graph-structured data based on an efficient variant of convolutional neural networks which operate directly on graphs and can be used to capture the structural features of a graph network. Initially, the GCN was designed for semi-supervised classification of graph data. Currently, it is being extended to time-series prediction through integration with RNN, LSTM and GRU [37,38], and has presented spatio-temporal graph convolutional networks for forecasting traffic flow. In the two methods, the GCN is combined with the GRU, the GCN is used to learn complex topological structures to capture spatial dependence and the GRU is used to learn dynamic changes of traffic data to capture temporal dependence. The framework combining the GCN with the GRU has good transferability to the short-term energy consumption prediction of multiple nodes. However, different from traffic systems with visible topological structure, manufacturing systems have fuzzy topological structures and consume various forms of energy, and hence both the topological model and the learning algorithm need to be reconstructed for the latter.

Based on the above background, this paper focuses on short-term energy consumption prediction of multiple nodes in manufacturing systems. Three hypotheses are under consideration. First, a manufacturing system is a hierarchical structure composed of a workshop, line, machine group and individual machine. Second, material flow in the manufacturing system is a mixture mode consisting of parallel machines and flow lines. Third, data on energy consumption of production nodes are collected minute-by-minute, and prediction depends only on the collected time series. Thereafter, a spatio-temporal deep learning network (STDLN) combining the GCN with the GRU is presented. The main contributions of this paper are as follows:

(1) A topology modeling method is presented to make clear the topological structure of the manufacturing system. The method constructs the explicit spatial relationships between the production nodes through hierarchically abstracting parallel machines and flow lines and finally describes it as an adjacency matrix.

(2) A deep learning network combining the GCN with the GRU is presented. The GCN is used to capture the implicit spatial relationship between production nodes through learning based on the adjacency matrix and historical time series. The GRU is used to capture the temporal relationship through learning based on the time series from the GCN.

(3) The case of an aluminum profile plant is provided to evaluate the model presented. A total of 140 nodes were chosen and the power consumption dataset was extracted from the energy management system of the plant studied. Several experiments were performed, and the results show that the method presented can predict the energy consumption of multiple nodes simultaneously, and can achieve a higher performance than models based on the GRU, the GCN, support vector regression (SVR), etc.

The rest of this paper is organized as follows. The methods are presented in Section 2. The experiments and results are described in Section 3. The interpretations of the experimental results are discussed in Section 4, and the conclusions drawn are presented in Section 5.

2. Methods

2.1. Problem Definition

Given a manufacturing system with prior knowledge of the technology process and minute-by-minute energy consumption time series of production nodes, the goal of this paper is to predict the energy consumption of multiple nodes in a certain time period. The prediction is short-term (minute-by-minute or hourly) and mainly used to support energy wastage identification, energy-saving plans and energy-saving control.

Definition 1.

Production Nodes V. A production node means a workshop, a line, a machine group or an individual machine, and it is associated with an energy consumption meter. V = {v₁, v₂, …, v_N}, V is the set of production nodes in the manufacturing system, N is the size of V and v_i is the ith production node.

Definition 2.

Edges between Production Nodes E. E = {e_ij|v_i, v_j ∈ V, i ≠ j}, where e_ij is the edge between v_i and v_j, which only exists if there is direct material flow between v_i and v_j.

Definition 3.

Energy Consumption Time Series X^N×P. X = {x_it|v_i ∈ V, t = 0, 1, …, P}, where x_it is the energy consumption value of v_i at tth time, P is the length of the time series, X ∈ R^N×P. For simplicity, it is assumed that only one type of energy needs to be predicted for one production node.

Definition 4.

Material Flow Network G. An unweighted graph G = (V, E) is used to describe the topological structure of the material flow network of the manufacturing system. G can be represented by an adjacency matrix A ∈ R^N×N which contains only elements of 0 and 1: the element in the ith row and the jth is 1 if e_ij exists, and 0 otherwise.

Therefore, the short-term energy consumption prediction of multiple nodes in the manufacturing system can be described as Equation (1):

$$[X_{t+1},X_{t+2},\dots ,X_{t+T}]=f(A;(X_{t-n},X_{t-n+1},\dots ,X_t))$$

(1)

where X_t ∈ R^N is the energy consumption values of V at t-th time, n is the length of the historical time series and T is the length of the time series that needs to be predicted.

2.2. Framework of Method

Based on the definitions above, the framework of the STDLN method is designed and shown in Figure 1. The method consists of three tasks: topology modeling, spatial relationship learning network modeling and temporal relationship learning network modeling. The topology modeling is to construct material flow network G based on prior knowledge of the technology process, and the spatial relationship learning network modeling and temporal relationship learning network modeling are to construct a prediction algorithm based on G and X. It is assumed that the data-collecting task for collecting energy consumption data from the energy meters was implemented in the industrial internet. The three tasks are described in the following subsections, respectively.

2.3. Topology Modeling

In general, material flow between machines is intricate and needs to be made clear through hierarchical abstracting. In production management, the manufacturing systems are categorized into flow shop system and parallel machine system based on the material flow dependency between the machines. Correspondingly, the material flow in a manufacturing system is categorized into liner flow and distributing flow in the topology modeling method. Liner flow means that the materials flow through each production node in sequence. Distributing flow means that the materials are distributed from one upper production node to multiple lower production nodes.

In this section, a 2-level abstracting method is presented to construct the topology model of a certain plant. The method works as follows:

• Identify visible production nodes including workshops and machines and build the distributing flow between the workshop and the machine based on ownership.
• Abstract production nodes at the workshop level.

  (a)

  Merge workshops with a similar process into one process node.

  (b)

  Build the distributing flow between the process node and the corresponding lower workshops.

  (c)

  Build the liner flow between the workshops (or process nodes if there are merged workshops).

• Abstract production nodes at the machine level.

  (a)

  Merge parallel machines into one process node and merge the flow machines into one process node.

  (b)

  Build the distributing flow between the process node and the corresponding downstream machines.

  (c)

  Build the liner flow between the machines (or process nodes if there are merged machines).

  (d)

  Build the liner flow between the flow machines.

• Merge redundant nodes. If one upper node owns only one lower node, the two nodes are redundant and need to be merged as one node.

After 2-level abstracting, the intricate material flow in a certain manufacturing system is decomposed into simple distributing flows and liner flows and finally converted into a composite flow.

An example of the modeling production topology is shown in Figure 2. A plant consists of three workshops with different processes, and Workshop 2 owns a production line with three machines. The three workshops are modeled by Node 1, Node 2 and Node 3. The three machines are modeled by Node 2.1, Node 2.2 and Node 2.3. The distributing flow and liner flow are shown on the left, and the composite flow is shown on the right. The composite flow is just the material flow network G and it can be represented by an adjacency matrix A∈R^N×N.

2.4. Spatial Relationship Learning Network Modeling

The GCN has been successfully applied in traffic flow prediction [37,38] and graph node classification [36], and has shown a strong capacity for capturing spatial features. Thus, the GCN was chosen to construct the spatial relationship learning network, and the schematic depiction of the GCN for energy consumption prediction is shown in Figure 3. The input of a GCN cell includes an adjacency matrix A (∈R^N×N) and a periodical energy consumption vector X_t (∈R^N), and the output is a vector X’_t (∈R^N) that represents the implicit spatial relationship between production nodes. The GCN model constructs a filter in the Fourier domain, and the filter acts on the nodes of the graph and its first-order neighborhood to capture spatial features between the nodes. The GCN can be built by stacking multiple convolutional layers [36,38]. As shown in Figure 3, the GCN model can obtain the implicit spatial relationship between Node 2 and its first neighbor nodes, and it represents the energy consumption on all nodes. In fact, the adjacency matrix A represents the explicit spatial relationship between production nodes based on embedding prior knowledge. The GCN can convert X_t to X’_t and represents the implicit spatial relationship between production nodes based on machine learning.

In this paper, a 2-layer GCN model is used to capture spatial relationships. The forward algorithm from A and X_t to X’_t can be described as follows [36]:

$$\tilde{A}=A+I$$

(2)

where $\tilde{A}$ is a matrix with a self-connection structure, and I is an identity matrix with the same size as A.

$$\tilde{D}={\displaystyle {\sum }_{\mathrm{j}}{\tilde{A}}_{ij}}$$

(3)

where $\tilde{D}$ is the degree matrix of $\tilde{A}$.

$$\widehat{A}={\tilde{D}}^{-\frac{1}{2}}\tilde{A}{D}^{-\frac{1}{2}}$$

(4)

$${X^{\prime }}_t=\mathrm{f}(X_t,A)=\mathsf{\sigma }(\widehat{A}\mathrm{Relu}(\widehat{A}{X}_t{W}_0)W_1)$$

(5)

where $\mathsf{\sigma }(.)$ and $\mathrm{Relu}(.)$ represent two activation functions, and $W_0$ and $W_1$ represent the weight matrix in the first and second layer, respectively.

It is not necessary to define loss function in the GCN model since the ${X^{\prime }}_t$ is not the final output.

2.5. Temporal Relationship Learning Network Modeling

As an improved RNN, the GRU has a strong capacity for capturing temporal relationships from time series. Moreover, the GRU has simpler structure and more efficient training process than LSTM. Thus, the GRU was chosen to construct the temporal relationship learning network. The schematic depiction of the GRU for energy consumption prediction is shown in Figure 4. In the middle-GRU cell, the input X’_t is the output of a GCN cell, and the output h_t is the prediction vector corresponding to observing vector X_t+T. u_t and r_t are the update gate and reset gate, respectively.

The forward algorithm from X’_t to h_t can be described as follows [22,23]:

$$u_t=\mathsf{\sigma }(W_{\mathrm{u}}[{X^{\prime }}_t,h_{t-1}]+b_{\mathrm{u}})$$

(6)

$$r_t=\mathsf{\sigma }(W_{\mathrm{r}}[{X^{\prime }}_t,h_{t-1}]+b_{\mathrm{r}})$$

(7)

$$c_t=\tanh (W_{\mathrm{c}}[{X^{\prime }}_t,(r_t\ast h_{t-1})]+b_{\mathrm{c}})$$

(8)

$$h_t=u_t\ast h_{t-1}+(1-u_t)\ast c_t$$

(9)

where W_u, W_r and W_c represent the weights in the training process, and b_u, b_r and b_c represent the deviations in the training process.

2.6. Loss Function

In the training process, the goal is to minimize the error between the observing of the energy consumption of the nodes and the predicted value. Y_t and ${\widehat{Y}}_t$ are used to denote the observing vector and the predicted vector of the nodes at time t. The loss function of the STDLN is defined as Equation (10).

$$loss=\sqrt{\frac{1}{NT}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^T{(y_{it}-{\widehat{y}}_{it})}^2}}}+\lambda L_{\mathrm{reg}}$$

(10)

where $y_{it}\in Y_t$ represents the observing value of the ith node, ${\widehat{y}}_{it}\in {\widehat{Y}}_t$ represents the predicted value of the ith node, N represents the number of nodes, T is the length of the time series that needs to be predicted, L_reg is the L2 regularization term that helps to avoid an over-fitting problem and λ is a hyperparameter.

3. Experiments and Results

3.1. Application Case

A large-scale aluminum profile plant located in Guangdong, China, was chosen as experimental object. The plant turns aluminum ingots into aluminum profiles applied in building or industrial products, and consumes a large amount of energy through power, gas, diesel oil, water, compressed air, etc. The authors anticipated the development of its energy management system, in which over 1000 energy meters were installed and the data were collected minute-by-minute. At the same time, the authors learned about its technology and production management and were authorized to access an energy consumption dataset.

3.1.1. Technology Topology Model

According to the two-level abstracting method presented in Section 2.3, the process model of the studied plant can be described as in Figure 5. At workshop level, the processes can be abstracted as a flow line consisting of melting, extruding and surface treating. At the machine level in the melting workshop, the processes can be abstracted as a flow line consisting of a melting surface and a homogenizer. At the machine level in the extruding workshop, the processes can be abstracted as a flow line consisting of a heating surface, extruder, straightener and aging furnace. The surface treating workshop consists of four parallel workshops, namely, the oxidation workshop, the electrophoretic workshop, the painting workshop and the fluorocarbon workshop. The abbreviated names of the workshops and the machines are written in parentheses after the full names in the figure.

The partial topology model of the studied plant is shown in Figure 6. The blue nodes are visible and the yellow are abstract. The extruding workshop (XW) node is the abstract of three parallel XWs (XW1, XW2 and XW3). Each XW node includes a flow line consisting of HF, EX, ST and AF, and an EX node includes parallel visible machines. The figure depicts a part of the topology model, and some nodes are not shown; e.g., the details of XW2 and XW3 are omitted.

3.1.2. Time Series Datasets

To evaluate the model presented, a dataset was extracted from the energy management system of the studied plant. The dataset consists of power consumption of 140 nodes, and the time span is one month. The energy consumption of each node was collected minute-by-minute and aggregated every 5 min, 15 min, 30 min and 60 min. Each aggregated dataset can be regarded as an independent dataset, and the sets were named the 5 min set, the 15 min set, the 30 min set and the 60 min set in the ensuing experiments.

In the experiments, the input data was normalized to the interval (0, 1). For every dataset, 80% of the data was used as a training set and the remaining 20% was used as a testing set. The energy consumption in the next three periods was the predicted object.

3.2. Evaluation Metrics

To evaluate the performance of the STDLN method, four metrics were used to measure prediction performance. They are defined as follows:

(1) Root mean squared error (RMSE):

$$RSME=\sqrt{\frac{1}{NT}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^T{(y_{it}-{\widehat{y}}_{it})}^2}}}$$

(11)

(2) Mean absolute error (MAE):

$$MAE=\frac{1}{NT}{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^T\left|y_{it}-{\widehat{y}}_{it}\right|}}$$

(12)

(3) Accuracy (Acc):

$$Acc=1-\frac{\sqrt{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^T{(y_{it}-{\widehat{y}}_{it})}^2}}}}{\sqrt{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^T{y}_{it}{}^2}}}}$$

(13)

(4) Coefficient of determination (R²):

$$R^2=1-\frac{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^T{(y_{it}-{\widehat{y}}_{it})}^2}}}{{\displaystyle \sum _{i=1}^N{\displaystyle \sum _{t=1}^T{(y_{it}-{\overline{y}}_{it})}^2}}}$$

(14)

where the symbols have the same meanings as in Equation (10).

Specifically, the RMSE and the MAE were used to measure the prediction error; the smaller the value, the better the prediction effect. The Acc was used to detect the prediction precision; the lager the value, the better the prediction effect. The R² was used to calculate the correlation coefficient, which measures the ability of the prediction result to represent the actual data; once more, the larger the value, the better the prediction effect [38].

3.3. Baseline Methods

The following baseline methods were used to compare the performance with the STDLN:

(1) Autoregressive integrated moving average (ARIMA) [13]: a well-known method for time-series prediction, which is an improvement on the autoregressive moving average model.

(2) Support vector regression (SVR) [18]: the kernel method based on support vectors has performed well in time-series prediction; the SVR with a radial basis function kernel was used as a the baseline method.

(3) XGBoost [14]: one of the most popular boosting tree algorithms for gradient boosting machines, which has been widely applied in prediction.

(4) GCN: Only the spatial relationship was considered. See Section 2.4 for details.

(5) GRU: Only the temporal relationship was considered. See Section 2.5 for details.

(6) STDLN_F: the STDLN with a fully connected adjacency matrix. It is assumed that there exists a spatial relation between each pair of production nodes. With the fully connected adjacency matrix, the graph convolution is similar to the convolution in CNN. Hence, the effect of the STDLN_F is also similar to combining the CNN with the GRU.

3.4. Implementation and Parameter Design

The STDLN method presented was developed using the Python language based on TensorFlow 2.0 and reusing some code provided by [38] on github.com. The baseline methods GCN, GRU and STDLN_F were implemented through reconfiguring STDLN code, and the baseline methods ARIMA, SVR and XGBoost were developed based on TensorFlow 2.0. The experiments were performed on a super-deep learning server with NVIDIA Quadro RTX8000 48G GDDR6 GPU *4, Intel Xeon Gold 5218R CPU *2 and DDR4 32G RAM *8.

Hyperparameters such as input length, learning rate, batch size, training time and number of hidden units needed to be set in the STDLN. Empirically, input length was set to 12, learning rate to 0.001, batch size to 32 and training time to 1000. Additionally, the Adam optimizer was used in the training process.

The number of hidden units was set through dichotomy experiments. A 15 min set was chosen to evaluate the performance, and the steps are described as follows:

• Three empirical numbers 0 < n₁ < n₂ < n₃ were initially chosen to perform the experiments.
• Experiments with n₁, n₂ and n₃ were performed and their performances were evaluated.
• The three numbers were updated in the new range according to the performance of experiments. If the best was n₁, the new range was (0, n₁); if the best was n₃, it was (n₃, +∞); if the best was n₂ and the second best was n₁, it was (n₁, n₂), otherwise it was (n₂, n₃).
• If the new range is small enough, return to the best number; otherwise, repeat Step 2 and Step 3.

The dichotomy experiments were performed and the performance comparison of the number of hidden units is shown in

Table 1. Performance comparison of the number of hidden units.

Number of Hidden Units	RMSE	MAE	Acc Ratio (-)	R2 Ratio (-)
16	10.160	5.269	0.939	0.996
64	8.606	4.801	0.945	0.997
80 *	6.301	4.320	0.953	0.998
90	9.560	5.108	0.940	0.996
100	8.384	4.263	0.946	0.997
128	8.741	4.261	0.944	0.997

* The best parameter was 80.

. The three numbers 16, 64 and 128 were chosen for Step 1, and the new range was 64–128 based on the previous performances. The three numbers 80, 90 and 100 were chosen in the second repetition; the number 80 was the best number of hidden units and was used in the ensuing experiments.

3.5. Experimental Results

The prediction performances of the STDLN method and the baseline methods on the datasets are shown in

Table 2. The prediction performances of the STDLN model and baseline methods on the datasets.

Dataset	Metrics	XGT	ARIMA	SVR	GRU	GCN	STDLN *	STDLN_F
5 min	RMSE	12.496	14.804	7.942	11.7691	7.877	6.412	17.793
	MAE	10.199	11.944	4.381	8.282	12.232	3.452	18.571
	Acc ratio (-)	0.694	0.652	0.634	0.910	0.610	0.961	0.173
	R2 ratio (-)	0.172	0.421	0.642	0.821	0.823	0.998	0.392
15 min	RMSE	56.475	76.31	46.147	35.67	33.651	31.840	134.345
	MAE	80.761	69.46	60.792	56.47	27.290	16.522	146.694
	Acc ratio (-)	0.478	0.314	0.516	0.671	0.742	0.945	0.172
	R2 ratio (-)	0.247	0.430	0.317	0.426	0.783	0.996	0.137
30 min	RMSE	68.537	51.483	28.383	51.773	447.318	57.734	106.108
	MAE	56.428	37.522	21.306	16.953	132.971	29.244	295.101
	Acc ratio (-)	0.412	0.391	0.637	0.955	0.611	0.957	0.155
	R2 ratio (-)	0.225	0.407	0.154	0.997	0.831	0.999	0.112
60 min	RMSE	73.512	84.376	66.146	99.561	894.196	132.485	1173.267
	MAE	84.374	77.153	44.046	31.646	256.101	73.247	517.062
	Acc ratio (-)	0.284	0.377	0.476	0.957	0.611	0.963	0.156
	R2 ratio (-)	0.117	0.305	0.331	0.997	0.831	0.998	0.105

* The STDLN method obtained the best prediction performance under all evaluation metrics and all datasets.

. It can be seen that the STDLN method obtained the best prediction performance under all evaluation metrics and all datasets, and the result proves the effectiveness of the STDLN method for short-term energy consumption prediction of multiple nodes in the case studied. The results were expanded according to algorithm, period and production node level in the following subsections.

3.5.1. Performance Comparison between the STDLN Method and the Traditional Methods

The XGT, ARIMA and SVR are classical algorithms which emphasize the importance of modeling the relation between input attributes and output attributes. Observing Table 1, it can be seen that the STDLN method obtained significantly better performance than the traditional methods above. For example, compared with the XGT, ARIMA and SVR for the prediction of the 5 min set, the RMSEs of the STDLN method were reduced by 48.69%, 56.69% and 19.26%, respectively; the MAEs were reduced by 66.15%, 71.10% and 21.21%, respectively; the Acc values were higher, at 76.70%, 30.90% and 32.70%; and the R² values were higher, at 82.60%, 57.70% and 35.60%. In fact, the Acc values of less than 70% shows that the performances of the XGT, ARIMA and SVR were rather unsatisfactory. The main reason for this result is that these traditional methods lack the ability to process complex high-dimensional time series. In addition, the Acc of the XGT for the 15 min set was only 0.194, and the result shows that it is hard to find a regression mode from over 100 dimension data points through assembling multiple simple prediction models.

3.5.2. Performance Comparison between the STDLN method and the Deep Learning Methods

The GRU is a novel deep learning algorithm which emphasizes the importance of modeling temporal features. The GCN is a novel deep learning algorithm which emphasizes the importance of modeling spatial features. The STDLN is a combination of the two algorithms above with material flow topology. The STDLN_F is a special form of STDLN with full connection topology in which material flow is ignored. To verify whether the STDLN method has the ability to capture spatial and temporal features from the energy consumption time series, the performance comparison between the deep learning networks for the 5 min set and the 15 min set are shown in Figure 7.

Observing Figure 7, it can be seen that the STDLN method obtained better performance than the GRU, the GCN and the STDLN_F methods, and the result shows that the STDLN method can improve prediction performance through capturing both spatial features and temporal features from both time series and production nodes. The GRU performed satisfactorily for the 5 min set and obtained an Acc of 0.910, but that of the STDLN method was still better by 0.05. The GCN performed satisfactorily for the 15 min set and obtained a lower RSME, but that of the STDLN was still better by 5.3%.

As shown in Table 1, the prediction performance of the GRU was generally better than that of the GCN, and this shows that the impact of temporal features is more significant than that of spatial features for the energy consumption of multiple production nodes. However, the stable Acc of the GCN under all datasets shows that the impact of spatial features should not be ignored.

All the prediction performances of the STDLN_F were the worst among the methods, and the results show the importance of the topology model of production nodes. It is usually thought that there exists a relationship between any two production nodes since they work together in a manufacturing system, and thus that the CNN may be effective in capturing spatial features without prior knowledge of material flow. However, the significant difference between the STDLN and the STDLN_F shows that the STDLN method only works well when inputting the appropriate topology model. In this paper, the two-level abstracting method was presented to construct the topology model, and the improvement in the topology model may be an interesting study to improve the prediction performance of the STDLN in the future.

3.5.3. Performance Comparison between the Datasets

The RSME and MAE metrics are not comparable between periodical datasets, and hence the normalized Acc was used. The scatter diagram of the Acc for all methods and all datasets is shown in Figure 8. The Accs of the STDLN for all datasets were the highest, and they fluctuated in a small range (0.94, 0.97). The results show that the STDLN method is insensitive to prediction periods. Thus, the STDLN can be applied for minute-by-minute and hourly energy consumption prediction. Although the GRU obtained satisfactory Acc for the 5 min, 30 min and 60 min sets, it fluctuated downward significantly for the 15 min set. The other methods show instability and an unsatisfactory performance with the datasets.

3.5.4. Performance Comparison between Production Nodes

Four production nodes XW, XW3, EX and EX2, which can be seen in Figure 6, were chosen to evaluate the effect of the STDLN method on production nodes at different levels. XW is an abstract workshop node, EX is an abstract machine node, XW3 is a visible workshop node and EX2 is a visible machine node. There is distributing flow from the top node XW to the bottom node EX2. The chart of Acc for the above four production nodes for the 5 min set and the 15 min set is shown in Figure 9. The prediction accuracy was found to decrease with the decline of the node level. In real manufacturing systems, the energy consumption of the lower node tends to have a more complex temporal law, and hence the corresponding prediction led to a worse performance. Even so, the Acc of the EX2 is still acceptable for application in energy wastage identification and energy-saving plans.

4. Discussion

4.1. Discussion of the Effect of GCN and Topology Model of Production Nodes

To better understand the effect of the GCN and the topology model of production nodes, the continuous prediction of all methods for the 15 min set of Node XW3 is shown in Figure 10. In the figure, the true curve and the STDLN prediction curve are highlighted by a heavy read line and a heavy green line, respectively. The STDLN prediction curve was found to fluctuate more gently than the true curve and the other prediction curves. There are two possible causes for this finding: (1) that the GCN model captures spatial features through the smooth filter in the Fourier domain; (2) that the topology model representing material flow between the production nodes leads the prediction curve to represent not the trend of individual change but rather that of overall change.

On the other hand, there was an obvious gap between the prediction curve of the STDLN_F and the true curve. The result proves the importance of the topology model of production nodes. The application case studied in this paper can be regarded as a typical manufacturing system consisting of parallel machines and flow lines. The parallel machines seem to be closely related because they work together in the same shop, but in fact they are independent of each other. In the STDLN_F, the unreasonable connections between the parallel nodes introduce serious noise interference and lead to worse prediction performance. By contrast, in the STDLN, the topology model of production nodes only includes dispatching flow and line flow; it filters the noise from unrelated nodes through graph convolution and leads to high prediction performance.

4.2. Discussion of the Effect of Prediction Period and Production Node Level

To better understand the effect of prediction period and production node level, the continuous prediction of the STDLN for all datasets of Node XW3 is shown in Figure 11, and the continuous prediction of the STDLN for all datasets of Node EX2 is shown in Figure 12.

As mentioned in the application case, XW3 is a node at the workshop level and EX2 is a node at the machine level. Observing Figure 11 and Figure 12, the STDLN method was always found to achieve satisfactory prediction results for the four prediction periods and two production nodes. As mentioned before, the energy consumption time series in different periods or at different node levels represent different temporal laws, but the STDLN can simultaneously capture these laws through spatio-temporal collaborative learning.

A certain error was also found between the true energy consumption and the prediction energy consumption. The error was mainly caused by the randomness of the machine work state and meter measurement error. Moreover, the error may have been caused by anomalies such as long idleness, machine fault and over-processing. This type of error is simply the basis of energy consumption anomaly diagnosis. Generally, prediction is the main method for building a baseline for energy consumption anomaly diagnosis. The authors of [1,12] studied energy consumption anomaly diagnosis using the parametric prediction method. Sequential prediction methods for a single node are also frequently seen in the literature related to anomaly diagnosis. Although parametric and sequential prediction methods for a single node can achieve high prediction precisions for time series, the STDLN method is expected to have two advantages: (1) efficiency, which means simultaneously predicting the energy consumption of multiple nodes, and (2) effectiveness, which means that the anomalies can be detected by the state of adjacent nodes. Hence, the application of the STDLN method in energy anomaly diagnosis in manufacturing systems is an interesting study topic for the future.

5. Conclusions

This paper presented a spatio-temporal deep learning network (STDLN) model for short-term energy consumption prediction of multiple nodes in a manufacturing system. The model combines a graph convolutional network (GCN) and a gated recurrent unit (GRU) and predicts the short-term energy consumption of multiple nodes based on a topology model of material flow and the historical energy consumption time series. The GCN is aimed at capturing spatial relationships with the adjacency relation, and the GRU is aimed at capturing dynamic change laws with the time series. Several experiments were performed on the power dataset from an aluminum profile plant. The results show that the model presented can predict the energy consumption of multiple nodes simultaneously and achieve a higher accuracy than models based on the GRU, GCN, SVR, etc.

The analyses of the results show that the STDLN can improve prediction performance through capturing both spatial features and temporal features from the energy consumption series of multiple nodes, obtain a stable performance on minute-by-minute and hourly datasets and obtain an acceptable performance for production nodes at different levels. The STDLN method is expected to support energy wastage identification, energy-saving plans and energy-saving control.

To build on this research, we suggest the construction of a topology model supporting more than one energy consumption type in order to improve the STDLN model through adding an attention mechanism and apply the STDLN in the anomaly diagnosis of energy consumption.