Continual Learning for Time Series Forecasting: A First Survey ^†

Abstract

Deep learning has brought significant advancements in the field of artificial intelligence, particularly in robotics, imaging, sound processing, etc. However, a common major challenge faced by all neural networks is their substantial demand for data during the learning process. The required data must be both quantitative and stationary to ensure the proper computing of standard models. Nevertheless, complying to these constraints is often impossible for many real-life applications because of dynamic environments. Indeed, modifications can occur in the distribution of the data or even in the goals to pursue within these environments. This is known as data and concept drift. Research in the field of continual learning seeks to address these challenges by implementing evolving models capable of adaptation over time. This notably involves finding a compromise on the plasticity/stability dilemma while taking into account material and computational constraints. Exploratory efforts are evident in all applications of deep learning (graphs, reinforcement learning, etc.), but to date, there is still a limited amount of work in the case of time series, specifically in the context of regression and forecasting. This paper aims to provide a first survey on this field of continuous learning applied to time series forecasting.

1. Introduction

Machine learning applied to nonstationary environments, where changes can occur over time, is a rapidly growing field as it represents a challenge for future artificial intelligence applications. The exploration of new models with adaptive capabilities is currently grouped under the terminology of continual learning (CL). This domain encompasses themes like progressive learning [1], incremental learning [2], and lifelong learning [3]. Despite some specific differences, these various designations converge on the topic of a continuously learned model and the issue of catastrophic forgetting. Indeed, in real-world applications, obtaining an exhaustive and representative dataset before learning is often impossible. Thus, continual learning advantage lies in its incremental learning ability over time, avoiding complete retraining on a potentially substantial amount of data. Its interest, compared to standard models, is particularly significant in the case of data drifts where the data distribution changes over time [4]. Overall, the idea is to seek the perfect model able to evolve with nonstationary data, while dealing with plasticity/stability dilemma and without exhibiting symptoms of forgetting knowledge [5,6,7,8,9,10]. This is achieved while enhancing its efficiency when applied to new data as well as on previously seen data.

Several solutions exist for addressing the issue of catastrophic forgetting [11]. They can be grouped into three kind of strategies: rehearsal strategy, consisting of repeating a fraction of past data during learning; regularization-based strategy, applied on model parameters; and structural strategy, where the model grows over time when encountering new tasks [12,13,14,15,16]. Each strategy and its hybrid/cross versions [1,17,18,19] present a trade-off between constraints and advantages.

The study of continual learning takes multiple research directions depending on application domains, including strategies dedicated to graphs [20], reinforcement learning [21,22], classification [23], or regression [7] tasks. In this work, we focus on the review of forecasting tasks applied on time series data, drawing a parallel with the classification domain, which represents one of the most explored axes in continual learning. Additionally, this work seeks to provide concise insights about required notions in the topic of continual learning while presenting compelling analyses aimed at addressing inherent questions associated with the principles of continual learning.

2. Continual Learning Principles

The problem of catastrophic forgetting, which is the cornerstone of continual learning, is correlated with the notion of plasticity/stability dilemma. A model exhibits great stability when it learns from a fixed dataset, supposed to represent perfectly and extensively all the situations (perfect sampling). If the data do not change over time, meaning their distribution, patterns, or the environment remain constant, then the stability of a model enables it to predict the desired outcome with high precision throughout all its usage. Nevertheless, it fails to address changes in data distribution or in goals to achieve. On the other side, the plasticity of a model allows it to continuously and quickly adapt to new tasks and data, which is one of the main strengths of continual learning. However, the downside of excessive plasticity is the loss of retaining past knowledge: as the model evolves over time, the accuracy decreases on previously seen data. This is the principle of catastrophic forgetting. For a standard model, a fine-tuning mechanism could be applied occasionally to allow model readjustment to some extent, for new tasks and data. Nevertheless, this strategy remains limited by the amount of data needed for adaptation as well as by the fact that old knowledge and abilities of the network are still lost [24]. The goal of continual learning is to address this plasticity/stability dilemma, but the solution is often constrained by an increasing cost in performance (memory, time, model size, etc.).

To address the catastrophic forgetting issue, the literature of continual learning introduces three strategies: data rehearsal, parameter regularization, and model extension [24]. The principle of data replay (rehearsal) involves constantly reintroducing samples of past data into learning steps to adapt the model [25]. The goal is to prevent forgetting by continuously stimulating the model with past knowledge. This approach often yields the best results in terms of model accuracy, but it comes with a perpetual increase in the amount of data that the model must ingest over time. Thus, recent research is focusing on a rehearsal-free version [26]. Multiple axes can be studied regarding the management of data to be reintroduced, as presented in [13,27,28,29]. The approach of architectural modification (model extension), presented in [30,31], involves adapting the model directly according to the needs. The size of the layers is growing while encountering new tasks, avoiding modification of parts of the network that are used for managing previous tasks. By avoiding the modification of previously learned weights, there is no loss in performance on previously learned tasks. However, similar to the data replay strategy, its downside is the continuous growth in the model size. This issue can be addressed by incorporating a pruning mechanism [32] to remove components within the network as they become unnecessary. But this process is not well established and still has its drawbacks regarding performance. Lastly, the regularization approach holds the best potential in the long run, although it may potentially exhibit some forgetting over time. It involves penalizing the modification of neural network weights based on their impact on model accuracy. As presented in the EWC algorithm [12,33,34], the aim is to select, through penalization, a weight space of the model that is the best to address all the tasks learned beforehand. The major advantage is that the work takes place in a Fisher matrix to compute the model loss during its update, which does not imply network or data replay growth. On the contrary, if too many disjointed tasks are learned, it can lead the model to propose an average version of itself, satisfying a minimum performance on all tasks without performing in any of them. These three main strategies also have their hybrid versions, combining, for example, regularization with data replay to seek the best possible performances [17,19,35,36,37].

Currently, the field of continual learning research is mainly focused on classification tasks (imagery, natural language processing, medicine, etc.) and, particularly, on three scenarios for adapting the model over time, as presented in [2]:

• “Task Incremental Learning where it sequentially learns to solve a number of distinct tasks” (a task is associated with achieving a goal within a context, and if either or both change, we switch tasks);
• “Class Incremental Learning discriminate between incrementally observed classes”;
• “Domain Incremental Learning learn to solve the same problem in different contexts”.

In all these situations, the model is progressive, and its classification capabilities increase overall as it encounters new data over time. Thus, the goal within the classification research field is to propose new approaches to solve the catastrophic forgetting problem while achieving satisfactory precision results. Thanks to several standardized databases, such as MNIST and CIFAR, etc., used to compare and evaluate the performances of new models, new continual learning classification models [12,15,17,32] can be compared easily with already verified standard models.

3. Continual Learning for Time Series

Similarly to in the classification research field, continual learning applied to time series forecasting aims to address drifts in tasks and data, with the ability to continuously learn and make a model evolve over time, while resolving catastrophic forgetting issues. Although there are relatively common databases representing time series, most research on continuous learning for regression tasks focuses on specific applications for each use case which. Even if these applications promise concrete improvements in the field of time series forecasting, the comparison between standard models as well as other continuous learning algorithms is not straightforward.

3.1. Main Principles and Overview

Compared to classification domain, two scenarios can be encountered in the forecasting domain, as outlined in [38]—page 4:

• Incremental learning of the data domain, which refers to the situation where the underlying data generation process is changing over time, due to the nonstationarity of the data stream. This means that the distribution of the data relative to the same objective is varying over time.
• Incremental learning of the target domain, which refers to the situation where the output of the model varies over time. This is the case when the number or properties of prediction targets is changing (prediction of new variables like in multioutputs networks, changing in time prediction horizon, etc).

The literature on continual learning for regression and forecasting does not address, at all, the task incremental scenario, as defined previously for classification problems. When ‘task’ is mentioned, it most often refers to one of the two scenarios defined here (target or data domain incremental learning).

The incremental learning of the data domain refers either to tasks change defined by external supervision, human, or machine, as in the case of labels for classification, or to tasks that are drifting “naturally” from one to another because of changes in the environment that are not always identifiable (apparent drift vs. real drift). The fact is that most research works on continual learning for time series are considering task management based on an arbitrary separation of the dataset into fixed-size subsets, each one corresponding to a new task [24,27,29,39,40,41,42,43,44,45,46]. These approaches are called Task-based approaches. Although operational and useful for evaluation of CL systems, this task management may not represent real transitions between tasks. It mainly relies on an incremental learning approach where the addition of tasks allows the model to progress through the data, leaving catastrophic forgetting management solely to the implemented strategy. On the contrary, correct task management (either based on explicit supervision of the system by an expert or based on an unsupervised supervision of the data) can helps the model to avoid catastrophic forgetting (including better selection of data for replay in the rehearsal approach or proper selection of weights to retain in the regularization approach), as well as to deal more precisely with plasticity/stability dilemma. Ref. [47] particularly addresses the issues related to arbitrary task definition and proposes a new task detection system through loss analysis during model learning (referred to as Task-free approach). Other research works, such as [35], propose the idea of “novelty” data buffer based on their distance from the prediction accuracy.

Considering the application domain of continual learning for forecasting, we can observe that it is mainly applied in the energy domain [24,35,36,39,40,41,46]. Other sub-cases of application refer to industrial maintenance [36,43], climatic [36,42,45], or traffic analysis [27,36]. Each application uses a distinct dataset, hindering direct model comparison. It is noteworthy that, in all presented application cases, even the lowest performances still surpass those of standard baseline models. Also, we can observe that even if the issue of catastrophic forgetting [11] was already known, it was not widely emphasized for improving model performance. Only the last few years have seen a trend of consolidating research in this field under a common banner like continual learning, with the question of solving the ‘catastrophic forgetting’ matter.

3.2. Continual Learning Analysis

Continual learning techniques have already proven to be effective in various domains. In the case of forecasting data from time series, ref. [24] highlights the interest of the principle (named here lifelong learning) (see Figure 1). It uses a mechanism of replaying sample from past data coming from the time series.

This approach is compared with fine-tuning and Joint-training, which provide an upper bound of performances in terms of prediction accuracy. The upper bound is represented by the joint dataset of all available learning tasks at a given time. Lifelong learning demonstrates performances close to the upper bound, contrary to fine-tuning, which exhibits a loss of precision on old tasks.

He et al. [41] (see Figure 2) also illustrate the superiority in term of MSE of some continual learning models (LWF, EWC, O-EWC, SI) compared to a baseline in the context of target and data-domain incremental scenarios. This is achieved most often at the expense of a cumulative increase in learning time. The dots show MSE metric and the learning time for each experiment. Squares indicate the average test error for each task during training, and stars indicate the average test error for each task after training all tasks.

In the same paper [41] (see Figure 3), the authors provide an analysis of the forgetting rate (Equation (1)) along with the new tasks encountered in time series forecasting. The results show a better overall accuracy (Figure 2), as well as a much lower forgetting rate over time, when using continual learning models, compared to the standard model without any help against catastrophic forgetting (Figure 3).

$$forgettingratio={\displaystyle \frac{max(0,Lwarm\_u{p}^2-Lwarm\_u{p}^1)}{Lwarm\_u{p}^1}},$$

(1)

where $Lwarm\_u{p}^1$ indicates the MSE on the warm-up dataset at the end of the warm-up phase and $Lwarm\_u{p}^2$ indicates the error on the same dataset at the end of the update phase.

In the case of highly periodic datasets, the question of the usefulness of a continual model against a standard model arises. The idea is that a training on a sufficiently large dataset encompassing an entire periodicity may suffice on its own. Presented in [35] is an experiment based on an artificial dataset with daily and annual periodicity as well as an application on a dataset representing wind farm energy production (Experiment 1—Table 9—[35]). The models were trained on a subset of data in an offline manner before being applied to online learning. On average, the results on periodic data showed worse performances than the standard model due to the plasticity of continual learning. Specifically, 1120 models were trained, and only 32.1% showed better results than the standard model. The parameterization of the CLeaR model [35] applied here defines its ability to understand the notion of data periodicity. With the right selection, the continual learning model can still be close to a standard model. That said, the application of the model to the second dataset from wind farms distinctly demonstrates the advantage that continual learning can bring to time series forecasting. Although the wind-dependent dataset exhibits daily and seasonal periodicity, climatic variations have applied variations within the data shift. This can be observed in the results given in

Table 1. Average MSE for the three instance of the same model over 10 wind farms forecasting datasets (Table 9—[35]). Model C is the CLeaR continual learning model (best in bold).

	Fitting Error MSE (e-2)
	Instance A	Instance B	Instance C	Baseline
Mean	5.138	5.442	2.829	3.190

, where three instances, “A”, “B”, and “C”, of the same model are compared: instance “C” represents the continual learning model, which adapts more effectively than the standard model “A” and the fine-tuned model “B” to changes. Furthermore, this study shows a lower average catastrophic forgetting rate, for instance, “C”, both in the AE (autoencoder) part and in the predictor, then “B” (fine-tuning instance), highlighting the contribution of strategies to provide assistance against knowledge forgetting when learning (see

Table 2. Average forgetting ratio over the 10 wind farms forecasting datasets from CLeaR model instances (Table 11—[35]).

	Forgetting Ratio
	Instance B (AE)	Instance C (AE)	Instance B	Instance C
Mean	1.402	1.171	3.550	1.161

).

3.3. Trends and Challenges

The development of deep learning in time series forecasting is still growing, with little concrete application cases [48]. However, emerging sub-domains like continual learning are already being studied for use in dynamic environments. The literature shows promising application outcomes designed to face the challenge of unstable environments where data evolve over time. Two main trends in the research can be seen:

• The first one focuses on model evolution and the crossover of different common continual learning strategies to more effectively mitigate catastrophic forgetting [35].
• The second distinguishes itself on a structural level by implementing models with a biologically inspired approach, aiming for behavioral mimicry of living organisms [36].

Future challenges revolve around adapting complex and recent popular deep models, such as those used in image or language processing. Notably, encoder–decoder systems and attention mechanisms [49] show great promise, with transformers or autoencoders that have proven their ability to handle time series data [24,35,36,41,43]. Furthermore, while it is important to diversify the application of such models and their strategies to different use cases to provide a broad range of expertise, a lack of common benchmarks in the literature on continual learning for time series prediction is noticeable and highlights the need for resolution of this.

Also, most strategies against catastrophic forgetting were initially developed for supervised classification tasks and then adapted for regression tasks. This explains the low number of research papers proposing task-free solutions, where tasks are not predefined (contrary to what is mainly seen in image classification). Also, papers that implement incremental learning models, like [50,51,52,53,54,55], do not adopt an approach against catastrophic forgetting and focus only on adaptive approaches over time. This denotes an area for improvement and bridges the development gap between classification and regression.

Other challenges involve developing new relevant metrics to better describe the effectiveness of a model under continual learning strategy, including the concept of a model forgetting rate and the stability/plasticity related to the adaptation speed of a model’s loss function during its learning process. It would also be relevant to propose complementary measures related to the cost of continual learning strategies in terms of resource usage (CPU, GPU, memory, electricity) [56]. This would allow another kind of comparison with standard incremental approach such as joint training. Also, it is currently recommended to provide solutions to make the model’s results explainable and interpretable. This explainability serves the purpose of understanding how the model operates as well as providing legitimacy to the results obtained. If this was already investigated in the field of time series [57,58], continual learning becomes, at the same time, a new playground as well as a potential source of explainability to explore.

4. Concluding Remarks

The application of deep learning in the field of time series forecasting is still growing compared to common statistical approaches. One of the strengths of neural networks lies in their learning abilities. Although they are incremental in nature, their application is mostly static, with separate training and testing batches to validate their capacity. Their application is becoming increasingly ubiquitous in various domains of society, particularly in resource monitoring and demand prediction to effectively adapt any underlying process. Continual learning, by its name, is one of the techniques currently under development that are seen as key to address these issues by offering adaptability and stability solutions and ensuring the evolution and relevance of a service. The pioneering domain is for classification tasks, which currently receive much more attention than any other domains such as time series prediction. The three major approaches currently seen in the literature of continual learning focus on architecture, replay, and parameter regularization to meet expectations. These approaches provide the groundwork for building hybrid solutions more suited to each application case.

In recent years, there has been specific research applied on time series forecasting with continual learning strategies. One of the key points to consider in this domain, as in the others, is how patterns are reoccurring and how to avoid catastrophic forgetting. The other key point comes with the notion of tasks which are neither well defined nor managed in time series. In this domain, tasks are often implicit and we need task-free mechanisms and user interactions depending on application domains.

We also need to propose and agree on common benchmarks dedicated to continual learning. Similar to classification, this would allow for easy assessment of the performance of each model using multiple standard metrics including MSE, MAE, and RMSE, as well as domain-specific metrics such as, for example, the forgetting rate of a model or stability/plasticity metrics to define. It would also be relevant to propose measures related to the resource consumption (CPU, GPU, energy) needed over time for continual learning, in comparison with traditional methods. Analysis could be performed, considering the benefits in performances on one side compared to computational and energy resources involved. Budgeted strategies can also be explored.

Applications of continual learning are starting to emerge, especially in the fields of energy and economic forecasting for businesses. However, this remains relatively minor considering the numerous application domains where continual learning could have a real impact, such as in medicine and environmental studies (meteorology, climate, and hydrology) [10,27,59,60]. Further research and development are essential to unlock the full potential of continual learning in time series forecasting across diverse fields, paving the way for its broader adoption and integration into various real-world applications.