Machine Learning for Time Series Forecasting

Time series forecasting using machine learning (ML) enables businesses to use ML-driven digital solutions. They help analyze observed data to predict future values for specific time periods. In this article, we explore how ML plays a key role in transforming time series forecasting. Access insights into the components of time series data, various forecasting models, and the challenges and successes of implementing time series projects across industries.

What is time series forecasting? 

Time series forecasting is considered one of the most applied data science techniques that are used in different industries such as finance, supply chain management, production, and inventory planning. 

Stock price forecasting, weather forecasting, business planning, and resource allocation are only a few of the many possible applications for time series forecasting.

The time series forecast models based on machine learning found wide implementation in time series projects required by various businesses for facilitating the predictive distribution of time and resources.

In this post, we want to share our experience while working on deep learning for forecasting projects.

First, we'll talk about time series data and its role in forecasting techniques. Later we'll discuss various classical and machine learning methods for analysis and forecasting so that you can gain a deeper understanding of using machine learning in business. We'll walk through the main steps taken while implementing time series machine learning forecast projects and analyze the main challenges that may arise during the project.

Time series forecasting in machine learning

Before anything else, it is important to review first what time series is, as well as time series analysis and forecasting.

Time Series pertains to the sequence of observations collected in constant time intervals, be it daily, monthly, quarterly or yearly.

Time Series Analysis involves developing models used to describe the observed time series and understand the "why" behind its dataset. This involves creating assumptions and interpretations about a given data.

Time Series Forecasting makes use of the best-fitting model essential to predicting future observations based on the complex processing of current and previous data.

Machine learning forecasting proved to be the most effective in capturing the patterns in the sequence of both structured and unstructured data and its further time series analysis forecasting.

Speaking of applying a suitable model for deep learning for time series forecasting, it is important to understand the components of the time series data:

✔Trends (to describe increasing or decreasing behavior of the time series frequently presented in linear modes).

✔Seasonality (to highlight the repeating pattern of cycles of behavior over time) 

✔Irregularity/Noise (to regard the non-systematic aspect of time series deviating from the common model values)

✔Cyclicity (to identify the repetitive changes in the time series and define their placement in the cycle). 

Modeling time series

Among the numerous methods aimed at achieving precision and minimizing errors and losses within forecasting, there are several classical and modern machine learning methods that prove their accuracy and computational relevance. 

There are various types of time series analysis. The most widely applied models for deep learning for time series forecasting projects by categories are as follows: 

1. Classical Methods

Tried-and-tested classical methods are the most widely applied to develop time series forecasts. These models can deliver accurate time series predictions using spreadsheets and formulas applied to data sets.

However, classical methods have some drawbacks that make them insufficient for building accurate forecasts in rapidly changing environments.

1.1 Naïve model

In most cases, naïve models are applied as a random walk (with the last observed value used as a unit for the next period forecast) and a seasonal random walk (with a value from the same period of the last observed time span used as a unit of the forecast).

1.2 Exponential Smoothing Model 

The method uses the foundation of machine learning time series classification. Forecasts are made on the basis of captured weighted averages and according to weights decreasing as the observer tracks back in time. 

The method implies a smoothing constant that helps make previous data less essential and the newest data more crucial to build time-series predictions. The constant adjustments when building forecasts make more recent data more or less valuable compared to historical data. The model can not help detect trends and seasonality. However, a number of extensions of the simple exponential smoothing (SES) have been introduced to include the trend/damped trend and seasonality.

1.3 ARIMA/SARIMA

ARIMA stands for the combination of Autoregressive (AR) and Moving Average (MA) approaches within building a composite model of the time series. ARIMA models include parameters to account for season and trend (for instance, dummy variables for weekdays and they're distinguishing). In addition, they allow for the inclusion of autoregressive and moving average terms to handle the autocorrelation embedded in the data.

1.4 SARIMA stands for Seasonal Autoregressive Integrated Moving Average: it widens the application of the ARIMA by including a linear combination of past seasonal values and/or forecast errors. 

1.5 Linear Regression method 

Linear regression is the simple statistical technique commonly used for predictive modeling. Breaking it down to basics, it comes to providing an equation of independent variables. The defined relationship between variables helps predict future values.

2. Machine Learning Methods

ML-driven methods use machine learning to analyze large amounts of data using trained models.

2.1 Multi-Layer Perceptron (MLP)

As an applied machine learning approach, the time series model forecasting implies the triple structure of the initial layer of the network, which takes in an input, a hidden layer of nodes, and an output layer used to make a prediction.

Benefits: 

• Simplicity — a simple and straightforward architecture that enables the opportunity to create ML models quickly.
• Flexibility — can handle a large assortment of inputs as it's not limited to sequential data.
• Efficiency — requires less computational power and training time with fewer parameters.

Application example: The multi-layer perceptron model forecasts sales figures by analyzing past sales records. Also, it can analyze external and seasonal factors like weekends, holidays, etc.

2.2 Recurrent Neural Network (RNN)

RNNs are basically neural networks with memory that can be used to predict time-dependent targets. Recurrent neural networks can memorize the input's previously captured state to decide on future time steps. Recently, lots of variations have been introduced to adapt Recurrent Networks to a variety of domains.

Benefits: 

• Sequence handling — RNN is capable of processing sequential data for analyzing a series of inputs submitted in a certain order.
• Memory capability — can keep information about previous inputs for developing better predictions when analyzing time-dependent data.
• Adaptability — can be extended and adapted to handle more complex sequence-based tasks.

Application example: The RNN analyzes the sequence of past market data to predict future changes in a stock market.

2.3 Long Short-Term Memory (LSTM)

LSTM cells (special RNN cells) were developed to find the solution to the issue with gradients by presenting several gates to help the model make a decision on what information to mark as significant and what information to ignore. GRU is another type of gated recurrent network.

Image Source

Besides methods mentioned above, Convolutional Neural Network models, or CNNs for short, as well as decision tree-based models like Random Forest and Gradient Boosting variations (LightGBM, CatBoost, etc.) can be applied to time series forecasting.

Benefits: 

• Long-term dependency learning — helps capture long-term dependencies in data, surpassing the limitations of RNNs.
• Stability — a reliable architecture for training on long sequences. 
• Versatility — can be applied to a large assortment of domains like speech recognition, finance, etc.

Application example: An ML model analyzes historical data on energy consumption to identify patterns, considering long-term dependencies and trends. It helps deliver predictions by analyzing large datasets with records over long periods.

Evaluating Model Accuracy 

Speaking of time series methods, we should mention that visually identifying which machine learning model has the best accuracy does not always produce reliable results. 

Among the time series prediction models, the method of calculating the MAPE (Mean Absolute Percent Error) shows the best results as a quicker way to compare the overall forecast accuracy of a proposed model.  

The measures represent the percentage of average absolute error that occurred. The overall idea of calculations for the evaluation of model accuracy comes to the following proportion: the lower the MAPE, the better the forecast accuracy.

Classical vs ML-Driven Analysis

Classical time series analysis methods are good for simple datasets comprising records that have defined dependencies.

Benefits	Drawbacks
Simple to use Interpretability Low computation requirements Fast for small datasets	Basic analysis Assumption-based results Poor performance for large datasets Limited flexibility

ML models advance in capturing complex patterns and processing vast datasets. They help identify hidden dependencies and adjust algorithms, considering past inputs.

Benefits	Drawbacks
Advanced model complexity Automatic learning Top-tier scalability Accurate predictions	High computation requirements Large datasets are required  Configuration and tuning complexity

 Time series forecasting process

To avoid any detrimental consequences and ensure the project's success in terms of designing the predictive time model, deep learning for time series forecasting is being implemented by taking the following steps.

1. Project Goal Definition

It is the first step of the time series machine learning tutorial. Prior to discussing the project in detail, make sure that you understand the subject. It means understanding the specifics of the business domain of forecast operation, including terms and key definitions, as well as common business models pertaining to the particular domain. Thus, this stage implies defining the project specifics through extensive research within the area of knowledge.

Stage outcome: A clear and well-defined project scope and objectives. It helps create a single document outlining the final goals, including forecasting needs, key metrics, and success criteria. The documents can be shared with all stakeholders and involved experts to avoid misunderstanding.

2. Data Gathering and Exploration

Defining the basics leads to a clear view of the scope of data you need to collect to facilitate the further discovery of data insights. By implementing techniques for building plot graphs and visualization charts, the domain knowledge reception reaches a level required for strategic data exploration and estimating hinges and trends to evaluate the variations in volume further. It also helps specify the forecasting task and conduct preliminary exploratory analysis successfully.

Stage outcome: A set of data that will be utilized for machine learning model training and testing. The data may be collected from various sources. Moreover, datasets can be augmented by generating synthetic data.

3. Data Preparation

At this stage, the development team performs cleaning data for relevant insights and further subtracting the variables of importance. The data preparation process for feature engineering is being launched. The key component of feature engineering is targeting the areas of knowledge of the domain that is crucial for designing the new features in the existing dataset.

Stage outcome: A clean and pre-processed dataset ready for modeling, comprising no errors or missing records. Inconsistencies and outliers are detected and corrected to achieve top-tier accuracy of the developed result.

4. Applying the Time Series Forecasting Method

On the basis of preliminary data preparation and exploratory analysis of a range of time series forecasting conducted at the previous stage, the team works with several models and chooses one based on the criteria of relevance and projected accuracy of the forecast. Fitting the model for project performance ensures the proper model development and consideration of variables essential within the forecasting process.

Stage outcome: A set of trained machine learning models designed for analyzing datasets and delivering certain insights. 

5. Evaluation/Validation and Performance Comparison

This step covers the optimization of the forecasting model parameters and achievements of high performance. By applying a cross-validation tuning method implying the data split, data scientists train forecasting models with different sets of hyper-parameters. Completing this stage requires applying performance score estimation and evaluation of diverse test datasets. It is important to take an out-of-sample approach to receive adequate performance evaluation when processing a particular type of data.

Stage outcome: The implication of the testing datasets for evaluating the performance of ML models and the accuracy of delivered insights. Fine-tuned and optimized model parameters to achieve the best possible results.

6. Deployment

This stage includes the forecasting model integration into production. At this particular stage, we highly recommend setting a pipeline to aggregate new data to use for the next AI features. It helps in data preparation work when performing your future projects.   

The forecast development requires agility in approach as it is an iterative process.

The iterative loops incorporate a series of exploration and visualization to get data. Once visualization has been performed, it might be required to take a step back to gather additional data. The models are revised and updated as new data and new insights are made available.

Thus, this stage shifts focus to the ongoing development and refinement of one or more models until a relevant level of performance is achieved.

Stage outcome: An iterative loop of continuous ML model utilization, testing, and improvement. 

USE CASES OF ML TIME SERIES FORECASTING

ML models can uncover patterns and trends that traditional methods might overlook. It can be applied in many industries and adjusted to solve niche-specific problems.

1. Retail and eCommerce

ML-driven time series analysis helps explore historical data to predict customer demand or sales. It can also consider seasonality or other repetitive events to develop accurate forecasts. The core peculiarities of using ML-driven forecasting in retail are:

• Demand prediction—the technology enables the opportunity to process large volumes of unstructured data from various sources (e.g., sales, prices, weather, promotions, holidays). Unlike traditional methods, machine learning is capable of extracting complicated patterns from this data to build accurate forecasts.
• Dynamic pricing—the insights delivered by ML time series forecasts can help a company to balance promotion activities and profitability in the most effective way. The marketing team can plan promotions, considering future demand spikes and drops.
• Improved customer experience—forecast delivered by time series analysis helps understand what customers are likely to buy along with certain production, offering the best options to upsell/cross-sell.

2. Finance and Trading

A trading company uses machine learning to analyze historical stock price changes. Also, the technology helps identify the correlation between external factors and price spikes/drops. The key applications of ML are:

• Comprehensive analysis—many factors like market sentiment, trading volumes, and news events are considered along with historical trading data. The detailed algorithms help understand patterns but also when exactly stock price is going to change its trend.
• Algorithmic trading—insights delivered by an algorithm help run high-frequency trading, automatically triggering a rapid sequence of buy orders. The thorough forecasts help algorithms know when to exactly trigger buy or sell actions.
• Portfolio management—the detailed analysis of assets helps better understand which are likely to perform well over the short and long term.

3. Manufacturing

Manufacturers use ML-driven time series forecasting for running predictive maintenance to understand when equipment is likely to fail by analyzing data gathered by sensors. Also, deep data analysis helps optimize operations, resulting in decreased downtime, with the following key components:

• Predictive maintenance—ML time series forecasting analyzes sensor data from machinery like vibration, temperature, pressure, and more. Continuous analysis helps identify patterns and detect anomalies that can lead to failure. The ML-driven analysis helps understand when a defective mechanism will fail to schedule maintenance and identify a problem’s root causes. 
• Raw materials demand forecasting—manufacturers need to fulfill the needs of their clients that depend on product demand spikes and drops. ML time series analysis helps adjust raw materials sourcing strategies to fully fulfill the clients’ request without delays and overstocking.
• Operations optimization—the ability to analyze various data sets enables the opportunity to optimize inventory processes, energy consumption, and more by getting detailed insights and root cause analysis.

4. Web Resources

Internet providers and web resources use ML time series forecasting to help predict traffic demand. Insights deliver detailed information on how to optimize server and bandwidth allocation. The key applications of the technology are:

• Web traffic prediction—go beyond basic user patterns by analyzing external events that may cause one-time traffic spikes or drops that distract the audience. For example, a system can accurately predict when a site is about to experience high demand for certain content types by analyzing social media trends.
• Downtime prevention—the analysis of external factors helps determine when a website is likely to experience a one-time significant rise in traffic caused by certain events. Hence, a web provider can adjust configuration to handle expected traffic. 
• Hosting cost optimization—access to accurate traffic forecasts helps a company optimize server capacity during off-peak time.

CODEIT EXPERTISE IN TIME SERIES FORECASTING

Our team has extensive expertise in developing ML models for analyzing time series. Let’s learn more about one of our cases in the manufacturing industry.

Problem

A client has contacted the CodeIT team, requesting us to develop a digital solution for running predictive maintenance, using the 500gb dataset with parameters of equipment. He wanted a model to thoroughly predict when a piece of equipment is likely to fail so that he could update production schedules and order new equipment in advance.

Solution Implementation

We’ve analyzed project requirements and prepared a proof of concept (PoC) to validate the client’s idea. After collecting and analyzing the feedback from the client, our team has developed a minimum viable product (MVP). We’ve released the MVP and supplemented it with new features that were released incrementally.

The developed solution gathers data from shop floor sensors and analyzes it using AI. Insights delivered by the systems help spot anomalies and detect trends. The system builds predictive models that help understand when exactly the piece of equipment will reach the point of failure. Also, we’ve enabled users to create custom dashboards. Hence, different user roles can compose personalized dashboards to access crucial information in an easy-to-digest way.

BEST PRACTICES FOR ML-DRIVEN TIME SERIES FORECASTING

Avoid pitfalls and faulty ML-driven time series analysis with these best practices. 

1. Understand the nature of data — Analyze your data sources, ensuring their quality and determining if they are suitable for developing predictive models. Check how data is collected to ensure it contains no random or missing records.
2. Develop appropriate evaluation metrics — Implement advanced error metrics to avoid possible misleading results that mean percentage error can deliver. Also, visualization tools are recommended for comprehensive result assessment.
3. Make data stationary — Check if the data is stationary, meaning its statistical properties do not change over time. If not, transformations like differencing can be applied to make it stationary.
4. Start from basic models — Make a project kick-off by developing a simple ML model for time series forecasting. Increase the complexity of trained models over time. 
5. Investigate autocorrelations — Thoroughly analyze delivered results to ensure that predictions don't have autocorrelation but are a reflection of genuine learning.
6. Ask critical questions — Be skeptical of results and avoid drawing premature conclusions. Evaluate results to ensure that a model does not deliver random results.

COMPANIES THAT USE ML-ENABLED FORECASTING

ML-driven solutions are widely used by top companies in many different industries. 

Let's learn more about three successful technology adopters and explore machine learning demand forecasting examples.

BlackRock

BlackRock is considered the most successful company in the investment, advisory, and risk assessment industries. The company managed to achieve a significant competitive advantage by becoming an early adopter of machine learning and artificial intelligence. 

For instance, Aladdin Risk, one of the company's products, uses machine learning to analyze time series to deliver sophisticated risk analytics.

Amazon

Amazon is the leading company in the eCommerce industry. The company constantly evolves and implements new technologies to keep the leading positions.

It collects a lot of data about its customers. The company uses ML models to analyze all the data to get detailed insights, build accurate time series predictions, spot unusual activity, etc. With the help of machine learning, the company can analyze time-series data to offer the best purchasing suggestions to its customers and minimize the fraud rate.

Walmart

One of the most popular retail companies in the US uses machine learning to predict sales and optimize its supply chain. The company applies forecasting algorithms to turn raw data collected by its stores into useful insights. The use of ML-driven forecasts helps the company satisfy the ever-changing customer demand and optimize its operations.

Time Series Forecasting Project Challenges 

We would like to share the experience we acquired in performing forecasting projects and pinpoint the challenges that the development team might face. 

Lack of Data

The bigger the datasets are, the more training data the system can access, which leads to higher accuracy of predictions. However, there are limitations to using machine learning associated with a lack of historical or seasonality data for a target variable. Consequently, a lack of data might result in an overall decrease in forecasting precision.

How to overcome: Run synthetic data generation to increase the size of your dataset or partner with a company to train an ML model using the data gathered by another company. Also, you can obtain models pre-trained on relevant data.

Lack of Domain Knowledge

Apparently, without adequate domain knowledge, the stage of feature engineering itself as the key component of ML implementation falls under high risk. In general, domain knowledge can help improve the quality of models in any project. To prevent the problem arising from the lack of domain knowledge, the expertise of business niche specialists is required.  

In addition to the problems already mentioned, while working on the stock price forecasting project, our major concerns were related to the heteroscedasticity and chaoticness of stock prices.

How to overcome: The foremost recommendation is to hire an experienced vendor to avoid delays and budget overruns because of failed ML model training and time series analysis. In addition, you can leverage pre-built solutions that require minimal tech expertise for gathering datasets and building ML models.

Poor Data Quality

The quality of data directly affects the accuracy of results. Data cleaning is a heavy-lifting task that should be performed to achieve the best results. 

The main problems that should be tackled when preparing data for further analysis are:

• Missed values
• Improperly formatted data
• Inconsistent values
• Unreliable data sources
• Irrelevant data attributes

How to overcome: Implement automated data cleaning tools to remove bias or outliers. Use the tools to detect and correct errors in your datasets. Also, advanced data imputation tools can be used to automatically generate missing values by analyzing the nearest values.

TOOLS FOR ML-DRIVEN FORECASTING

Software engineers use a large number of different tools to develop well-tailored ML-driven solutions. Python is the most popular programming language used for creating machine learning time series predictions due to its many advantages.

The most popular services that offer access to versatile tools for developing ML-enabled forecasts are:

Google Cloud

Google Cloud offers access to a broad range of products that help develop AI- and ML-enabled solutions. The most popular are:

• TensorFlow
• BigQuery
• VertexAI

Software engineers widely use these tools to train machine learning models for getting data-driven forecasts. BigQuery and VertexAI can use Google's cloud computing infrastructure to analyze large datasets.

Azure Machine Learning

Azure Machine Learning is a service developed by Microsoft. It offers access to many tools for the end-to-end development of machine learning models. Also, users can train and deploy their models using the solution. 

Using the Azure Time Series Insights tool, developers can visualize all the data in live time for further analysis.

AWS Machine Learning

AWS Machine Learning is a forward-looking service from Amazon. It offers access to a cloud-based infrastructure and versatile tools that help analyze datasets. Amazon Forecast is a popular tool that helps analyze time series data to develop predictive models. The solution helps inspect time series data used for machine learning and pick the best models.

Final Thought

The complexity of time series forecasting project implementation demands the highest quality of development, which our team experts can provide.

With the knowledge the company accumulated in similar project performance, we surely meet your project requirements in regards to thoroughly considering domain specifics and business goals of the particular time series prediction example.