How to make a Transformer for time series forecasting with PyTorch
This post will show you how to transform a time series Transformer architecture diagram into PyTorch code step by step
Transformer models have shown state of the art performance in a number of time series forecasting problems [1][2][3].
In this post, you will learn how to code a transformer architecture for time series forecasting in PyTorch. Specifically, we’ll code the architecture used in the paper “Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case” [2] and we will use their architecture diagram as the point of departure.
So step by step, I will show how to code each of the components of the diagram. This way, you will learn the generalizable skill of interpreting a transformer architecture diagram and converting it to code.
I will explain the process as if you have never before implemented a transformer model. I do, however, assume that you have basic knowledge of PyTorch and machine learning in general. The final result will be a class that we will callTimeSeriesTransformer where everything comes together.
I will also explain what the inputs to the model’s forward() method must be and how to create them.
