Transformer时间序列预测：多变量输入与多变量输出

最新推荐文章于 2024-10-28 19:59:23 发布

原创

最新推荐文章于 2024-10-28 19:59:23 发布 · 2.7k 阅读

39 ·

CC 4.0 BY-SA版权

文章标签：

#1024程序员节 #时间序列 #预测 #深度学习 #transformer #python #异常检测

文章目录

引言

任务描述

使用100个时间步的多变量时间序列预测未来20个时间步的目标序列。自变量有10个，因变量也有10个。

代码参考：

https://github.com/oliverguhr/transformer-time-series-prediction

https://github.com/RuifMaxx/Multidimensional-time-series-with-transformer

数据集描述：

https://archive.ics.uci.edu/dataset/321/electricityloaddiagrams20112014

数据集下载：

https://drive.google.com/file/d/13FyJqP_MVVHzqQ3G0egpglelO0G-dsEa/view

运行配置

以下代码在GPU上运行成功。

import torch
import torch.nn as nn
from torch.utils.data import DataLoader, TensorDataset
import numpy as np
import random
import math
from sklearn.preprocessing import MinMaxScaler
import pandas as pd
from datetime import date
import time
import matplotlib.pyplot as plt
from matplotlib_inline import backend_inline 
backend_inline.set_matplotlib_formats('svg')
 
input_window = 100
output_window = 20 # 预测一段时间序列未来 20 个时间步的值
batch_size = 32 
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

位置编码

class PositionalEncoding(nn.Module):
    
    def __init__(self, d_model, max_len=5000):
        super(PositionalEncoding, self).__init__()       
        pe = torch.zeros(max_len, d_model)
        position = torch.arange(0, max_len, dtype=torch.float).unsqueeze(1)
        div_term = torch.exp(torch.arange(0, d_model, 2).float() * (-math.log(10000.0) / d_model))
        pe[:, 0::2] = torch.sin(position * div_term)
        pe[:, 1::2] = torch.cos(position * div_term)
        pe = pe.unsqueeze(0).transpose(0, 1)
        self.register_buffer('pe', pe)

    def forward(self, x):
        return x + self.pe[:x.size(0), :] # [input_window, batch size, embed dim]

transformer

这里的解码器用一个全连接层表示，然后再加一个全连接层得到最后输出。

class TransAm(nn.Module):
    def __init__(self, series_dim, feature_size = 80, num_layers=3, dropout=0.5):
        super(TransAm, self).__init__()
        self.model_type = 'Transformer'
        self.src_mask = None
        self.input_embedding = nn.Linear(series_dim, feature_size)
        self.pos_encoder = PositionalEncoding(feature_size)
        self.encoder_layer = nn.TransformerEncoderLayer(d_model=feature_size, nhead=10, dropout=dropout)
        self.transformer_encoder = nn.TransformerEncoder(self.encoder_layer, num_layers=num_layers)        
        self.decoder = nn.Linear(feature_size, feature_size//2)
        self.out = nn.Linear(feature_size//2, series_dim)
        self.init_weights()

    def init_weights(self):
        initrange = 0.1    
        self.decoder.bias.data.zero_()
        # 将解码器的权重初始化为均匀分布的随机值，范围在 [−0.1,0.1] 之间
        self.decoder.weight.data.uniform_(-initrange, initrange)

    def forward(self,src