Interpretable Trend-seasonality Pattern of Transformer in Time Series Forecasting

MPhil Thesis Defence


Title: "Interpretable Trend-seasonality Pattern of Transformer in Time 
Series Forecasting"

By

Mr. Yaowei HUANG


Abstract

In this thesis, we propose an interpretable transformer being able to 
visualize the trend and seasonality patterns separately. Our model uses 
learned positional embedding as the trend patterns and learned temporal 
embedding as the seasonality patterns. We compute these two patterns 
separately with individual self-attention modules added afterward. In 
previous work based on the Transformer, the learned positional embedding 
and fixed temporal encoding are concatenated together into a 
self-attention module. However, this operation may bring noise since it 
adds heterogeneous vectors bearing different information. Unlike previous 
work, we use the learned temporal embedding instead of fixed temporal 
encoding to extract the temporal information, we model and visualize the 
trend and seasonality patterns separately for better performance and 
practical applications in the real world. Experiments on the real world 
datasets show that our model gives interpretable results with most 
state-of-the-art performances.


Date:  			Thursday, 25 November 2021

Time:			9:00am - 11:00am

Zoom meeting:		https://hkust.zoom.com.cn/j/9125182610

Committee Members:	Prof. Tong Zhang (Supervisor)
 			Prof. Xiaofang Zhou (Chairperson)
 			Prof. James Kwok


**** ALL are Welcome ****