Windowed Multi-Item-Query Attention for Long-Sequence Recommendation

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


MPhil Thesis Defence


Title: "Windowed Multi-Item-Query Attention for Long-Sequence Recommendation"

By

Mr. Mingshi XU


Abstract:

Sequential recommendation aims to predict a user's next interaction from their
historical behaviour. Transformer-based sequential recommenders commonly
construct the attention query for each position using only the representation of
the current item. However, a user's immediate intent may be reflected jointly by
several recent interactions rather than by the most recent item alone.

This thesis investigates Windowed Multi-Item-Query Attention, a method that
constructs attention queries from a short window of recent item representations
while retaining the causal attention structure of Transformer-based
recommenders. The proposed approach is evaluated on sequential recommendation
models, including SASRec and S3Rec, under full-catalogue next-item ranking.
Experimental results examine its effectiveness across different datasets and
analyse the conditions under which incorporating multiple recent interactions
provides benefits over conventional single-item-query attention. The study
further discusses the robustness of the method and the dataset-dependent role of
recent contextual information in sequential recommendation.


Date:                   Tuesday, 17 November 2026

Time:                   1:00pm - 3:00pm

Venue:                  Room 3494
                        Lifts 25/26

Chairman:               Prof. Raymond WONG

Committee Members:      Dr. Wilfred NG (Supervisor)
                        Dr. Yangqiu SONG