Exploiting Co-occurrence for Implicit Feedback Recommendation

PhD Thesis Proposal Defence


Title: "Exploiting Co-occurrence for Implicit Feedback Recommendation"

by

Mr. Farhan KHAWAR


Abstract:

Recommender systems serve as bridges between users and items by 
recommending items to users that they might find interesting. 
Collaborative filtering (CF) is a technique commonly used in recommender 
systems. It predicts a user's preference for an item based on past 
user-item interactions. These user-item interactions, called feedback, are 
of two types: explicit and implicit. In explicit feedback, a user is asked 
to explicitly indicate their preference for the items they have consumed. 
However, this requires additional effort and cooperation from the users 
and is often inflicted with user biases. An alternative is to use implicit 
feedback in which we record the user consumption behavior when they 
interact with items. Consumption may refer to the user clicking, buying, 
or watching an item. Implicit feedback it is the simplest form of user 
feedback that can be used for item recommendation. Moreover, it is easy to 
collect and is domain independent.

The simplicity of implicit feedback brings the challenge of the sparseness 
of the signal. Specifically, it is positive-only feedback since it only 
contains the positive signal of a user consuming an item. Unlike explicit 
feedback, it does not possess any negative signal to show a user's 
dislike. With such data, a valuable piece of information that can be used 
for making recommendations is the co-occurrence of items or users; that 
is, two items being co-consumed by users or two users co-consuming the 
same item.

In this thesis, we explore the role of co-occurrence in implicit feedback 
recommendation. In the first part, we show that efficient co-occurrence 
estimation can lead to improved recommendations by two popular 
recommenders. Firstly, we show that the memory-based recommenders rely on 
co-occurrence estimation but due to the finite sample size, this 
estimation is noisy. Using insights from Marchenko–Pastur law we remove 
this noise by clipping small eigenvalues of the co-occurrence matrix. 
Also, we can shrink the largest eigenvalue to remove the "global" effects 
of the system. Both these cleaning strategies lead to better co-occurrence 
estimation, and this is translated into more accurate and diverse 
recommendations. Secondly, we show that matrix factorization based 
recommenders can be seen as simultaneously cleaning the user and item 
co-occurrence matrices by performing eigenvalues clipping. In addition, 
suppressing the largest eigenvalue also results in more diverse 
recommendations and decreased popularity bias.

In the second part, we introduce methods that further exploit the 
co-occurrence information by building models on top of the item 
co-occurrence. We introduce the notion of multi-dimensional user 
clustering, where each dimension is a group of co-occurring items. These 
co-occurring items represent the users' tendency to consume these items 
together and thus define a latent "taste". For each such latent taste, we 
cluster all the users into two groups: those that have a preference to 
consume these co-occurring items and those who don't. We present two 
methods to perform this multi-dimensional user clustering. Unlike existing 
latent vector methods, the resulting models learn interpretable latent 
dimensions that lend themselves easily for explanations. In addition, they 
exhibit a better warm and cold start performance.

In the third part, we introduce structure learning for deep learning based 
implicit feedback recommenders. We use the item co-occurrence to learn the 
structure of auto-encoder based recommenders. We first find overlapping 
item groups based on item co-occurrence. These overlapping groups are then 
used as the skeleton of the structure for the encoder and decoder of an 
auto-encoder. The resulting sparse structure can be seen as a structural 
prior for network training and it guides the parameter estimation. This 
leads to improved performance over state-of-the-art deep-learning based 
recommenders due to a smaller spectral norm of the weight matrices and 
hence a better generalization performance. In addition, the structure aids 
in better cold start performance.

Finally, we explore the case when additional features information is also 
available with implicit feedback. When a user consumes an item we can 
treat their features as co-occurring. However, existing methods model all 
feature co-interactions. Moreover, they model each of these feature 
co-occurrences using the same function. Since all feature co-occurrences 
are not relevant and every feature co-occurrence has a varying complexity 
of feature interaction, we propose a neural architecture search based 
approach to search for which feature interactions to model and to what 
degree to model these interactions. The results show that this approach 
outperforms state-of-the-art feature interaction based recommenders using 
a fraction of the parameters and flops and it learns meaningful feature 
co-occurrences.


Date:			Thursday, 14 November 2019

Time:                  	3:00pm - 5:00pm

Venue:                  Room 5501
                         lifts 25/26

Committee Members:	Prof. Nevin Zhang (Supervisor)
 			Prof. Dik-Lun Lee (Chairperson)
 			Dr. Yangqiu Song
 			Dr. Raymond Wong


**** ALL are Welcome ****