Scribd Logo
Upload

Welcome to Scribd!

  • 42 views

    Matrix Factorization Techniques For Recommender Systems: Collaborative Filtering

    Uploaded by

    Shashank Baravani
    This document discusses matrix factorization techniques for recommender systems. Matrix factorization involves finding lower-dimensional factor vectors that describe users and items in a way that can be used to predict unknown ratings. It addresses overfitting through regularization and includes additional parameters for biases and temporal effects to improve accuracy. The document evaluates previous research on matrix factorization and discusses open questions around applying it to datasets with different item types and explicit dependencies between items.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Matrix Factorization Techniques For Recommender Systems: Collaborative Filtering

    Uploaded by

    Shashank Baravani
    42 views20 pages
    This document discusses matrix factorization techniques for recommender systems. Matrix factorization involves finding lower-dimensional factor vectors that describe users and items in a way that can be used to predict unknown ratings. It addresses overfitting through regularization and includes additional parameters for biases and temporal effects to improve accuracy. The document evaluates previous research on matrix factorization and discusses open questions around applying it to datasets with different item types and explicit dependencies between items.

    Original Title

    pres1-matrixfactorization

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses matrix factorization techniques for recommender systems. Matrix factorization involves finding lower-dimensional factor vectors that describe users and items in a way that can be used to predict unknown ratings. It addresses overfitting through regularization and includes additional parameters for biases and temporal effects to improve accuracy. The document evaluates previous research on matrix factorization and discusses open questions around applying it to datasets with different item types and explicit dependencies between items.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    42 views20 pages

    Matrix Factorization Techniques For Recommender Systems: Collaborative Filtering

    Uploaded by

    Shashank Baravani
    This document discusses matrix factorization techniques for recommender systems. Matrix factorization involves finding lower-dimensional factor vectors that describe users and items in a way that can be used to predict unknown ratings. It addresses overfitting through regularization and includes additional parameters for biases and temporal effects to improve accuracy. The document evaluates previous research on matrix factorization and discusses open questions around applying it to datasets with different item types and explicit dependencies between items.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 20

    Matrix Factorization Techniques

    For Recommender Systems

    Collaborative Filtering

    Markus Freitag, Jan-Felix Schwarz 28 April 2011


    Agenda

    1.  Paper Backgrounds


    2.  Latent Factor Models
    3.  Overfitting & Regularization
    4.  Learning Algorithms
    5.  Biases & Temporal Dynamics
    6.  Paper Evaluation
    7.  Application For KDD Cup
    8.  Discussion

    Matrix Factorization Techniques For Recommender Systems


    Paper Backgrounds

    Yehuda Koren, Yahoo Research

    Robert Bell and Chris Volinsky,


    AT&T Labs-Research

    Paper published in August 2009

    Authors won the grand Netflix Prize


    in September 2009

    Matrix Factorization Techniques For Recommender Systems


    Latent Factor Models

    ■  Find features that describe the characteristics of rated objects


    ■  Item characteristics and user preferences are described with
    numerical factor values
    ■  Assumption: Ratings can be inferred from a model put together
    from a smaller number of parameters

    0
    Drama | Comedy

    Matrix Factorization Techniques For Recommender Systems


    Latent Factor Models

    ■  Items and users are associated with a factor vector


    ■  Dot product captures the user’s estimated interest in the item:

    ■  Challenge: How to compute a mapping of items and users to


    factor vectors?

    ■  Approaches:
    □  Singular Value Decomposition (SVD)
    □  Matrix Factorization

    Matrix Factorization Techniques For Recommender Systems


    Singular Value Decomposition

    6 User Feature Matrix (F x N)

    f1 1 -4 1
    Rating Matrix (N x M)
    f2 -2 0 -3

    5 3 5 f3 0 -5 1

    4 2 1
    Movie Feature Matrix (F x M)
    0 3 3

    f1 -1 0 -2

    f2 4 -4 1

    f3 0 2 2
    Matrix Factorization Techniques For Recommender Systems
    Singular Value Decomposition

    .
    =

    Matrix Factorization Techniques For Recommender Systems


    SVD - Problems

    ■  Conventional SVD is undefined for incomplete matrices!

    ■  Imputation to fill in missing values


    □  Increases the amount of data
    □  “SVD of ginormous matrices is... well, no fun“ (Simon Funk)

    ■  We need an approach that can simply ignore missing ratings

    Matrix Factorization Techniques For Recommender Systems


    Matrix Factorization

    : known rating of user u for item i


    remember:
    predicted rating

    Matrix Factorization Techniques For Recommender Systems


    Matrix Factorization - Overfitting

    10

    A model is built to represent the training data – not to reproduce


    the training data.

    Matrix Factorization Techniques For Recommender Systems


    Matrix Factorization - Regularization

    11

    Idea: penalize complexity

    : constant to control the extend of regularization


     determined by cross-validation

    Matrix Factorization Techniques For Recommender Systems


    Learning Algorithms

    ■  Stochastic gradient descent


    □  Modification of parameters (qi, pu) relative to prediction error
    □  Recommended algorithm

    ■  Alternating least squares


    □  Allows massive parallelization
    □  Better for densely filled matrices

    Matrix Factorization Techniques For Recommender Systems


    Learning Algorithms

    ■  Calculation of the prediction error


    □  Error = actual rating – predicted rating
    □ 

    ■  Modification
    □  By magnitude proportional to γ
    □  In the opposite direction of the gradient
    □ 

    Matrix Factorization Techniques For Recommender Systems


    Biases

    ■  Item or user specific rating variations are called biases


    ■  Example:
    □  Alice rates no movie with more than 2 (out of 5)
    □  Movie X is hyped and rated with 5 only
    ■  Matrix factorization allows modeling of biases

    ■  Including bias parameters in the prediction:

    Matrix Factorization Techniques For Recommender Systems


    Temporal Dynamics

    ■  Ratings may be affected by temporal effects


    □  Popularity of an item may change
    □  User’s identity and preferences may change
    ■  Modeling temporal affects can improve accuracy significantly

    ■  Rating predictions as a function of time:

    Matrix Factorization Techniques For Recommender Systems


    Paper Evaluation

    ■  High-level overview of matrix factorization techniques

    ■  Mathematical foundations are pointed out but not elaborated

    ■  Many useful references to related work

    ■  Authors do not reveal their secret implementation tidbits

    Matrix Factorization Techniques For Recommender Systems


    Application For KDD Cup

    ■  Different item types


    □  We must assume different prediction models!
    □  We have explicit dependencies between items

    ■  How to apply Matrix Factorization to the KDD data set?


    □  Segment training data in separate sets for each type
    □  Consider ratings for dependent items to make a prediction

    Matrix Factorization Techniques For Recommender Systems


    Application For KDD Cup - Hypotheses

    ■  Users change their taste in music

    ■  Users tend to rate new songs better


    □  Users get tired of songs

    ■  Some artists and albums are hyped for a while

    ■  Evergreens
    □  Loved by many, but maybe also hated by some

    ■  ...

    Matrix Factorization Techniques For Recommender Systems


    References

    [1] C. Volinsky et al.: “Matrix Factorization Techniques for Recommender Systems“ In:
    IEEE Computer, Vol. 42 (2009) , pp. 30-37 .

    [2] Simon Funk on the Stochastic Gradient Descent algorithm: http://sifter.org/~simon/


    journal/20061211.html (Dec 2006)

    [3] Y.Koren et al.: “Collaborative filtering with temporal dynamics“ In: Proceedings of
    the 15th ACM SIGKDD international conference on Knowledge discovery and data
    mining (2009), pp. 447-456.

    [4] Chih-Chao Ma: “A Guide to Singular Value Decomposition for Collaborative Filtering“
    In: csientuedutw (2008)

    [5] Y.F. Hu, Y. Koren, C. Volinsky: “Collaborative Filtering for Implicit Feedback
    Datasets“ In: Proc. IEEE Int‘l Conf. Data Mining (2008), pp. 263-372

    Matrix Factorization Techniques For Recommender Systems


    Discussion

    Summary
    ■  Matrix factorization is a promising approach for collaborative
    filtering
    ■  Factor vectors are learned by minimizing the RSME
    ■  Regularization to prevent overfitting
    ■  Addition of bias parameters and temporal dynamics further
    improve accuracy

    Outlook
    ■  Develop strategies for applying matrix factorization on our data
    set with different item types
    ■  Make use of the available dependencies between items
    ■  Explorer biases and rating behaviors specific for our music domain
    Matrix Factorization Techniques For Recommender Systems

    You might also like