Evaluating Recommendation Algorithms by Graph Analysis

Title: Evaluating Recommendation Algorithms by Graph Analysis

Main Research Question: How can we evaluate recommendation algorithms by analyzing the implicit graph underlying the recommender dataset?

Methodology:

1. We present a novel framework for evaluating recommendation algorithms based on the 'jumps' they make to connect people to artifacts. This approach emphasizes reachability within the implicit graph structure and serves as a complement to evaluation in terms of predictive accuracy. 2. We illustrate the approach with a common jump called the 'hammock' using movie recommender datasets.

Results:

1. The framework allows us to consider questions related to algorithmic parameters and properties of the datasets. For instance, what is the average path length from a person to an artifact, or what choices of minimum ratings and jumps maintain a connected graph? 2. We found that the 'hammock' jump can be an effective way to connect people to artifacts, especially in cases where the data is sparse or the artifacts are highly interconnected.

Implications:

1. This approach provides a new way to evaluate recommendation algorithms that goes beyond predictive accuracy. It allows us to understand the connectivity and reachability within the implicit graph underlying the recommender dataset. 2. The framework can be applied to any recommender system where people and artifacts can be represented as nodes in a graph, and where recommendations can be seen as connections between these nodes. 3. The results can inform the design of more effective recommendation algorithms and the development of personalized information systems.

Link to Article: https://arxiv.org/abs/0104009v1 Authors: arXiv ID: 0104009v1