Please use this identifier to cite or link to this item:
http://hdl.handle.net/2067/46350
Title: | Elliot: A Comprehensive and Rigorous Framework for Reproducible Recommender Systems Evaluation | Authors: | Anelli, Vito Walter Bellogin, Alejandro Ferrara, Antonio Malitesta, Daniele Merra, Felice Antonio Pomo, Claudio Donini, Francesco Maria Di Noia, Tommaso |
Issue Date: | 2021 | Abstract: | Recommender Systems have shown to be an effective way to alleviate the over-choice problem and provide accurate and tailored recommendations. However, the impressive number of proposed recommendation algorithms, splitting strategies, evaluation protocols, metrics, and tasks, has made rigorous experimental evaluation particularly challenging. Puzzled and frustrated by the continuous recreation of appropriate evaluation benchmarks, experimental pipelines, hyperparameter optimization, and evaluation procedures, we have developed an exhaustive framework to address such needs. Elliot is a comprehensive recommendation framework that aims to run and reproduce an entire experimental pipeline by processing a simple configuration file. The framework loads, filters, and splits the data considering a vast set of strategies (13 splitting methods and 8 filtering approaches, from temporal training-test splitting to nested K-folds Cross-Validation). Elliot(https://github.com/sisinflab/elliot) optimizes hyperparameters (51 strategies) for several recommendation algorithms (50), selects the best models, compares them with the baselines providing intra-model statistics, computes metrics (36) spanning from accuracy to beyond-accuracy, bias, and fairness, and conducts statistical analysis (Wilcoxon and Paired t-test). |
URI: | http://hdl.handle.net/2067/46350 | ISBN: | 9781450380379 | DOI: | 10.1145/3404835.3463245 |
Appears in Collections: | D1. Contributo in Atti di convegno |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
SIGIR2021_Elliot_RecSys_Framework.pdf | Green Open Access | 755.98 kB | Adobe PDF | View/Open |
SCOPUSTM
Citations
10
68
Last Week
0
0
Last month
0
0
checked on Apr 17, 2024
Page view(s)
59
Last Week
0
0
Last month
0
0
checked on Apr 20, 2024
Download(s)
28
checked on Apr 20, 2024
Google ScholarTM
Check
Altmetric
All documents in the "Unitus Open Access" community are published as open access.
All documents in the community "Prodotti della Ricerca" are restricted access unless otherwise indicated for specific documents