Please use this identifier to cite or link to this item:
Title: How to perform reproducible experiments in the ELLIOT recommendation framework: Data processing, model selection, and performance evaluation
Authors: Anelli, Vito Walter
Bellogín, Alejandro
Ferrara, Antonio
Malitesta, Daniele
Merra, Felice Antonio
Pomo, Claudio
Donini, Francesco Maria 
Di Sciascio, Eugenio
Di Noia, Tommaso
Issue Date: 2021
Recommender Systems have shown to be an effective way to alleviate the over-choice problem and provide accurate and tailored recommendations. However, the impressive number of proposed recommendation algorithms, splitting strategies, evaluation protocols, metrics, and tasks, has made rigorous experimental evaluation particularly challenging. ELLIOT is a comprehensive recommendation framework that aims to run and reproduce an entire experimental pipeline by processing a simple configuration file. The framework loads, filters, and splits the data considering a vast set of strategies. Then, it optimizes hyperparameters for several recommendation algorithms, selects the best models, compares them with the baselines, computes metrics spanning from accuracy to beyond-accuracy, bias, and fairness, and conducts statistical analysis. The aim is to provide researchers a tool to ease all the experimental evaluation phases (and make them reproducible), from data reading to results collection. ELLIOT is freely available on GitHub at
Rights: Attribution-NonCommercial-NoDerivatives 4.0 International
Appears in Collections:D1. Contributo in Atti di convegno

Files in This Item:
File Description SizeFormat
paper14.pdfPublished paper488.89 kBAdobe PDFView/Open
Show full item record

Page view(s)

Last Week
Last month
checked on May 17, 2022


checked on May 17, 2022

Google ScholarTM


This item is licensed under a Creative Commons License Creative Commons