Challenges in Translating Research to Practice for Evaluating Fairness & Bias in Recommendation Systems

Abstract

Calls to action to implement evaluation of fairness and bias into industry systems are increasing at a rapid rate. The research community has attempted to meet these demands by producing ethical principles and guidelines for AI, but few of these documents provide guidance on how to implement these principles in real world settings. Without readily available standardized and practice-tested approaches for evaluating fairness in recommendation systems, industry practitioners, who are often not experts, may easily run into challenges or implement metrics that are potentially poorly suited to their specific applications. When evaluating recommendations, practitioners are well aware they should evaluate their systems for unintended algorithmic harms, but the most important, and unanswered question, is how? In this talk, we will present practical challenges we encountered in addressing algorithmic responsibility in recommendation systems, which also present research opportunities for the RecSys community. This talk will focus on the steps that need to happen before bias mitigation can even begin.

Related

March 2023 | Frontier on Big Data: Recommender Systems

A Survey on Multi-objective Recommender Systems

Dietmar Jannach and Himan Abdollahpouri

March 2023 | Intelligent User Interfaces (IUI)

Enabling Goal-Focused Exploration of Podcasts in Interactive Recommender Systems

Yu Liang, Aditya Ponnada, Paul Lamere, Nediyana Daskalova

September 2022 | RecSys

Identifying New Podcasts with High General Appeal Using a Pure Exploration Infinitely-Armed Bandit Strategy

Maryam Aziz, Jesse Anderton, Kevin Jamieson, Alice Wang, Hugues Bouchard, Javed Aslam