Jointly Leveraging Intent and Interaction Signals to Predict User Satisfaction with Slate Recommendations


Detecting and understanding implicit measures of user satisfaction are essential for enhancing recommendation quality. When users interact with a recommendation system, they leave behind fine grained traces of interaction signals, which contain valuable infor- mation that could help gauging user satisfaction. User interaction with such systems is often motivated by a specific need or intent, of- ten not explicitly specified by the user, but can nevertheless inform on how the user interacts with, and the extent to which the user is satisfied by the recommendations served. In this work, we consider a complex recommendation scenario, called Slate Recommendation, wherein a user is presented with an ordered set of collections, called slates, in a specific page layout. We focus on the context of mu- sic streaming and leverage fine-grained user interaction signals to tackle the problem of predicting user satisfaction. We hypothesize that user interactions are conditional on the specific intent users have when interacting with a recommenda- tion system, and highlight the need for explicitly considering user intent when interpreting interaction signals. We present diverse approaches to identify user intents (interviews, surveys and a quan- titative approach) and identify a set of common intents users have in a music streaming recommendation setting. Additionally, we iden- tify the importance of shared learning across intents and propose a multi-level hierarchical model for user satisfaction prediction that leverages user intent information alongside interaction signals. Our findings from extensive experiments on a large scale real world data demonstrate (i) the utility of considering different interaction sig- nals, (ii) the role of intents in interpreting user interactions and (iii) the interplay between interaction signals and intents in predicting user satisfaction.


August 2020 | KDD

Bandit based Optimization of Multiple Objectives on a Music Streaming Platform

Rishabh Mehrotra, Niannan Xue, Mounia Lalmas

August 2020 | KDD

Advances in Recommender Systems: From Multi-stakeholder Marketplaces to Automated RecSys

Rishabh Mehrotra, Ben Carterette, Yong Li, Quanming Yao, James Tin-Yau Kwok, Isabelle Guyon, Qiang Yang

August 2020 | KDD

Counterfactual Evaluation of Slate Recommendations with Sequential Reward Interactions

Praveen Chandar, James McInerney, Brian Brost, Rishabh Mehrotra, Benjamin Carterette