Jointly Leveraging Intent and Interaction Signals to Predict User Satisfaction with Slate Recommendations

Abstract

Detecting and understanding implicit measures of user satisfaction are essential for enhancing recommendation quality. When users interact with a recommendation system, they leave behind fine grained traces of interaction signals, which contain valuable infor- mation that could help gauging user satisfaction. User interaction with such systems is often motivated by a specific need or intent, of- ten not explicitly specified by the user, but can nevertheless inform on how the user interacts with, and the extent to which the user is satisfied by the recommendations served. In this work, we consider a complex recommendation scenario, called Slate Recommendation, wherein a user is presented with an ordered set of collections, called slates, in a specific page layout. We focus on the context of mu- sic streaming and leverage fine-grained user interaction signals to tackle the problem of predicting user satisfaction. We hypothesize that user interactions are conditional on the specific intent users have when interacting with a recommenda- tion system, and highlight the need for explicitly considering user intent when interpreting interaction signals. We present diverse approaches to identify user intents (interviews, surveys and a quan- titative approach) and identify a set of common intents users have in a music streaming recommendation setting. Additionally, we iden- tify the importance of shared learning across intents and propose a multi-level hierarchical model for user satisfaction prediction that leverages user intent information alongside interaction signals. Our findings from extensive experiments on a large scale real world data demonstrate (i) the utility of considering different interaction sig- nals, (ii) the role of intents in interpreting user interactions and (iii) the interplay between interaction signals and intents in predicting user satisfaction.

Related

July 2021 | ACL

Modeling Language Usage and Listener Engagement in Podcasts

Sravana Reddy, Mariya Lazarova, Yongze Yu, Rosie Jones

July 2021 | SIGIR

Podcast Metadata and Content: Episode Relevance and Attractiveness in Ad Hoc Search

Ben Carterette, Rosie Jones, Gareth Jones, Maria Eskevich, Sravana Reddy, Ann Clifton, Yongze Yu, Jussi Karlgren and Ian Soboroff

July 2021 | SIGIR

Current Challenges and Future Directions in Podcast Information Access

Rosie Jones, Hamed Zamani, Markus Schedl, Ching-Wei Chen, Sravana Reddy, Ann Clifton, Jussi Karlgren, Helia Hashemi, Aasish Pappu, Zahra Nazari, LongQi Yang, Oguz Semerci, Hugues Bouchard, Ben Carterette