Exploring Goal-oriented Podcast Recommendations

March 23, 2023 Published by Yu Liang, Aditya Ponnada, Paul Lamere, Nediyana Daskalova

Goal-setting in recommender systems

Recommender systems typically look to users’ past consumption to predict what they may want next. In practice, this approach tends to work best when what the user wants is similar to what they have consumed recently, and when it is relatively easy for that person to evaluate new items. On the other hand, this approach works less well when what the user wants is something different and potentially something that is not captured by their past consumption [1]. This issue is made more difficult still when the recommended items take some time to evaluate, as is the case for things like movies, books, and — the focus of this study — podcasts. 

So how might recommender systems adapt to address these shortcomings? In a new study, we explore an approach that puts users in control by giving them the ability to specify their listening goals and receive recommendations designed to help meet those goals. To do so, we created an interactive research prototype, called “GoalPods,” and evaluated it with avid podcast listeners, seeking answers to two specific research questions:

RQ1: What (if any) goals do users have for multimedia content consumption and what do these goals entail?

RQ2: How does goal-focused multimedia consumption change the way users interact with personalized recommendations?

Methodology

We conducted our study in two phases. First, we conducted a large-scale survey to learn about the various goals that listeners would like to accomplish via podcast listening (to answer RQ1). Second, we used the information from the survey to build an interactive prototype and assess how users would go about setting goals that they wish to achieve by listening to podcasts (to answer RQ2).

Podcast goals survey

To address the first question — RQ1: What (if any) goals do users have for multimedia content consumption and what do they entail? — our sample of users received an in-situ survey on the Spotify platform. The survey was triggered when users visited a podcast episode page, to ensure that only recent podcast listeners on the platform receive it. The survey consisted of two questions asking about (1) users’ immediate intent for listening to the podcast episode that triggered the survey and (2) any long-term goals that they wish to achieve by listening to podcasts.

We collected responses from 68K premium account subscribers in the US. From the responses, we observed that even when users had a long-term goal (e.g., to learn something new), they tended to instead be listening to podcasts that met their immediate needs (e.g. to entertain themselves). This highlights a gap between what users aspire to achieve and what they typically consume, pointing to a clear opportunity for recommender systems to better assist with goal-focused consumption.

GoalPods overview

To answer our second research question — (RQ2): How does goal-focused multimedia consumption change the way users interact with personalized recommendations? — we incorporated the learnings from the survey to construct our prototype, GoalPods. 

The interactive prototype allowed users to specify listening goals and then receive personalized podcast recommendations catered to those goals. The interface contained four key components, visually separated into columns, corresponding to each of the following user actions: 

• Review past 90-day podcast consumption [3] (Panel A)
• Set weekly listening goals (e.g., learn something; Panel B)
  •  In this example, the user has selected to learn from past topics and selected the “Business & Technology” category, along with “Investing” and “Personal Finance” as sub-categories
• Explore recommended podcast episodes via a ranked list personalized to the user (Panel C)
• Create a playlist of podcast episodes to match that goal (Panel D)
  • In this example, the user has selected four episodes amounting to 3.3 hours of listening to be added to his/her Spotify library.

Evaluation

In the second part of our study, we explored how users interacted with the interactive prototype to consume goal-focused recommendations over the course of one week. After one week, users were interviewed to learn about their experience of listening to the podcast episode from their goal playlists.

Findings

Our qualitative analysis revealed four main themes.

Theme 1: Two types of user goals: low-involvement (e.g. “relieve boredom”) and high-involvement (e.g. “learn something new”) goals. 

We identified two kinds of goals that users had for listening to podcasts. First, their high-involvement goals were life goals that users felt podcasts could help them with, such as self-improvement, knowledge, and learning. They were often geared towards personal development and using podcasts as a means to gain knowledge that can be applied in the real world or shared with others. Low-involvement goals, on the other hand, were centered around immediate requirements of entertainment or catching up on interests. These goals often involved keeping up with the times and keeping one-self entertained while doing other things. 

Theme 2: Users need more structure and support to set high-involvement goals.

We found that users found it easy to identify relevant recommendations for their low-involvement goals. They often focused on familiar content to decrease the cognitive load required to choose goal-relevant content to listen to. However, when it came to high-involvement goals, users needed more scaffolding and support. Since those goals require higher cognitive effort, recommender systems could help users achieve them by lowering the barrier to discovering short cognitively-heavy podcasts.

Theme 3: Setting goals helped users discover content outside their filter bubbles

By anchoring on users’ personal goals to explore recommendations, the interactive prototype (and goal-focused podcast consumption) led to insightful content discovery outside the users’ filter bubbles. Overall, personalization was most useful for the low-involvement goals as it helped identify familiar and similar content to the user’s listening history. However, in the context of the high-involvement goals, users aimed to find drastically different content, and the current recommender system was not optimized for that use case. 

We find that our tool for goal-setting helped users to discover content in two main ways: (1) it helped users get into the mindset of being open to new content and elicit their desires for branching out of their usual habits, and (2) the structured setup of the tool helped surface novel content in a convenient way, supporting the need for recommender systems that drive discovery. As one participant summarized, “it’s kind of hard to know where to start. And so I think this feature gives you a good place to be like, okay, this is what I want to do for this week. And this is how I can do it. And here are some options for what I can listen to instead of just scrolling through a tab for like 10 minutes, trying to find one thing you might find interesting. So I think that was very helpful.”

Theme 4: Contextual cues and descriptors were necessary to help users pick goal-relevant content.

Regardless of the cognitive effort required to follow through on a goal, we found some signals that helped users identify what content they would like to listen to. These signals are important cues for future recommender systems for podcasts and other audio content to focus on.

What does this mean for future recommender systems?

We developed GoalPods as a research prototype to probe users on their podcast-related goal setting, recommendation evaluation, and goal-focused episode consumption. While there are no plans to integrate this prototype into the Spotify experience, some important lessons were pulled to inform future product iterations. Our study highlights a few important design implications for recommender systems, especially where the level of investment on part of the user is higher, such as movies and audiobooks, in addition to podcasts. We saw that users tended to plan more high-involvement goals for the long term (such as learning and gaining new knowledge/perspectives), while their instant listening intent was centered around low-involvement goals (such as being entertained). This emphasizes the importance of understanding the user’s underlying motives and aspirations for listening to a particular podcast in addition to focusing on past consumption behavior. 

More details about this work can be found in our paper:
Enabling Goal-Focused Exploration of Podcasts in Interactive Recommender Systems
Yu Liang, Aditya Ponnada, Paul Lamere, Nediyana Daskalova
IUI 2023

References

[1] Michael D. Ekstrand and Martijn C. Willemsen. 2016. Behaviorism is Not Enough: Better Recommendations through Listening to Users. RecSys 2016.
[2] Zahra Nazari, Praveen Chandar, Ghazal Fazelnia, Catherine M. Edwards, Benjamin Carterette, and Mounia Lalmas. Choice of Implicit Signal Matters: Accounting for User Aspirations in Podcast Recommendations. The Web Conference 2022
[3] S. A. Munson and S. Consolvo, “Exploring goal-setting, rewards, self-monitoring, and sharing to motivate physical activity,” PervasiveHealth 2012