The Dynamics of Exploration on Spotify

July 26, 2022 Published by Lillio Mok, Samuel F. Way, Lucas Maystre, Ashton Anderson

“Variety is the spice of life”, as the saying attributed to poet William Cowper goes. People crave heterogeneity and avoid boredom across all aspects of their online experiences. As services that provide vast amounts of content for user consumption, streaming platforms like Spotify are keen to better understand how people seek out variety to keep themselves interested and engaged. If we can figure out when and how listeners want to expand their musical tastes, then mechanisms like recommender systems could help them find novel pieces of music.

Our existing knowledge of how people find varied pieces of content is limited to self-reported, often inconsistent, evidence provided by small groups of people in narrow time slices. For instance, the scientific literature is somewhat undecided on how people look for heterogeneity over time, with two apparently incompatible ideas:

• Option 1: People “ossify” as they grow older and become less interested in variety because they already know what they like and are less easily bored. 
• Option 2: People broaden as they grow older and become more interested in variety because they’ve experienced more things and have refined palates. 

So, how do people actually consume heterogeneous content as they grow older? In this study, we investigate how a large group of Spotify listeners actively find musical variety at different points of their life cycles.

Exploration on Spotify

First of all, what does “variety” even mean? One way of measuring variety-seeking behaviours is to figure out when people are exploring individual pieces of novel content. If a person is listening to a track for the very first time on Spotify then that would be a truly novel song discovery. On the other hand, if the listener hasn’t listened to a track in a while and comes back to it later, then the track is only locally novel. To move between pieces of locally novel content across different time frames (weekly, for the purpose of this study), listeners need to turn over the tracks they’re listening to. 

Conceptual schematic of exploration at different time scales.

We measured how often people discovered and turned over musical content across a hundred thousand US-based Spotify Premium users between 2016 and 2019, a complete listening trace of over 8 billion unique listening events. 

The Offline Lifecycle: Ageing and Exploration

The first and most straightforward way of thinking about lifecycles is the trajectory along which users grow offline – i.e. their age. We compared how much and when people explored across different age groups and found three key results.

Firstly, younger listeners consistently explore less across our dataset. For the same number of tracks listened to and at different points in calendar time, they are likely to have fewer discoveries and to have lower content turnover. At a first glance, this suggests that listeners do not ossify or become narrower as they age, and in fact broaden their consumption habits as they grow.

Expected number of discoveries per stream listened to split by mean user in age group (left) and overall distribution (right). Baselines in grey (solid: each listen is a discovery; dashed: discoveries drawn from a fixed set).

Secondly, exploration never stops. People do not behave like they are drawing from a finite set of content that will satisfy them, instead continuing to accumulate novel content over time. However, this is diluted by repeat listening of previously-discovered tracks, which can be seen in the growing need to turnover content over time. People increasingly need to cycle through tracks between time frames to listen to the increasing number of tracks they discovered. Again, it does not look like people ossify and stop exploring over time.

Expected content turnover rate each week, split by age group. 

Thirdly, exploration is subject to other factors like seasonality and the way in which listeners interact with the platform. Exploration spikes around Christmas, for example, during which people may be influenced to seek out season-specific music. Additionally, users are likely to discover novel music at a consistent rate when listening to programmed music like through algorithmically-curated radio.

The Online Lifecycle: Exploratory Phases

We have looked at the offline lifecycle, but what happens across an individual user’s online lifecycle – specifically their tenure on the platform? This time, we studied how people distribute exploratory behaviours within their individual traces. Is exploration spread out and dispersed, or is it clumped together and clustered?

Illustration of how exploration can be grouped together or spread out over time.

To figure this out, we looked at how much listeners explored between time windows (again, we used a week) in their individual traces and measured the discovery rate interquartile range (IQR) over windows with multiple streams. In short, the higher the IQR, the more exploration is clustered into exploratory windows with many discoveries. We found that young users have much lower IQRs than older users, indicating that they explore more evenly than older users. In contrast, older listeners group their discoveries into clusters. 

Distribution over discovery rate IQRs between different time windows, within each user. If 25% of a user’s activity in windows is made of pure discoveries while the remaining 75% windows had 50% discoveries, their IQR would be 50%.

But what about the distribution of these windows? We found that users actually chain exploration-heavy windows together. If you already started exploring, you are more likely to keep going; if you have not started exploring yet, you are also more likely to not start in the next week. So, there seem to be phases of both content exploration and content revisitation. 

Probability of a week having many discoveries after a previous week of many discoveries (deciles). Listeners are likely to chain many discoveries together (bottom right square) or go on extended runs with few discoveries (top left square). 

Content Exploration and Taste Diversity

We’ve discussed exploration a lot so far. While it has its advantages as a measure of variety – it is granular, considers individual pieces of content, and is time-sensitive – it also does not take into account how similar two pieces of content are. A person who listens to House and Electronica music is probably not as variety-loving as someone who listens to Bluegrass and K-Pop. So, we also compared our exploration metrics to taste diversity (see here). This diversity metric measures how broad (and not necessarily novel) someone’s listening habits are, and is scaled so that 0 is very specialised and 1 is very diverse consumption.

Left: a listener’s weekly taste diversity predicts their diversity in the next week; Right: weekly taste diversity is poorly correlated with content exploration.

It turns out that diversity and exploration are practically unrelated, both at the weekly timescale and across the entire trace. By way of comparison, a user’s diversity in one week is fairly predictive of their diversity in the next week. In fact, it appears as if younger listeners are the most diverse, whereas older listeners are more specialised. Through the lens of diversity, then, there is some support for the ossification hypothesis. As listeners grow older, they appear to listen to a narrower set of content.

Listeners’ weekly taste diversity over time, split by age groups. Age ordering is reversed compared to discoveries over time.

What do these results tell us? For one, that variety is a complex construct. The different, seemingly incompatible theories about how people seek variety as they grow may be explained by different ways of satisfying heterogeneity needs. Our results are consistent with younger people being generalist repeat consumers who do not often look for very novel music compared to what they know, but the music that they do know is relatively broad. Older listeners are specialist explorers, who look for a specific and potentially ossified set of music but are also constantly discovering and turning content over. 

Furthermore, we uncovered different factors that may impact variety-seeking habits and thus listeners’ receptiveness to new music. On the one hand, exploration could depend on how people interact with programmed music, new releases, and different genres. On the other, exploration also fluctuates across seasons in the calendar, as well as in phases within individuals’ listening trajectories. These factors could potentially help online platforms identify when and how users should be guided towards varied content to meet their needs and keep them interested. More information can be found in our paper:

The Dynamics of Exploration on Spotify
Lillio Mok, Samuel F. Way, Lucas Maystre, Ashton Anderson
ICWSM 2022