Hypothesis-Driven Shelf Generation for Personalised Recommendation

TL;DR: We investigate whether personalised shelf concepts can be generated from listener profiles and grounded in Spotify's catalogue, rather than selected from a fixed inventory of hand-designed templates. The pipeline runs entirely offline, with no language-model inference at serving time. Early evaluation under randomised exposure on Spotify Home shows promising results for some content types.

Shelves as recommendation units

Spotify Home organizes recommendations into shelves: themed horizontal rows such as "More of What You Like" or "New Releases for You." A shelf is not just a ranked list of items. It is a recommendation unit where the title frames the entire row, telling the listener why these items belong together and why they might be relevant.

In most production systems, creating a new type of shelf means defining a template and building dedicated retrieval logic. This works well for broad, recurring intents. But individual taste often involves narrow intersections of genre, era, mood, scene, and familiarity that are impractical to maintain as reusable templates. The long tail of listener preferences outpaces any hand-curated shelf inventory.

In this work, we ask whether shelf concepts themselves can be generated rather than selected from a fixed set, and whether those generated concepts can be grounded in real catalogue items at production scale.

Figure 1. An illustrative example of a shelf hypothesis derived from intersecting listening preferences. The example is conceptual and does not represent a specific listener or production output.

The core idea: separating planning from fulfilment

The central architectural decision is a decomposition between two problems that production shelf systems typically conflate:

1. Planning: What shelf would make sense for this listener?

2. Fulfillment: Which real catalog items best bring that shelf concept to life?

The interface between these stages is a shelf hypothesis: a structured representation combining a natural-language description of the shelf concept with the metadata needed for retrieval: a target content type, a familiarity level (familiar vs. discovery), optional routing constraints (market, freshness), and provisional title and subtitle. 

A listener with strong affinities for Nordic ambient, post-rock, and modern classical might receive a hypothesis such as “glacial ambient post-rock with orchestral textures.” This is not a query the listener writes. The system infers it from behavioural signals and expresses it in a form that a downstream fulfilment model can act on.

This separation has practical consequences. The planning model can focus on understanding taste and generating useful concepts. The fulfilment model can focus on retrieving valid catalogue items. Each stage can be evaluated, improved, and swapped independently.

Figure 2. The four-stage shelf-generation architecture from the paper. Full listener-profile information is used during hypothesis generation; downstream stages operate on the generated hypothesis, applicable constraints, and catalog items.

Four steps from an idea to shelf

Stage 1: Hypothesis generation

The first stage turns a listener profile into a small set of candidate shelf hypotheses. Inputs include recent listening, longer-term affinities, market context, familiar content, and podcast engagement. Each hypothesis specifies not just a broad genre but a particular intersection of attributes (e.g., style, era, mood, artist neighbourhood) along with the content type and familiarity orientation. Early development used a large proprietary LLM. For production, we distilled this into a compact open-source model. Offline evaluation showed effectively identical hypothesis quality between the two, supporting the use of the smaller model at scale.

Stage 2: Catalogue fulfilment

A hypothesis is only useful if it can be grounded in valid catalogue items. We use generative retrieval for this: rather than scoring all candidates, the model generates Semantic IDs (i.e.,  compact discrete identifiers learned over catalogue entities) which are resolved into Spotify items. Decoding is constrained so that generated identifiers always correspond to valid entities of the appropriate type. The fulfilment model does not receive the listener's full profile. It works from the generated hypothesis and its constraints. Personalisation has already occurred during planning; fulfilment solves the narrower problem of realising a concept with actual catalogue items.

Stage 3: Candidate selection and shelf alignment

Retrieving plausible items is not the same as producing a coherent shelf. A candidate list might contain individually relevant albums while the shelf title mischaracterises the retrieved set. Shelf quality is fundamentally set-level: even reasonable items can fail as a row if they don't cohere under the displayed title. The alignment stage addresses this. Given the broader candidate set and enriched item metadata, an LLM selects the final items and rewrites the title and subtitle to reflect what will actually appear, thus treating the shelf as a single recommendation unit rather than a collection of independent items.

Stage 4: Offline serving

All upstream stages run offline. The finished shelves are supplied to Spotify Home as additional ranking candidates, and not pinned placements, competing with existing shelves through the platform's standard ranking mechanisms.

Evaluation approach

Evaluating generated shelves poses a different problem from evaluating conventional ranked lists. There is no historically labelled “correct” answer for a newly imagined shelf concept, and item-level relevance alone does not capture whether a complete row is coherent or whether its title is accurate. We use two LLM judges as directional offline signals, mirroring the planning/fulfilment decomposition. 

• A User-to-Hypothesis Judge evaluates whether a generated hypothesis fits the listener's observed interests: is it grounded in their history, specific enough to constrain retrieval, and does it offer useful discovery? 

• A Hypothesis-to-Shelf Judge evaluates whether the retrieved items actually realise the hypothesis: do they cohere as a set, and does the title accurately describe what appears? 

Both use a 0–2 ordinal scale. These judges are not validated against human agreement in this work; they serve as directional signals, while online behaviour is the final measure.

Results

Generative retrieval vs. baselines

We compared generative retrieval against BM25, dense embedding retrieval, and a hybrid of the two on the same 10,000 shelf hypotheses from 1,000 users. Generative retrieval achieved an overall Hypothesis-to-Shelf Judge score of 0.71, compared with 0.56 for the strongest baseline (approximately 27% higher on this 0–2 scale). It scored higher across every reported dimension, and the differences remained significant after Bonferroni correction.

Why does this matter? Narrowly defined shelf concepts can depend on relationships not stated directly in item metadata: overlapping artistic scenes, stylistic associations, or attribute combinations that no single catalogue description names explicitly. Generative retrieval helps when fulfilling a hypothesis requires connecting these more indirect signals.

Effect of shelf alignment

Consider a shelf titled “Ambient Post-Rock for Late Nights” whose items are mostly upbeat electronic playlists. Even if individual recommendations are reasonable, the shelf has failed: the items do not match what the title led the listener to expect. The alignment stage exists to close this gap.

We compared shelves after catalogue fulfilment with shelves after alignment using the Hypothesis-to-Shelf Judge. The two cohorts were non-overlapping (10,000 pre-alignment, over 16,000 post-alignment), so comparisons use independent-samples t-tests with Bonferroni correction. Alignment increased the overall score from 0.71 to 1.27 (+78%). The largest gain appeared in title-promise fulfilment, which increased from 0.66 to 1.31 (+99%). Shelf coherence, item relevance, and style match also improved substantially.

Figure 3. Offline LLM-judge scores for overall shelf quality and title-promise fulfilment, before and after alignment. The two groups are separate, non-overlapping cohorts; these differences should not be interpreted as a paired causal estimate.

Early online signals

Offline evaluations assess whether a shelf concept makes sense and whether its contents match its title. They do not establish how listeners respond. To examine early online behaviour, we compare hypothesis-driven shelves with existing shelf families under uniform random exposure on Spotify Home, a protocol where shelf ordering is randomised independently of the production ranker.

The outcome is the share of shelf impressions leading to at least 30 seconds of streaming. The table compares the strongest hypothesis-driven shelf with the strongest existing comparator within each content-type pool; uncertainty values are 95% bootstrap confidence-interval half-widths.

These comparisons use randomised exposure to level the playing field between shelf families, but they are not a causal A/B test of overall Home impact. With that caveat, hypothesis-driven shelves are competitive for albums, while the weaker spoken-word and playlist results indicate clear room for improvement.

What is next

The core contribution is using a structured natural-language hypothesis as the interface between listener-specific planning and catalogue fulfilment. This decomposition lets the system propose shelf concepts beyond a fixed template inventory, evaluate each stage independently, and integrate the results into an existing recommendation surface without request-time inference.

Several questions remain open: evaluating these shelves under production ranking, improving spoken-word shelf generation, tighter coordination between planning and fulfilment, and moving toward more responsive generation as listener interests change.

For more information, please refer to our paper:

 Hypothesis-Driven Shelf Generation for Personalised Recommendation

Aleksandr V. Petrov, Tarun Chillara, Matthew D. Moellman, Lucas de Haas, Yabai Song, Alina Susoykina, Melissa Crawford, Gabriel Negash, Erik Franco, Tasnim Rahman, Binal Jhaveri, Shubham Bansal, Hugues Bouchard, Roberto Mirizzi, Mounia Lalmas, and Aloïs Gruson.

RecSys 2026