Let Me Ask You This: How Can a Voice Assistant Elicit Explicit User Feedback?

Abstract

Voice assistants offer users access to an increasing variety of personalized functionalities. The researchers and engineers who build these experiences rely on various signals from users to create the machine learning models powering them. One type of signal is explicit in situ feedback. While collecting explicit in situ user feedback via voice assistants would help improve and inspect the underlying models, from a user perspective it can be disruptive to the overall experience, and the user might not feel compelled to respond. However, careful design can help alleviate friction in the experience. In this paper, we explore the opportunities and the design space for voice assistant feedback elicitation. First, we present four usage categories of explicit in-situ context for model evaluation and improvement, derived from interviews with machine learning practitioners. Then, using realistic scenarios generated for each category and based on examples from the interviews, we conducted an online study to evaluate multiple voice assistant designs. Our results reveal that when the voice assistant is framed as a learner or a collaborator, users were more willing to respond to its request for feedback and felt that the experience was less disruptive. In addition, giving users instructions on how to initiate feedback themselves can reduce the perceived disruptiveness to the experience compared to asking users for feedback directly in the form of a question. Based on our findings, we discuss the implications and potential future directions for designing voice assistants to elicit user feedback for personalized voice experiences.

Related

September 2022 | RecSys

Identifying New Podcasts with High General Appeal Using a Pure Exploration Infinitely-Armed Bandit Strategy

Maryam Aziz, Jesse Anderton, Kevin Jamieson, Alice Wang, Hugues Bouchard, Javed Aslam

September 2022 | Interspeech

Unsupervised Speaker Diarization that is Agnostic to Language Overlap Aware and Free of Tuning

M Iftekhar Tanveer, Diego Casabuena, Jussi Karlgren, Rosie Jones

September 2022 | Interspeech

Exploring audio-based stylistic variation in podcasts

Katariina Martikainen, Jussi Karlgren, Khiet Truong