Not so Autonomous, Very Human Decisions in Machine Learning. AAAI Spring Symposium on UX for ML: Designing the User Experience of Machine Learning Systems.

Abstract

Until the machines are fully autonomous and generate themselves, human design decisions affect Machine Learning outcomes every step of the way. This position paper outlines multiple stages at which design decisions affect machine learning outcomes, and how they interact. This includes: dataset curation and data pipelines, selection of optimization targets, and the designed dialogue with end-users with its implicit and explicit feedback mechanisms. We specifically also call out another user group that appears somewhat overlooked in the research literature – the data curators and editors often involved in selecting and annotating the data that machines learns from.

Related

December 2020 | NeuRIPS

Model Selection for Production System via Automated Online Experiments

Zhenwen Dai, Praveen Chandar, Ghazal Fazelnia, Benjamin Carterette, Mounia Lalmas

October 2020 | CIKM

Query Understanding for Surfacing Under-served Music Content

Federico Tomasi, Rishabh Mehrotra, Aasish Pappu, Judith Bütepage, Brian Brost, Hugo Galvão, Mounia Lalmas

October 2020 | ISMIR - International Society for Music Information Retrieval Conference

“Play Music”: User Motivations and Expectations for Non-specific Voice Queries

Jennifer Thom, Angela Nazarian, Ruth Brillman, Henriette Cramer, Sarah Mennicken