Playing By Ear: A Computational Approach (Seb Silas, Hanover Music Lab)

Abstract

Playing by ear is an essential skill in many musical styles, supporting tasks like improvisation – and even sight-reading. Whilst jazz education often emphasises ear training, more contemporarily – and perhaps paradoxically – it also features visually-notated practice books to enable a more systematic approach to acquiring melodic representations. However, empirical research has yet to comprehensively explain how playing by ear learning strategies (e.g., by sight vs. by ear) affect memory and recall performance. Additionally, despite being a task apt for a computer, a computerised way of managing the learning of large melodic vocabularies by assessing musicians via produced musical input remains unrealised. To address this, we developed a software architecture to measure and record playing by ear and singing skills in real time. Using large corpora of jazz solos and sight-singing exercises, we tested participants under different learning conditions (e.g., visual vs. auditory, with or without singing first). With such data, we developed scoring approaches and sought complementary statistical modelling frameworks to analyse how people learn to play and sing melodies by ear.

Reflecting on these results, first, I will discuss how various cognitive and musical factors—such as general working memory, musical training, and item-level melodic characteristics—shape the ability to play melodies by ear. Additionally, I will examine learning patterns and how different learning strategies impact the later recall of melodies. Second, I will discuss how these results can be incorporated into a computational model which predicts how well somebody might be able to play a given melody by ear based on these factors. Specifically, I will frame my discussion in terms of item response theory-inspired approaches and suggest a musical “DASH” (Difficulty, Ability, Study History; Mozer & Lindsey, 2017) model. This approach helps manage melodic item banks more effectively, making it easier to develop melodic skills in both playing and singing by ear. Ultimately, this research can help design better educational tools which integrate aural learning with structured practice. In this light, finally, I will present two prototype computerised learning applications: Slonimsky — to improve playing by ear skills — and Songbird, for melodic singing skills.

Biography

Seb Silas is a PhD Researcher at Hanover Music Lab, supervised by Reinhard Kopiez and Daniel Müllensiefen. He researches computational approaches to musical learning and memory, with a particular emphasis on learning melodies and playing by ear. He is an active music teacher and saxophonist, improviser and composer, most notably with the band Don’t Problem. He draws upon his experience in these fields in his academic research.