Masked Expressiveness: Conditioned Generation of Piano Key Striking Velocity Using Masked Language Modeling

Abstract

Creating and experiencing expressive renditions of composed musical pieces are fundamental to how people engage with music. Numerous studies have focused on building computational models of musical expressiveness to gain a deeper understanding of its nature and explore potential applications. This paper examines masked language modeling (MLM) for modeling expressiveness in piano performance, specifically targeting the prediction of key striking velocity using vanilla Bidirectional Encoder Representations from Transformers (BERT). While MLM has been explored in previous studies, this work applies it in a novel direction by concentrating on the fine-grained conditioned prediction of velocity information. The results show that the model can predict masked velocity events in various contexts within an acceptable margin of error, relying solely on the pitch, timing, and velocity data encoded in Musical Instrument Digital Interface (MIDI) files. Additionally, the study employs a sequential masking and prediction approach toward rendering the velocity of unseen MIDI files and achieves more musically convincing results. This approach holds promise for developing interactive systems for expressive performance generation, such as advanced piano conducting or accompaniment systems.