Reconstructing Human Expressiveness in Piano Performances with a Transformer Network
This work addresses the problem of modeling subtle expressive variations in music for applications in computational musicology and performance synthesis, though it notes ongoing challenges in fully reconstructing expressiveness.
The paper tackles the challenge of capturing human expressiveness in piano performances by proposing a multi-layer bi-directional Transformer encoder to reconstruct expressiveness from transcribed scores, achieving state-of-the-art results in generating human-like performances.
Capturing intricate and subtle variations in human expressiveness in music performance using computational approaches is challenging. In this paper, we propose a novel approach for reconstructing human expressiveness in piano performance with a multi-layer bi-directional Transformer encoder. To address the needs for large amounts of accurately captured and score-aligned performance data in training neural networks, we use transcribed scores obtained from an existing transcription model to train our model. We integrate pianist identities to control the sampling process and explore the ability of our system to model variations in expressiveness for different pianists. The system is evaluated through statistical analysis of generated expressive performances and a listening test. Overall, the results suggest that our method achieves state-of-the-art in generating human-like piano performances from transcribed scores, while fully and consistently reconstructing human expressiveness poses further challenges.