Structural complexity in music modelling and generation with deep neural networks

Abstract

The automatic composition of music is a historical challenge that dates back to the ancient Greeks, and remained up to and beyond Mozart with the "Dice Game" and Ada Lovelace, speculating that the "calculating engine" might compose elaborate and scientific pieces of music of any degree of complexity or extent. In recent years, data-driven generative systems based on deep learning architectures have achieved impressive results on symbolic music, and they can also produce acoustically realistic outputs when trained on the raw audio. Nevertheless, composing musical ideas longer than motifs and phrases is still an open challenge in computer-generated music, a problem that is commonly referred to as the lack of long-term structure in the generations. In addition, the evaluation of the structural complexity of artificial compositions is still done manually - requiring expert knowledge, time and involving subjectivity which is inherent in the perception of musical structure. The thesis addresses this specific research gap by introducing a collection of methods and tools for the automatic evaluation of structural complexity in music. First, a music modelling framework is introduced, allowing experimenters to design, train and evaluate their music models, and compare their performance with state of the art methods. This is fundamental considering that the structural problem stems from the notorious technical challenges of the task on which these generative systems are trained - the problem of learning long-term dependencies in sequential data. Second, to evaluate structural complexity from arbitrary music pieces, a novel algorithm for music structure analysis and a set of metrics were designed. The former is used to segment music hierarchically by detecting nested structural segments at all possible temporal levels. This is followed by the extraction of a set metrics to formally describe the decomposition process of music into its structural components. The algorithm was found to achieve state-of-the-art performance in hierarchical music structure analysis, and the structural metrics succeeded in distinguishing music belonging to different structural complexity groups. Finally, the thesis concludes with an investigation on the use of structural properties of music to potentially improve the performance and the interpretability of methods for music information retrieval, with a case study in music emotion recognition.

Date of Award	1 Aug 2022
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Xiaojun Zeng (Supervisor) & Angelo Cangelosi (Supervisor)