Robotics: Fast and curious: A CNN for ethical deep learning musical generation

Recent progress in musical generation and classification has largely been achieved by deep learning. While many of these systems represent improved results in comparison to older methods, they have significant broader ethical implications that have not been as widely examined from a creative perspective. These costs are beginning to be explored in broader functional AI research, but we believe that they have not been adequately addressed in creative work. They include the prohibitive training requirements and subsequent environmental impacts, as well as the exclusion of artists without adequate computational resources. Other challenges include the ownership of material and the data that has been trained on, often coming from external sources with no input. For the research process itself, these networks include the lack of transparency in system design – emphasised through the removal of the need for domain expertise – which itself increases the lack of transparency and interpretability.

As a counter system, we present a human-first deep learning design allowing agency over interaction and generation. The system is simple, easy to train, and allows effective, interactive real-time system for generating musical improvisations in performance with human musicians. The system is comprised of a generative convolutional neural network and uses a novel data format that appears to allow improved learning of nonlocal dependencies and repetitive structure across beats within musical phrases. We have observed that the system is able to learn to generate convincing and coherent improvisations from relatively small amounts of data. It can run effectively with limited computational resources and produces pleasing musical interactions in a live performance setting. We discuss our proposed data format, the system design, and the affordances of the system.