In a Silent Way: Communication Between AI and Improvising Musicians Beyond Sound
This addresses the challenge of enhancing collaboration and trust between AI and human musicians in creative improvisation, though it is incremental as it builds on existing work in AI and human-computer interaction.
The researchers tackled the problem of building trust in AI-human musical improvisation by developing an AI drummer that communicates its confidence through emoticon-based visualizations, resulting in a positive correlation between this extra-musical communication and increased human musical engagement as measured by the FSS-2 questionnaire.
Collaboration is built on trust, and establishing trust with a creative Artificial Intelligence is difficult when the decision process or internal state driving its behaviour isn't exposed. When human musicians improvise together, a number of extra-musical cues are used to augment musical communication and expose mental or emotional states which affect musical decisions and the effectiveness of the collaboration. We developed a collaborative improvising AI drummer that communicates its confidence through an emoticon-based visualisation. The AI was trained on musical performance data, as well as real-time skin conductance, of musicians improvising with professional drummers, exposing both musical and extra-musical cues to inform its generative process. Uni- and bi-directional extra-musical communication with real and false values were tested by experienced improvising musicians. Each condition was evaluated using the FSS-2 questionnaire, as a proxy for musical engagement. The results show a positive correlation between extra-musical communication of machine internal state and human musical engagement.