Flip, Freeze, Flow: Exploring Interactive Sonification of Breakdancing

Whilst movement sonification has been applied within sports training, motor rehabilitation, and several modern dance forms, it has not yet been researched in the context of the dance style of breaking (breakdancing in common parlance) despite significant potential to enhance dancer expressivity and audience engagement. As part of a student project, we designed and developed a realtime camera-based sonification system able to respond to various aspects of the dancer’s movement in a musical and intuitively movement-congruent manner using a sample-based audio architecture in Max/MSP. Parameter mappings were grounded in six conceptual movement-sound interactions informed by the fundamental movement vocabulary specific to breaking. Through reallife tests and interviews with two professional dancers, we found that the movement-sound interactions based on vertical motion (e.g. head height on the Y-axis) were the most intuitive and engaging, although the unintended activation of certain other effects caused frustration. In addition, the perceptually subtle mappings were often overlooked or ignored, highlighting the importance of salient sonic changes to the dancer experience. Nevertheless, we believe that our technological framework and mapping design philosophy can inform future research on the sonification of breaking.