Gesture and Spatiality in Electroacoustic Improvisation with Digital Video

SHORT PAPER. This paper discusses a multidisciplinary approach to improvisation in which electroacoustic performance is informed by its capacity to transform and respond to characteristics of digital video, along with visual and architectural features of virtual spaces. We describe technical and creative strategies that are focused on leveraging mobile devices in a performance setting for navigating audio-visual relationships in which digital spaces are conveyed through video playback and processing, while gestural control highlights actual physical spaces (e.g., gallery spaces). This creates a hybrid venue that benefits live electroacoustic creation with digital artists whose work extends beyond the domain of sound. The presentation visual analogues to sonic transformations of gesture and spatiality are used to highlight aspects of electroacoustic performance that develop out of a gradual exploration between emulation and obfuscation of referential sound objects. Mappings between electroacoustic instrument and digital video also help delineate the creative use of mobile devices and controllers as being more than tools. At the same time, the transmission of MIDI and OSC (Open Sound Control) messages from mobile instruments allows collaborating artists to resource electroacoustic sound in non-aural contexts.