Gesture Sonification. An Interaction Design Approach to an Artistic Research Case

Abstract

INTRODUCTION: The subject of this paper is a phenomenological study of cognitively plausible relationships between gesture and sound mediated by technologies. In this discussion, the term ‘gesture’ indicates a movement of the body consciously performed and able to express or communicate something. The expression ‘cognitively plausible’ refers to an interactive sound response that is immediate, continuously varying and enactively coherent with the generating (sonified) gesture.

OBJECTIVES: The objectives of this paper are two. The first is an artistic exploration of gesture sonifications in various contexts. The second is rather methodological, i.e., to provide a possible general paradigm for Artistic Research (AR).

METHODS: More in detail, the AR phenomenological approach is modeled according to an Interaction Design (IxD) research paradigm.

RESULTS: A number of case studies of gesture sonification are presented and discussed according to the above methodological framework.

CONCLUSION: We claim that the introduction of such methodological framework was successful in terms of providing robust guidelines for our research and for a clear and structured presentation of its results.