Sensors for the Senses: Meaning-making via self-active entertainment experiences

Abstract

In his ACM Computers in Entertainment article, titled "Artist and Audience: Emerging the Nano-entertainment experience", the author posited on how Inhabited Information Spaces, created as core catalyst of research, may be questioned as a multisensory future virtual work of art. This themed Human-Computer Interaction for Entertainment contribution for the EAI INTETAIN 2015 conference builds upon the earlier work by questioning meaning making from such self-active entertainment experiences. Contextually, self-active relates to actor empowerment via ICT, whilst entertainment refers to HCI paradigms that are fun, engaging, and enjoyable. Conceptualizing, designing and realizing alternative digital media entertainment situations in stage performance, interactive installations and exhibitions at leading Museums for Modern Art, National and International major events, contributed to development of a sensor-based system conceived as a platform to investigate meaning making having societal impact beyond art. The system involves arrays of selectable sensor profiles mixed and matched according to requirements. Sensing of human input can be through worn (biosignal e.g. EEG, ECG, EMG, GSR), held, and/or non-worn sensors (volumetric, linear and planar interface profiles). Mapping of sourced human data is to a variety of digital content including art-based (music making, digital painting, lighting effects), video games, Virtual Reality and robotic devices. System adaptability promotes participant profile matching e.g. according to desired outcome. All ages and abilities are potential users. Preceding the commonly known camera-based game controllers such as EyeToy, Wii, and Kinect; the SoundScapes Virtual Interactive Space system has been used in institutes, hospitals and clinics to empower people with impairment to unconsciously push their limits of functionality via creative and playful expression. Rehabilitation is less mundane and boring, where variety of ICT-based intervention motivates whilst offering creative opportunities for facilitators. Additionally, in a randomized intervention study with frail elderly patients, Hagedorn and Holm reported significant clinical rehabilitation impact up to 400% in the ICT feedback training specific performance. This work highlights how co-informing paradigms between computer and human science and art, games and healthcare, can result in societal impact as well as commercial patented product and industry start-ups. At the core of the research is a rethinking in the way a human interacts with ICT. Current research explores a concept of dynamic air and alternative physical-virtual environments. Conclusions introduce a suggested need of increased researcher awareness of post-research effect following intervention/trials within healthcare and rehabilitation.