A Wearable Personal Assistant for Surgeons – Design, Evaluation, and Future Prospects

Abstract

In this paper, we present our body-and-mind-centric approach for the design of wearable personal assistants (WPAs) motivated by the fact that such devices are likely to play an increasing role in everyday life. We also report on the utility of such a device for orthopedic surgeons in hospitals. A prototype of the WPA was developed on Google Glass for supporting surgeons in three dierent scenarios: (1) touch-less interaction with medical images, (2) tele-presence during surgeries, and (3) mobile access to Electronic Patient Records (EPR) during ward rounds. We evaluated the system in a clinical simulation facility and found that while the WPA can be a viable solution for touch-less interaction and remote collaborations during surgeries, using the WPA in the ward rounds might interfere with social interaction between clinicians and patients. Finally, we present our ongoing exploration of gaze and gesture as alternative input modalities for WPAs inspired by the hospital study.