Exploring Interactive Gestures with Voice Assistant on HMDs in Social Situations

Voice assistants (VAs) are growing in popularity within VR and AR. However, speaking to the VA publicly can lead to misunderstanding and awkwardness. In this paper, we explored the interactive technique that enabled users to indicate whether to talk to the VA in a sociable manner. We initially collected 62 switching gestures leveraging various body parts (e.g., head and hand) through a brain-storming session with ten experts, then voted on the best ones. Our findings emphasized the speak-to-shoulder gesture. We then developed the detection system and assessed its performance using real data collected from 15 participants. The results showed an accuracy of 98.0%, and a false triggering rate of 0.10 times per minute.

Conference paper