FPSI-Fingertip pose and state-based natural interaction techniques in virtual environments

Simple and natural interaction has a vital role in any realistic virtual environment (VE). This research proposes a set of lightweight gesture-based techniques for interaction in VEs with a focus on high accuracy, performance, and usability. The proposed techniques use a single fingertip pose and state for object/task selection, translation, navigation, rotation, and scaling. Four different techniques are proposed for interaction, i.e., MSGE (Menu-based task selection and gesture-based task execution), GSGE (Gesture-based task selection and gesture-based task execution), SGTE (Single gesture for task selection and execution), and TSGE (Time slice-based task selection and gesture-based task execution). Keeping in mind the concept of re-usability, the index-tip spatial position is used for task operation in all techniques. For experimental evaluation of the proposed techniques, a VE is designed in Unity3D, while interaction is carried out using the Leap Motion controller. The experimental study was conducted with forty (40) volunteer participants and two experts (authors). Experimental results show improved accuracy for TSGE (participants 97.22%, and experts 97.22%) as compared to others (participants: SGTE 95.55%, GSGE 94.44%, and MSGE 92.75%, experts: SGTE 94.44%, GSGE 94.44%, and MSGE 91.67%). Similarly, the results show high task performance for TSGE (participants 112.9 seconds, SD 5.3, experts 101.75 seconds, SD 3.3) as compared to others (participants: SGTE 117.2 seconds, SD 5.7, GSGE 121.8 seconds, SD 8.0, and MSGE 126.7 seconds, SD 12.9 and experts: SGTE 107.0, SD 5.7, GSGE 113.25, SD 3.5, and MSGE 122.0 seconds, SD 3.6). In addition, usability analysis shows high usability for the proposed interaction techniques, i.e., TSGE (SUS score 98.5), SGTE (SUS score 95.75), GSGE (SUS score 95.25), MSGE (SUS score 94.75). Furthermore, a comparative study with state-of-the-art interaction techniques showed a high accuracy rate, multiple tasks, and reusability support, use of easy to learn and use and fewer features-based gestures (fingertip gestures), and multiple interaction techniques (four techniques) support for the proposed techniques.