Controlling Hands in Virtual Reality

Consumer-oriented Virtual Reality (VR) hardware now allows users to interact with virtual environments with their own hands through the use of hand-held spatially tracked controllers. However, the physics constraints of the virtual environment do not apply to the user's real hands. This implies that if the virtual hands follow the movements of the real hands exactly, the user can make the virtual hands break the expected physics rules of the virtual environment e.g. by moving their real hands inside of virtual objects. This work addresses this problem by prototyping ways of controlling hands in VR that allow the virtual hands to deviate from the tracking data by adjusting the position, rotation and finger placement when there are conflicts between the user input and the virtual environment constraints. The value of this hand control model is assessed by comparing one of the developed prototypes with a hand control model that imitates the hands in the critically and commercially acclaimed VR game Job Simulator in controlled experiments. Much work remains to be done, but the results of the experiments are positive and seem to indicate that the user experience could be improved using this type of hand control.