Abstract
For testing the controllability of controller in virtual environment, we trace multi-fingers and hand movement, and consider the interaction between virtual buttons and hand. We propose the evaluation method of button control on handheld devices using the SPIDAR-H and AR technology for tracking the fingertip and hand movement.