AirContour: Building Contour-based Model for In-Air Writing Gesture Recognition

Recognizing in-air hand gestures will benefit a wide range of applications such as sign-language recognition, remote control with hand gestures, and “writing” in the air as a new way of text input. This article presents AirContour, which focuses on in-air writing gesture recognition with a wrist-worn device. We propose a novel contour-based gesture model that converts human gestures to contours in 3D space and then recognizes the contours as characters. Different from 2D contours, the 3D contours may have the problems such as contour distortion caused by different viewing angles, contour difference caused by different writing directions, and the contour distribution across different planes. To address the above problem, we introduce Principal Component Analysis (PCA) to detect the principal/writing plane in 3D space, and then tune the projected 2D contour in the principal plane through reversing, rotating, and normalizing operations, to make the 2D contour in right orientation and normalized size under a uniform view. After that, we propose both an online approach, AC-Vec, and an offline approach, AC-CNN, for character recognition. The experimental results show that AC-Vec achieves an accuracy of 91.6% and AC-CNN achieves an accuracy of 94.3% for gesture/character recognition, both outperforming the existing approaches.