Abstract—The recently developed depth sensors, e.g., the Kinect sensor, have provided new opportunities for human-computer interaction (HCI). Although great progress has been made by leveraging the Kinect sensor, e.g., in human body tracking, face recognition and human action recognition, robust hand gesture recognition remains an open problem. Compared to the entire human body, the hand is a smaller object with more complex articulations and more easily affected by segmentation errors. It is thus a very challenging problem to recognize hand gestures. This paper focuses on building a robust part-based hand gesture recognition system using Kinect sensor. To handle the noisy hand shapes obtained from the Kinect sensor, we propose a novel dis-tance metric, Finger-Earth Mover’s Distance (FEMD), to measure the dissimilarity between hand shapes. As it only matches the finger parts while not the whole hand, it can better distinguish the hand gestures of slight differences. The extensive experiments demonstrate that our hand gesture recognition system is accurate (a 93.2 % mean accuracy on a challenging 10-gesture dataset), ef-ficient (average 0.0750 s per frame), robust to hand articulations, distortions and orientation or scale changes, and can work in uncontrolled environments (cluttered backgrounds and lighting conditions). The superiority of our system is further demonstrated in two real-life HCI applications. Index Terms—Finger-Earth Mover’s Distance, hand gesture recognition, human-computer interaction, Kinect system.

In this era of an incredibly fast and evoluting industry of usability the topic of User Experience and User interaction has been increasingly popular and widespread lately. As life is getting easier and more plentiful, people are demanding more. They are no longer satisfied with powerful features and solid quality, the more intuitive way to use the function and more decent user impression is what they need and prefer. In the present day framework of interactive, intelligent computing, an efficient human computer interaction is assuming utmost importance. Gesture based computing enables humans to interface with the machine (HMI) and interact naturally without any dedicated devices. Building a richer bridge between machines and humans than primitive text user interface or even (graphical user interfaces) GUIs, which still limit the majority of input to keyboard and mouse. In fact we are bridging this gap by bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures. Gesture Based Computing provides an attractive alternative for human computer interaction (HCI). This project proposes comprehensive review for a real-time system capable of understanding commands through a survey on tool, techniques and algorithms with emphasis on hand gestures which focuses on the analysis and comparisons from the review. Also discuss and highlights the challenges, applications and future scope.