International Journal on Future Revolution in Computer Science & Communication Engineering ISSN: 2454-4248

Volume: 4 Issue: 6 59– 62

_______________________________________________________________________________________________
Hand Gesture Recognition using Python

Mr. Deepak K. Ray Mayank Soni

Department of Electronics and Telecommunication Department of Electronics and Telecommunication
BharatiVidyapeeth Deemed to be University College of BharatiVidyapeeth Deemed to be University College of
Engineering Engineering
Pune, Maharashtra, India Pune, Maharashtra, India
dkray@bvucoep.edu.in sonimayank110@ymail.com

Prabhav Johri Abhishek Gupta

Department of Electronics and Telecommunication Department of Electronics and Telecommunication
BharatiVidyapeeth Deemed to be University College of BharatiVidyapeeth Deemed to be University College of
Engineering Engineering
Pune, Maharashtra, India Pune, Maharashtra, India
prabhavjohri@gmail.com abhizz311@gmail.com

Abstract—– The goal for the project was to develop a new type of Human Computer Interaction system that subdues the problems that users
have been facing with the current system. The project is implemented on a Linux system but could be implemented on a windows system by
downloading some modules for python. The algorithm applied is resistant to change in background image as it is not based on background
image subtraction and is not programmed for a specific hand type; the algorithm used can process different hand types, recognizes no of fingers,
and can carry out tasks as per requirement. As it is stated within this paper, the main goals were reached. The application is capable of the
gesture recognition in real-time. There are some limitations, which we still have to be overcome in future.

Keywords-Python, pyautogui, Human Computer Interface,

__________________________________________________*****_________________________________________________

required at sight of activity and that is a limitation of the user.

I. INTRODUCTION
The proposed system can be used to control various soft panels
In today‘s world, the computers have become an important like HMI systems, Robotics Systems, Telecommunication
aspect of life and are used in various fields however, the System, using hand gestures with help of programming by
systems and methods that we use to interact with computers are within python using pyautogui module to facilitate interaction
outdated and have various issues, which we will discuss a little within different functions of computer through the Camera to
later in this paper. Hence, a very new field trying to overcome capture video frames.
these issues has emerged namely HUMAN COMPUTER
INTERACTIONS (HCI). Although, computers have made II. APPLICATIONS OF HAND GESTURE RECOGNITION
numerous advancement in both fields of Software and SYSTEM
Hardware, Still the basic way in which Humans interact with A Hand Gesture Recognition System recognizes the Shapes
computers remains the same, using basic pointing device and or orientation depending on implementation to task the
(mouse) and Keyboard or advanced Voice Recognition System, system into performing some job. Gestures is a form of non-
or maybe Natural Language processing in really advanced verbal information. A person can make numerous gestures at a
cases to make this communication more human and easy for us. time. As humans through vision perceive human gestures and
Our proposed project is the Hand gestures recognition system for computer we need a camera, it is a subject of great interest
to replace the basic pointing devices used in computer systems for computer vision researchers such as performing an action
to reduce the limitations that stay due to the legacy systems based on gestures of the person.
such as mouse and Touchpad. The proposed system uses hand A. Virtual presence:
gesture, mostly no of fingers raised within the region of Interest Sometimes in a situation like machine, electricity
to perform various operations such as Play, Pause, seek failure, emergency hostiles condition or some remote areas
forward, seek back word in video player (for instance VLC which are inaccessible to humans, it could very dangerous for
media player). A static control board restrains the versatility of human operators to be physically appear to operate the
client and limits the capacity of the client like a remote can be machines or in the working conditions. So, we can take help of
lost, dropped or broken while, the physical nearness of client is the telepresence where telepresence is the area of intelligence

59
IJFRCSCE | June 2018, Available @ http://www.ijfrcsce.org
_______________________________________________________________________________________
International Journal on Future Revolution in Computer Science & Communication Engineering ISSN: 2454-4248
Volume: 4 Issue: 6 59– 62
_______________________________________________________________________________________________
which provides the ability of physical operation. For instance, algorithm‘ used for this process and gray scale picture are
the robotic arm maps and repeat the actions performed by the converted into binary picture having area of interest as the hand
operator arm to carry out a specific operation. The prospects of and the background.
Virtual Presence or tele-presence also include applications like There are two main approaches to segmentation:
space missions, underwater mission, maintenance of nuclear 1. Pixel-based or local methods having: -
power reactor and anywhere the human presence is not possible
a) Edge detection
or risky.
b) Boundary detection.
B. Bomb disposal:
Bomb disposal is safer when human beings are 2. Region-based approach: -
substituted by the robot arm which will work on the same a) The region merging
concept of hand gestures recognitions. It leads to reduction in b) The region splitting
the risk of life of a human and it also encourages the efficient c) Thresholding method
handling of the situation. The robot arm can be used to
recognize the postures of a human made by him or her from
remote place and it also performs corresponding related
function.
C. Based on Wheel-chair:
The Individuals in wheel-chair regularly confront issues
with manual systems used to control the developments of seat.
Motions of the hand can be embraced with the goal that each
signal would be allotted to a specific development. At the point
when a specific signal is given the wheelchair moves in a
comparing course. There are numerous utilizations of hand
motions acknowledgment which are as of now being used like
the gaming business adjusts a considerable measure of hand
motion acknowledgment systems in which the utilization of
joysticks or consoles is absolutely superfluous these days.
Automated surgeries are likewise led where the specialists
work the patients from remote area and the robots play out the
errands.

III. ALGORITHM
a) Data obtaining – The initial move is to capture the
image from camera and to define a region of Interest
in the frame, it is important as the image can contain a
lot of variables and these variables can result in
unwanted results and the data that needs to be
processed is reduced to a large extent. To capture the
image a web-camera is used that continuously Fig. Algorithm Deployed.
captures frames and is used to get the raw data for
processing. The input picture we have here is uint8. Thresholding techniques is employed in partitioning the image
The Procured image is RGB and must be processed pixel histogram by using a single threshold technique.
before i.e. pre-processed before the components are We in this project have used otsu‘sthresholding technique.
separated and acknowledgement is made. Otsu‘s thresholding is used to automatically perform cluster-
b) Data Pre-Processing: based thresholding. The method assumes there are two classes
Pre-processing method can be completed 2-steps of pixels following bi modal histogram and then it calculates
process: optimum threshold separating the two classes
• Segmentation
• Morphological filtering 1. MORPHOLOGICAL FILTERING: After
First Process is the Segmentation process. It is done to change thresholding we have to make sure that there will be
over grey-scale picture into the binary picture so we can have no noise is present in image, so we are using
just two Area of Interest in picture. That is, one will be hand morphological filtering Techniques, These
and another one is background. Algorithm can be ‗Otsu Techniques are divided in 3 parts.

60
IJFRCSCE | June 2018, Available @ http://www.ijfrcsce.org
_______________________________________________________________________________________
International Journal on Future Revolution in Computer Science & Communication Engineering ISSN: 2454-4248
Volume: 4 Issue: 6 59– 62
_______________________________________________________________________________________________
a. Dilation 3. Math Operations:
b. Erosion this can be calculated by -
c. Opening and angle = math.acos((b**2 + c**2 - a**2)/(2*b*c)) * 57
d. Closing This formula determines the angles between the two fingers,
If the division is not continuous, then there may have some ‗1s‘ for distinguish between the different fingers and to identify
in the background which is called as background noise, Also, them all. We can also determine the length of each raised or
there is a possibility that system generated an error in collapsed finger coordinate points taking centroid as a source
recognizing gesture this may be termed as gesture noise. If we of perspective point, this can be done keeping in mind the end
want flawless contour detection of a gesture, then above- goal to extricate the correct number of finger brought up in the
mentioned errors should be nullified. A morphological picture.
separating (filtering) approach is employed utilizing grouping
IV. CONCLUSION
of dilation (enlargement) and erosion (disintegration) to
accomplish a smooth, shut, and finish the contours of a hand With the help of observations that we have, we can
motion. conclude that the results should be depend upon:

c) Extraction of Features:Pre-prepared or preprocessed 1. Thresholding. This is the concept of finding contours

picture is accessible to be utilized and different while converting the gray image to the binary image
highlights of the resultant picture are removed. as the concept of thresholding states. For instance, the
Following are the features that can be extracted: lighting over the photo of the hand possibly uneven
i. Finding Contours which is caused by drawing shapes of the contours
ii. Finding and correcting convex hull which is around the dim regions notwithstanding the
iii. Mathematical Operations form around the hand. Changing the limit should be
kept that from happening.
2. Background of the pictures should be plain to get
accurate analysis of recognition of gestures.
3. Extra occasional keep an eye on minutes is helpful
for checking whether the forms of the layout picture
and the picture of individual have same shape.
4. For maintaining the performance, the database should
contain the template images of small dimension.
We observe the following output by implementing algorithm:

1. Contours: It implies the direction of hand i.e.

regardless of whether the hand is on a horizontal plane
or vertically set. Initially, we try to find the orientation
by length to width ratio with a presumption that if the
hand is vertical ten length of the box bounding them
will be more than the width of the same box and, if Fig. Play/Pause Operation on Recognizing 2 Fingers
hand is horizontally placed then width of bounding
box is larger than width of the box binding the hand
will be more than that of the length of the box.
2. Finding and correcting convex hulls: A hand posture
is recognized by its ownorientation and by the fact
that how many fingers are shown. For getting the
aggregate of how many fingers are shownin hand
motions that we have to process just area of the finger
of the hand that we have in past advance by figuring
out and analyzing the centroid.
Fig. Seek Back Operation on Recognizing 3 Fingers
61
IJFRCSCE | June 2018, Available @ http://www.ijfrcsce.org
_______________________________________________________________________________________
International Journal on Future Revolution in Computer Science & Communication Engineering ISSN: 2454-4248
Volume: 4 Issue: 6 59– 62
_______________________________________________________________________________________________
[3] Thad Starner and Alex Pentland, ―Real-time American sign
language recognition from video using hidden markov models‖,
Technical Report No. 375, M.I.T Media Laboratory Perceptual
Computing Section, 1995.
[4] Dharmendra Kumar, Kanak Kumar,―Review on different
Techniques of Image Segmentation using MATLAB‖,
International Journal of Science, Engineering and Technology,
Volume 5 Issue 2: 2017

Fig. Seek Forward Operation on Recognizing 4 Fingers

Here, what we do is, we just open our script file it will
automatically launch a video player. Here we have chosen
VLC Media Player. Then script stops execution for pre-
defined time to load the media player. After video file is being
played then system invokes the tools that we required to run it
for instance- OpenCV, Camera, pyautogui. Now, we are ready
to do just sit back and control without using any conventinal
method. By, pointing out finger in plain background, we can
get following output.
NO OF FINGERS ACTION PERFOMED
0,1 No action
2 Play/Pause
3 Seek back
4 Seek forward
Table: Gesture-Action table

V. FUTURE WORK
This project can be more interactive with the help of tracking
real time hand movements and controlling mouse pointer on
screen. The shortcoming of requiring a plain background can
be overcome with the help of Background Image substraction
or Machine Learning Techniques

ACKNOWLEDGMENT
We would like to thank respected MR. DEEPAK RAY SIR for
giving me such a wonderful opportunity to expand our
knowledge for our E&TC branch in software and giving us all
the important guidelines to present this report. Sir helps us a lot
in realizing that what are the motive & applications for which
we study.

REFERENCES
[1] H. Renuka and B.Goutam ―Hand Gestures Recognition system
to control soft front panels‖, International Journal of
Engineering Research and Technology, December 2014.
[2] Henrik Birk and Thomas Baltzer Moeslund, ―Recognizing
Gestures From the Hand Alphabet Using Principal Component
Analysis‖, Master‘s Thesis, Laboratory of Image Analysis,
Aalborg University, Denmark, 1996.

62
IJFRCSCE | June 2018, Available @ http://www.ijfrcsce.org
_______________________________________________________________________________________