Swap Nil
Swap Nil
Swap Nil
A
Seminar Report
On
Smart Mirror
Bachelor of Engineering
in
Computer Science & Engineering
of
Sant Gadge Baba Amravati University, Amravati
Submitted by
Mr. Swapnil Ravindra Murkute
Certificate
Dr. D.R.Dhotre
Head, CSE Dept
Date:-19/09/18
Acknowledgement
The real spirit of achieving a goal is through the way of excellence and lustrous
discipline. I would have never succeeded in completing my task without the
cooperation, encouragement and help provided to me by various personalities.
Also I would like to thanks to all teaching and non-teaching staff of the
department for their encouragement, cooperation and help. My greatest thanks are to all
who wished me success especially my parents, my friends whose support and care
makes me stay on earth.
i
CONTENTS
1. INTRODUCTION 1
2. LITERATURE SURVEY 2
3. METHODOLOGY 3
3.1 Smart mirror as a mirror 4
3.2 Smart mirror as Information system 4
3.3 Smart mirror as security system 4
3.4 Algorithm for Information System 5
3.5 Algorithm for Thief Detection 5
4. DESIGN 6
4.1 Hardware components overview 8
4.1.1 Raspberry Pi 8
4.1.2 Webcam 8
4.1.3 LCD Screen 8
4.1.4 Mirror 8
4.1.5 Microphone 9
4.2 Software components overview 9
4.2.1 OpenCV 9
4.2.2 RaspbianOS 10
4.2.3 NodeJS 10
4.2.4 Python 10
5. WORKING PRINCIPLE 11
5.1 Two-way glass Mirror 13
5.2 Monitor 13
5.3 Raspberry Pi 3 Model 13
5.4 Channel Relay 13
5.5 Camera 14
6. FUNCTIONAL OVERVIEW 15
CONCLUSION 16
REFERENCES 17
ii
FIGURES
TABLES
iii
Smart Mirror
ABSTRACT
Every morning our day begins by watching ourselves at least once in mirror before
leaving our home. We interact with it to find out how we look and how our attire is.
Smart mirror or Magic mirror is one of the application of Raspberry Pie .A computer
screen embedded in mirror looks very futuristic .The Raspberry pie stays at back scenes
and controls the data displayed on mirror. Such mirror can be programmed to work as
AI and control home appliances by voice input or touch screen.
The proposed smart mirror will be operated by Raspberry Pi and will be
connected by real world through internet. The smart mirror will consist Raspberry Pi,
LED monitor, speakers, camera, microphone with two-way mirror and acrylic glass.
With the help of voice recognition api the mirror will communicate with the user
through voice commands and responds them accordingly. The mirror could also support
human gestures. The mirror will highlight some basic amenities like time, local news,
weather. The mirror will also perform some advance functions such as booking a ride
on Uber, face recognition, etc. This mirror with artificial intelligence will provide an
extraordinary experience to the user.
Intelligent mirrors, which continue the works today and will take its place in the
future technology, provide both mirror and computer aided information services to its
users. Thanks to the microcontroller cards onboard, these systems, which can connect to
the internet and take data from the internet, can show this information on the places
located on the mirror. In the scope of the study, the developed intelligent mirror system
includes the weather information, time and location information, current event
information, user information, and camera image taken from web services using
Raspberry Pi 3 microcontroller card. Some equipment can be controlled by voice
commands via the microphone on the smart mirror.
iv
Smart Mirror
1.INTRODUCTION
A smart mirror is a mirror that has been enhanced by technology. It is very simple
concept that is illustrated in figure1.1. A screen is placed behind a two-mirror by using
a black and white graphical user interface (GUI) only the white colors will penetrate the
mirror,resulting in an effect that makes it appear as if the mirror itself is a screen. The
idea of an intelligent mirror is to further this enhancement to include artificial
intelligence as well.
Smart mirrors are straight from science fiction. They’re part of an optimistic vision of
future that imagines a world where screens and data are everywhere, ready to feed you
what even information you need at a moment’s notice. Basically, the mirror is looks like
normal mirror but when someone stand in front of it the scene changes. The mirror
provides a functional, user friendly and interactive UI to its user for accessing their
social sites, messengers, etc. It has widgets for displaying the current whether conditions,
Time, Events, Latest news headlines The Smart Mirror would help in developing smart
houses with embedded artificial intelligence, as well as finding its applications in
industries. Switching home appliances becomes easy with mirror. Virtual dressing, a
smart way of having trials with your fashion sense make things quite easy in malls.
Having such intellectual mirror will only surge the beauty of home. The raspberry pi is
programmed using python and connects to a monitor with inbuilt speaker so as to
provide an onscreen interface and voice assistance as well.
2. LITERATURE SURVEY
virtual keyboard.
3.METHODOLOGY
The device is look like a regular mirror but would have a screen inside. A smart mirror
is basically a mirror with a screen behind it. That screen can be an Android tablet or a
computer monitor. The project which would collect real world machine data such as
location based latest news and headlines, weather reports, and as well as show us the
local time. The data would be transmitted from the machine and would be managed in a
central database and would be managed by the Raspberry Pi. The Smart Mirror
implemented as a personalised digital device equipped with peripherals such as
Raspberry PI, microphone, speakers, LED Monitor covered with a sheet of reflective
one way mirror provides one of the most basic common amenities such as weather of
the city, latest updates of news and headlines and local time corresponding to the
location. The mirror display is provided by a flat LED display monitor
A one way mirror is used to provide real time display of what is located in front of the
Smart Mirror using Raspberry Pi thereby mimicking the function of a regular mirror
The mirror is eventually a technologically augmented interaction device. The objective
of designing the mirror is to provide a natural interface in the ambient home
environment for accessing various services such as location based weather, time,
calendar etc The project includes downloading the Raspbian operating system based on
Debi an and extracting the image on SD card, inserting the card in the Raspberry PiSD
slot an then performing the required steps.
Customized user profiles: The output of user recognition then triggers the
display of the interface. The interface is designed as per the user. The interface
allows a user to view Rich Site Summary (RSS) feeds of social media and
email, have access to maps, calendar, weather and time.
We can see our view as we can see it in a natural mirror while looking and
grooming with the help of one way mirror with high concentration of aluminum
content.
Time, Date, weather details and news are fetched from online using
predefined URL. News is fetched from websites like CCN, BBC etc. DHT22 –digital
sensor is used to get the humidity and temperature details.DHT22 is connected to GPIO
pins of Raspberry Pi board using jumpers.
Step 1: Start
Step 2: Setup the Camera
Step 3: Check whether PIR sensor output is high or low
Step 4: If it is low, go to step 3.
Step 5: If it is high, camera is turned ON.
Step 6: Image is captured and stored on raspberry pi.
Step 7: Check for Wi-Fi connection.
Step 8: If it is connected image is uploaded to drop box.
Step 9: Notification is updated in drop box.
4.DESIGN
Table 1 shows basic required objects for building mirror and their functionality.
Now see the basic design of smart mirror along with its software and hardware
implementation in detail.
For CPU, Raspberry Pi 3B mini computer was used and all software components were
installed into the operating system. The CPU will take the video and audio information
from the camera and microphone respectively and through facial recognition and voice
recognition model it will identify users. Once the smart mirror senses the presence of
any person, it will ‘wake up’ and display basic features like clock, calendar, weather
update, etc. Smart mirror will also have some basic AI features integrated to it. Users
will be able to interact with the mirror in real time and search for information on
internet, ask questions, perform certain tasks like setting alarm, reminder, etc. with the
help of smart mirror. All the information is displayed on the LCD screen connected
with the raspberry pi. All components reside behind a special mirror known as two way
see through mirror which is made of acrylic material.
The Smart Mirror CPU is the Raspberry Pi 3 computer. This is where all the software
components would lie in. The CPU takes in video information from the camera, runs
that information through the facial recognition model and identifies the user. After the
CPU knows the person standing in front of it, it would retrieve the information
programmed to show for that user. Finally, the CPU projects this information on to the
Screen.
All these components reside behind a special mirror known as a Two Way Mirror.
Let’s consider the hardware and software component of the backside design.
4.1.2 Webcam:
A simple USB powered webcam was used for facial recognition. It was used as
an input device to take video or image as input.
4.1.4 Mirror:
A special mirror known as a two way mirror or observation mirror is used in
this project. A two mirror is special as compared to an ordinary household
mirror. Unlike a household mirror, the two way mirror is not painted with an
opaque color on the back , instead its left untouched. This gives the property of
the mirror being reflective one side and transparent/translucent from the other.
4.1.5 Microphone:
One mode of interaction with the smart mirror is through microphones. Two
microphones are used to power the voice recognition capabilities of the device.
USB microphones has to be used beacause the Raspberry Pi does not have
regular microphone input.
4.1.6 Frame/Enclosure:
An enclosure is used to cover all the hardware behind the mirror so that the
mirror looks as close as possible to a normal mirror.
We installed Geany, which is a very lightweight IDE and used it to write all the HTML,
Javascript, CSS and Python code.
4.2.1 OpenCV:
It has C++, C, and Python, Java and MATLAB interfaces and supports
Windows, Linux, Android and Mac OS.
4.2.2 Raspbian OS :
4.2.3 NodeJS:
NodeJS is a javascript engine for server side applications. It comes included
with Electron and we used it to launch processes to control things that are not
available in web APIs such as the sensors and microphones for voice
recognition. We also used it to access the filesystem and read the app files.
4.2.4 Python:
Python is a high-level, general purpose, interpreted programming language. Its
very popular in the Raspberry Pi community and it has lots of support and
libraries. We used it with the microphone to detect sounds and also for control
of sensors.
5.WORKING PRINCIPLE
First, a frame for the mirror was built and attached the two-way see through mirror with
it. One side of the frame was for the viewing of user and on the other side was
assembled all the hardware components. A LCD was attached to display with the frame
and connect the LCD display with the raspberry pi via HDMI cable. USB microphone
and USB web were then connected with the raspberry pi. Finally, the power source for
both the raspberry pi and LCD display was established.
As the raspberry pi has its own operating system, the Raspbian operating
system was booted for the project into the raspberry pi. It was updated and upgraded to
increase the CPU speed. The default version of the operating system consisted of an
older version of the Node which did not consist of NPM. So, the Node v5.1.1 was
reinstalled which included the NPM v1.9. Next, pip was installed which was a package
installer of python and it helped to install numerous packages. However, python Speech
Recognition 3.8.1 was installed as it was used to convert audio into texts and for
processing the video inputs. An OpenCv, a library of programming functions was used
which focused on real time computer vision.
Now moving to the coding and designing part, HTML, CSS and JavaScript were used
to display the information on the LCD display in such a way that only the
information’s would appear before the user. And for the AI and other modules, python
programming language was used. However, an open source AI library was used.
AI will take voice information as input and through voice recognition, will
identify users. Processing audio or speech is more time consuming than processing text.
So, the audio input will be converted into text through speech software, here used
google speech to perform this task. To make AI understand the text, wit.ai was used
which would have intent and entity. Then the AI needed to retrieve relevant information
as per user request and this the knowledge phase was called. Once the information is
obtained, AI needs to form a complete sentence to express the information, so this is
done in the Natural Language Generation state.
AI will take voice information as input and through voice recognition, will
identify users. Processing audio or speech is more time consuming than processing text.
So, the audio input will be converted into text through speech software, here used
google speech to perform this task. To make AI understand the text, wit.ai was used
which would have intent and entity. Then the AI needed to retrieve relevant information
as per user request and this the knowledge phase was called. Once the information is
obtained, AI needs to form a complete sentence to express the information, so this is
done in the Natural Language Generation state.
The working of each components in smart mirror is explained in this section. Let’s talk
about them one by one:
5.2 Monitor:
The monitor is directly connected to Raspberry Pi via HDMI interface thus providing
display as well as voice output. For providing touch ability to monitor IR frames are
used which are explained further in next sub-section.
5.5 Camera :
The Camera is input device for the mirror, its used for face recognition as well as body
recognition. A concept called Virtual Dressing can be implemented using Image
Processing. We roam in malls in search of various clothes, we cannot practically try all
dressings that we like. So Virtual sorts this problem out, Images of this clothing’s are
saved in memory of mirror, whenever user stands in front of mirror and selects the
dress, the mirror fits the dress on our body reflection in mirror. So, no need for doing
trials every time. Face Recognition based authentication is another use of the camera, it
customizes the profile of different users on same mirror.
5.6 Microphone:
The microphone is used to give voice input to the mirror. Along with touch capability a
voice input makes the system very reliable and robust in working. A sensitive
microphone takes voice command from the user and processes it to do corresponding
action.
6.FUNCTIONAL OVERVIEW
On start after giving power supply to mirror the raspberry loads the OS and directly
runs the UI of the mirror, it may take while to load the weather and other internet based
widgets to display the information completely depending on the speed of the home Wi-
Fi network. A power saving mode is also provided to the mirror by attaching a
ultrasonic range detector sensor on the top of the mirror. The ultrasonic sensor activates
the mirror and awakes it when someone comes close enough. Thus, improving the
functionality and power efficiency of the system. On fully loading the information
required the mirror now shows notification from different social sites as per the user has
logged in. The events are loaded from calendar and reminded to the user. The user can
plan route to his destination before leaving home on the mirror. A mass of
entertainment is also loaded on the mirror, as its connected to home Wi-Fi YouTube
can be completely streamed on it and a music app is preloaded to beat your favorite
music directly. Besides several other applications like gallery is also installed to view
your pictures on mirror. The has a greeting pane which automatically prints greeting
message according to events and time of the day. You can book a cab before leaving
home using Uber cab booking interface made using Uber public API. Not only that, a
user who knows the working of system well can extend the ability of the system by
adding more feature to it. The Home automation embedded in this mirror is very easy to
use, as you have to simply connect the pins of the appliances to mirror’s slots and then
you can enjoy the control of appliances by touch interface.
CONCLUSION
Efforts have been made to build an efficient and intelligent smart mirror that optimizes
our time of doing works and increases our daily productivity. The Smart Mirror will
play an important role in the field of IoT and home automation. Not only this can
function as a normal mirror but can also provide other functionalities like weather
forecast, calendar, time, etc. which makes it more desirable. The functionality of the
mirror can be expanded by connecting it to other home appliances, mobile devices, etc.
Smart Mirror can be a great example of how AI can be integrated into home appliances
to make our life easier, efficient and more enjoyable. In future, Smart Mirror can be
made smarter by upgrading the AI. There is still a great scope to improve the AI. Soon,
normal mirrors will be replaced by smart mirrors if they can be made affordable.
REFERENCES
[1] Smart Mirror, Vinmay Patil, Mayuresh Hirnaik, Vaibhav Borole, Sonali Ajetrao,
Supriya Chaudhary ,International Journal of Engineering Research in Computer
Science and Engineering, Volumw 5,Issue 3, March 2018
[2] IoT based Smart Mirror using Raspberry Pi ,Lakshmi N S ,Chandana M
S,Ishwary P ,Nagarur Meena,Rajendra R Patil ,International Journal of
Engineering Research And Technology ,2018
[3] Design And Implementation of Smart Mirror As A Personal Assistant Using
Raspberry PI,Divyashree K.J, Dr. P.A.Vijaya,Nitin Awasthi,International
Research Journal Of Engineering and Technology,volume 5, Issue 5,May 2018
[4] Artificially Intelligent Smart Mirror using Raspberry Pi, Abdullahil Kafi,
M.Shaikh Ashikul Alam, Sayeed Bin Hossin, International Journal Of Computer
Applications, Volume 180-No.16 February 2018
[5] Raspberry Pi Powered Magic Mirror, Kanchan S. Gorde, International Journal
Of Advanced Research in Electrical, Electronics and Instrumentation
Engineering, Volume 6,Issue 12, December 2017
[6] Smart Mirror: A Reflective Interface to Maximize Productivity, Piyush
Maheshwari Maninder Jeet Kaur, Sarthak Anand,International Journal of
Computer Application Volume 166-No.9,May 2017