Rep g3
Rep g3
Rep g3
1. SYNOPSIS
In this ever-evolving digital world where speed, efficiency, and convenience are constantly
being optimized, it’s clear that we are moving towards less screen interaction. Virtual
assistants can tremendously save your time. Hence the main advantage of this voice search is
rapidity. Hence this idea is specially designed for Students Even the Simple tasks which we
handle manually can be handled easily.
1.2 Motivation:
A virtual assistant is an Artificial Intelligence technique. Voice assistants are changing the
way we use technology and the way we live. They have drastically improved over the last
five years and are finally available for every application. Voice assistants engage users like
no other interface. Users can speak to the applications naturally to ask for whatever they’d
like and can do so while multitasking. This also reduces the cognitive effort of the user.
Automating repeated tasks to a voice-activated personal assistant frees up human time and
resources.
Also, it can efficiently perform these mundane tasks with no errors, which often leads to an
improvement in customer satisfaction. While voice assistants are left
1
Voice Assistant- ‘Jarvisa’
to deal with routine tasks, humans can dedicate more time to duties where human
intervention is required for successful business solutions and services
1.3 Hypothesis
Voice assistants are programs on digital devices that listen and respond to verbal commands.
A user can say, “What’s the weather?” and the voice assistant will answer with the weather
report for that day and location.
Users also task virtual assistants to make and receive phone calls, create text messages, get
directions, hear news and weather reports, find hotels or restaurants, check flight reservations,
hear music, or play games.
We have to just give commands and it can perform further functionalities.
2
Voice Assistant- ‘Jarvisa’
3
Voice Assistant- ‘Jarvisa’
2. INTRODUCTION
In today’s era almost all tasks are digitalized. We have Smartphone in hands
and it is nothing less than having world at your finger tips. These days we aren’t even
using fingers. We just speak of the task and it is done. There exist systems where we
can say Text Dad, “I’ll be late today.” And the text is sent. That is the task of a Voice
Assistant. It also supports specialized task such as booking a flight, or finding
cheapest book online from various ecommerce sites and then providing an interface
to book an order are helping automate search, discovery and online order operations.
Voice Assistants are software programs that help you ease your day to day
tasks, such as showing weather report, creating reminders, making shopping lists etc.
They can take commands via text (online chat bots) or by voice. Voice based
intelligent assistants need an invoking word or wake word to activate the listener,
followed by the command.. We have so many voice assistants, such as Apple’s Siri,
Amazon’s Alexa and Microsoft’s Cortana., Likewise we have built an assistant
named ‘JARVISA’.
This project was started on the premise that there is sufficient amount of
openly available data and information on the web that can be utilized to build a voice
assistant that has access to making intelligent decisions for routine user activities.
4
Voice Assistant- ‘Jarvisa’
A voice assistant makes our life easier & helps users with functions that are the backbone of
our day. Someone who talks with you when you are alone or feel lonely someone who
manages your daily routines and takes care of your mental & physical health. The main
motivation is the voice assistant helps adults & disables people perform their day-to-day
tasks.
If we look at the daily life of a PC user, there are many tasks that must be automated instead
of wasting time doing them manually. Hence voice assistants can move out to handle such
operations eg., Managing notifications, Texting someone, web searching, etc.
5
Voice Assistant- ‘Jarvisa’
A user may have a privacy concern as personal assistants require a huge amount of data
and are always listening to take the command This passive data is then, retained and shifted
through humans that are employed by almost all of the major companies- Amazon, Apple
etc. In addition to this discovery over the Ai being able to record our audio interactions,
there have been concerns over the type of data such employees and contractors were
hearing. So, in the cloud-based voice assistant, a privacy policy must be there that will
protect the user information. Jarvisa removes all the privacy issues in voice Assistant
ecosystems.
3.2 SCOPE
Voice search has been a hot topic of discussion. Visibility of voice will undoubtedly be a
challenge. This is because the visual interface with voice assistants is missing. Users simply
cannot see or touch a voice interface unless it is connected to the Alexa or Google Assistant
app. Search behaviors, in turn, will see a big change voice assistants will continue to offer
more individualized experiences as they get better at differentiating between voices.
However, it’s not just developers that need to address the complexity of developing for voice
as brands also need to understand the capabilities of each device and integration and if it
makes sense for their specific brand. They will also need to focus on maintaining a user
experience that is consistent within the coming years as complexity becomes more of a
concern. This is because the visual interface with voice assistants is missing. Users simply
cannot see or touch a voice interface
6
Voice Assistant- ‘Jarvisa’
4.1 OBJECTIVES
The main objective of building personal assistant software (a virtual assistant) is using
semantic data sources available on the web, user-generated content, and providing knowledge
from knowledge databases.
The main purpose of an intelligent virtual assistant is to answer questions that users may
have. This may be done in a business environment, for example, on a business website, with a
chat interface. On the mobile platform, the intelligent virtual assistant is available as a call-
button-operated service where a voice asks the user “What can I do for you?” and then
responds to verbal input
As said earlier available assistants in the market today lack privacy. All the assistants provide
our data to the respective companies who have made them. like Apples Siri, Amazons Alexa,
etc. Violations of privacy id prevented when you built your own assistant and this is what we
have carried out in our project.
Feasibility Study
Technical feasibility:
It includes finding out technologies for the project, both hardware and software. For virtual
assistant, user must have a microphone to convey their message and a speaker to listen when
system speaks. These are very cheap now adays and everyone generally possess them.
7
Voice Assistant- ‘Jarvisa’
Besidesthe , system needs internet cone ction.While using JARVISA, make sure you have a
steady internet connection.
Operational feasibility:
It is the ease and simplicity of operation of the proposed system. System does not require any
special skill set for users to operate it. In fact, it is designed to be used by almost everyone.
Kids who still don’t know to write can readout problems for system and get answers.
Economical feasibility:
Here, we find the total cost and benefit of the proposed system over current system. For this
project, the main cost is documentation cost. User also would have to pay for microphone and
speakers. Again, they are cheap and available. As far as maintenance is concerned, JIA won’t
cost too much.
8
Voice Assistant- ‘Jarvisa’
Basic features
Youtube Automation
Youtube Search
Playing Videos Automatically
Handling operations of videos like Playing, pausing, muting etc
Changing Different Modes(like filmy mode,full screen mode,default mode)
• Opening browser
• Closing/opening a new tab
9
Voice Assistant- ‘Jarvisa’
5. METHODOLOGY
Python's large standard library provides tools suited to many tasks and is commonly cited as
one of its greatest strengths. For Internet-facing applications, many standard formats and
protocols such as MIME and HTTP are supported. It includes modules for creating graphical
user interfaces, connecting to relational databases, generating pseudorandom numbers,
arithmetic with arbitrary-precision decimals, manipulating regular expressions, and unit
testing.
Python is commonly used in artificial intelligence projects and machine learning projects
with the help of libraries like TensorFlow, Keras, PyTorch, and sci-kit-learning. As a
scripting language with a modular architecture, simple syntax, and rich text processing tools,
Python is often used for natural language processing.
Visual Studio Code is a streamlined code editor with support for development operations
like debugging, task running, and version control. It aims to provide just the tools a developer
10
Voice Assistant- ‘Jarvisa’
needs for a quick code-build-debug cycle and leaves more complex workflows to fuller
featured IDEs, such as Visual Studio IDE.
Working with Python in Visual Studio Code, using the Microsoft Python extension, is simple,
fun, and productive. The extension makes VS Code an excellent Python editor and works on
any operating system with a variety of Python interpreters. It leverages all of VS Code's
power to provide auto-complete and IntelliSense, linting, debugging, and unit testing, along
with the ability to easily switch between Python environments, including virtual
environments
1)Pyttsx3
2) Os module
The OS module in Python provides functions for interacting with the operating system.
OS comes under Python’s standard utility modules. This module provides a portable way of
using operating system-dependent functionality.
3)Wikipedia module
Wikipedia is a library in Python which it possible for the virtual assistant to process
queries regarding. This is an online library and needs an internet connection to fetch the
results. The no. of lines that the user wants to get as a result can be set manually
4)datetime module
A module named datetime can be imported to work with the date as well as time. Python
datetime module comes built into Python, so there is no need to install it externally.. Date and
11
Voice Assistant- ‘Jarvisa’
datetime are an object in python, so when you manipulate them, you are actually
manipulating objects and not string or timestamps.
5) webbrowser
6) Beautiful soup
Beautiful Soup is a Python library for pulling data out of HTML and XML files. It works
with your favorite parser to provide idiomatic ways of navigating, searching, and modifying
the parse tree. It commonly saves programmers hours or days of work.
7) pywhatkit
In Python3 pywhatkit module will not come pre-installed, so you can install it from web.
8) Request module
Requests is an HTTP library for the Python programming language. The goal of the project is
to make HTTP requests simpler and more human-friendly.
9)Keyboard module
Take full control of your keyboard with this small Python library. Hook global events,
register hotkeys, simulate key presses, and much more.
12
Voice Assistant- ‘Jarvisa’
5.1 WORKING :
Communicating with a voice assistant is so simple it almost makes us forget how impressive
the technology behind it is.
1. If you’ve ever used a voice assistant on your phone, then you know that it first needs
a keyword that activates it, like ‘OK, Google’. When you pose a question or give
a command to your voice assistant it is converted thanks to the use of Automatic Speech
Recognition (ASR), which enables the device to recognize and translate it from speech to
text.
13
Voice Assistant- ‘Jarvisa’
2. After that comes text interpretation: thanks to Natural Language Processing (NLP), the
machine can identify the intent behind what you said, the meaning of the words as well as
context.
3. Now that the machine knows what was said and (presumably) what your intention was, it
can search for a valid answer, and respond accordingly. The response is converted from
written to spoken form using a text-to-speech (TTS) technology, and voila: your voice
assistant speaks. All done in a matter of seconds
The proposed model of the voice assistant is as shown in the above figure 1. The model
consists of user input through microphone to accept commands from the user. These
commands are then go through Speech Recognition, it is the ability of a machine or program
to identify words and phrases in spoken languages and convert them to a machine-readable
format. On these input Natural Language Processing is applied, it is a field which is created
by amalgamating computer science and artificial intelligence. Using NLP, we are concerned
14
Voice Assistant- ‘Jarvisa’
with interactions between computers and human natural languages. Then the BRAIN check
whether it is a question or an action, if it is a action than the action is performed by the voice
assistant and acknowledgment is given to the user via a synthesis voice or if it is a question
than it is search in dialog box or knowledge base and then response via a synthesis voice to
the user. Our Voice assistant uses google text-to-speech API to understand all the words
spoken by the user, and based on certain conditions that satisfy being a command the voice
assistant sends responses to the user.
5.3 FLOWCHART
15
Voice Assistant- ‘Jarvisa’
Initially, the system is in idle mode. As it receives any wake up call it begins execution.
The received command is identified whether it is a questionnaire or a task to be performed.
Specific action is taken accordingly. After the Question is being answered or the task is being
performed, the system waits for another command. This loop continues unless it receives quit
command. At that moment, it goes back to sleep
16
Voice Assistant- ‘Jarvisa’
The above diagram shows entities and their relationship for a voice assistant
system. We have a user who will give commands through voice. The user here is
an entity.
We have other four entities here i.e, web search, youtube automation and browser
automation,system and settings. Each entities have their own attributes.
Single user can ask multiple questions. Each question will be given ID to get
recognized along with the query and its corresponding answer. User can also be
having n number of tasks. These should have their own unique id and status i.e.
their current state. A task
should also have a priority
value and its category
whether it is a parent
task or child task of an older
task.
DFD Level 0:
Level 0 of dfd gives overview components of a system. The user sends request to assistant
and thereby gets response as per given command.
17
Voice Assistant- ‘Jarvisa’
Level 1 still gives a a general overview of the system it gives the complwtw flow of
recognizing voice commands, searching for output at python back end and finally the Text-
to-speech conversion response to user.
Level 02 of Dfd gives detailed workflow of system.It shows the switching job of assistant to
various functionalities as required. Finally the response data is converted to speech
18
Voice Assistant- ‘Jarvisa’
19
Voice Assistant- ‘Jarvisa’
of voice recognition, its response time, etc.h ence different tests are performed to checks its
individual and integrated functionalities.
The main goal is to identify issues with the speech recognition and Natural Language
Understanding behavior of the assistant and your app. It consists of comprehensive testing of the
interaction model, creating a baseline set of results. These results, in turn, are the basis for making
improvements to the interaction model and the code of the skill – once revisions are completed,
additional tests are run to ensure everything is working as expected.
Test ID: T1
Test Objective: To make sure that the system response time is efficient.
Description:
21
Voice Assistant- ‘Jarvisa’
Time is very critical in a voice-based system. As we are not typing inputs, we are speaking
them. The system must also reply within seconds, User must get an instant reply to a given
query. However, in this system response time depends on the speed of the internet
connection.
Test ID: T2
Test Objective: To assure that answers retrieved by the system are accurate as per gathered
data. Description: A virtual assistant system is mainly used to get precise answers to any
question asked. Getting an answer in a moment is of no use if the answer is not correct.
Accuracy is of utmost importance in a virtual assistant system. It should accurately complete
any task or answer any question asked by the user
Test ID: t3
Description:
There are times when, we ask the assistant to make a google search, youtube search, etc.
22
Voice Assistant- ‘Jarvisa’
At this point approximation in answering is required, Both while listening and during text-to-
speech conversion. This is because our assistant is on a based o voice search mechanism.
Getting exact values in such cases is undesirable
Expected result: It should make proper speech recognition and text-to-speech comversion
Test ID: t4
Description:
The Assistant Should be able to access web pages and perform all the browser automation
like opening a new tab, closing tabs, opening web history, etc. However, the query will be
answered based on the speed of the internet connection.
Test ID: t5
Test Objective:
Description:
23
Voice Assistant- ‘Jarvisa’
The Assistant Should be able to access Online youtube videos. Also, it should be able to
access Local pc videos. It should perform all the video automation features like playing the
video, pausing it ,muting it, changing the modes of the screen etc.
Expected result: It should give relevant responses for Youtube and videos.
Test ID: 6
Description:
The Assistant Should be able to Perform system automation features. These features include
Launching settings,dealing with windows manager,opening widgets, date and time,etc.Also,
it should be able to access Local pc videos. It should perform all the video automation
features like playing the video, pausing it ,muting it, changing the modes of the screen etc.
Test status:Passed
Test ID: 6
Description:
24
Voice Assistant- ‘Jarvisa’
The Assistant Should be able to Perform all the real time functionalities. These features
include location search,etc .It should open google maps on commanding.
Test status:Passed
2. RESULT ANALYSIS
7.1 OUTPUT
25
Voice Assistant- ‘Jarvisa’
The following pictures show the resulting jobs done when we ask our assistant to do any
• Opening Notifications
In the Output given below Jarvisa opens notifications You just have to command “Show
Notifications”, and it will prompt u the notification box.
Showing weather
Similarly, it will show weather When u’ll ask “What’s the Weather?”
26
Voice Assistant- ‘Jarvisa’
• Showing widgets
It will open widgets on commanding as “Open widgets”and also the windows manager
layouts when we say “Windows Manager” in command
27
Voice Assistant- ‘Jarvisa’
• Opening Settings
28
Voice Assistant- ‘Jarvisa’
Youtube Automation
Loading video
29
Voice Assistant- ‘Jarvisa’
Browser Automation
Opening website
• Searching location
30
Voice Assistant- ‘Jarvisa’
7.1 CONCLUSION
Through this voice assistant, we have automated various services using a single
line command. It eases most of the tasks of the user like searching the web,
retrieving weather forecast details, vocabulary help and medical related
queries. We aim to make this project a complete server assistant and make it
smart enough to act as a replacement for a general server administration. The
future plans include integrating David with mobile using React Native to
provide a synchronised experience between the two connected devices.
Further, in the long run, David is planned to feature auto deployment
supporting elastic beanstalk, backup files, and all operations which a general
Server Administrator does. The functionality would be seamless enough to
replace the Server Administrator with David.
31
Voice Assistant- ‘Jarvisa’
REFERENCES
1. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and David Sun. Spatial pyramid pooling in
deep convolutional networks for visual recognition. In European Conference on Computer
Vision, pages 346–361. Springer,2014
3. Jyoti Dalal and Sumiran Daiya “Image Processing Based Optical Character Recognition
Using MATLAB” International Journal of Engineering Sciences & Research Technology,
ISSN: 2277-9655, CODEN: IJESS7, Dalal et al., 7(5): May 2018
5. https://www.geeksforgeeks.org/voice-assistant-using-python/
6. https://www.miquido.com/blog/what-are-voice-assistants/
7. https://www.academia.edu/40229169/
PROJECT_REPORT_ON_VIRTUAL_ASSISTANT_SUBMITTED_BY_NAAZNEEN_
JATU
32
Voice Assistant- ‘Jarvisa’
33
Voice Assistant- ‘Jarvisa’
34
Voice Assistant- ‘Jarvisa’
35
Voice Assistant- ‘Jarvisa’
36
Voice Assistant- ‘Jarvisa’
37
Voice Assistant- ‘Jarvisa’
II
38