Unit 2: AI Applications and Methodologies

Chatbots
Chatbot is one of the applications of AI which simulate conversation with humans either through voice
commands or through text chats or both. We can define Chatbot as an AI application that can imitate a
real conversation with a user in their natural language. They enable communication via text or audio
on websites, messaging applications, mobile apps, or telephones.

Some interesting applications of Chatbot are Customer services, E-commerce, Sales and Marketing,
Schools and Universities etc.
Types of Chatbots
Chatbots can broadly be divided into two types:
1. Rule based Chatbot
2. Machine Learning (or AI) based Chatbot risk

1. Rule – based Chatbot

This is the simpler form of Chatbot which follows a set of pre-defined rules in responding to user’s questions.
For example, a Chatbot installed at a school reception area, can retrieve data from the school’s archive to
answer queries on school fee structure, course offered, pass percentage, etc.
Rule based Chatbot is used for simple conversations, and cannot be used for complex conversations.
Machine Learning (or AI) based Chatbot

Such Chatbots are advanced forms of chatter-bots capable of holding complex conversations in real-
time. They process the questions (using neural network layers) before responding to them. AI based
Chatbots also learn from previous experience and reinforced learning and it keeps on evolving.
AI Chatbots are developed and programmed to meet user requests by furnishing suitable and relevant
responses. The challenge, nevertheless lies in aligning the requests to the most intelligent and closest
response that would satisfy the user.
Natural Language processing (NLP)

Every time you throw a question to Alexa, Siri or Google assistant, they use NLP to answer your
question.

1
NLP is a sub – area of Artificial Intelligence deals with the capability of software to process and analyse
human language, both verbal and written language.
Natural language processing has found applications in various fields which are listed as follows
1. Text Recognition (in an image or a video)
You might have seen / heard about cameras that read vehicle’s number plate

camera / machine translation

using NLP => KL 85 C 4780

Camera / machine captures the image of number plate, the image is transferred to neural network
layer of NLP application, NLP extracts the vehicle’s number from the image. However, correct
extraction of data also depends on the quality of image.
2. Summarization by NLP
NLP not only can read and understand the paragraphs or full article but can summarize the article into
a shorter narrative without changing the meaning. It can create the abstract of entire article. There are
two ways in which summarization takes place – one in which key phrases are extracted from the
document and combined to form a summary (extraction-based summarization) and the other in
which the source document is shortened (abstraction-based summarization).

3. Information Extraction
Information extraction is the technology of finding a specific information in a document or searching
the document itself. It automatically extracts structured information such as entities, relationships
between entities, and attributes describing entities from unstructured sources.
4. Speech processing
The ability of a computer to hear human speech and analyse and understand the content is called
speech processing. When we talk to our devices like Alexa or Siri, they recognize what we are saying to
them.
1.1 Computer Vision (CV): Getting started
Have you ever heard the term Computer Vision often abbreviated as CV? In very simple terms CV is a
field of study that enables the computers to “see”. It is a subfield of AI and involves extraction of
information from digital images like the videos and photographs, analysing, and understanding the
content thereof. CV has been most effective in the following:
 Object detection
 Optical Character Recognition
 Fingerprint Recognition
2
Source: https://towardsdatascience.com/everything-you-ever-wanted-to-know-about-computer-vision-heres-a-look-why-it-s-so-
awesome-e8a58dfb641e & https://nordicapis.com/8-biometrics-apis-at-your-fingertips/

‘AI for Ocean’ by Code.org is designed to quickly introduce students to machine learning, a branch
of artificial intelligence and is based on the idea that machines can recognise patterns, make sense
of data and make decisions with very little human involvement. Over the years machine learning
and computer vision have come to work very closely. Machine learning has also improved the
effectiveness of computer vision. Students will explore how training data is used to enable a machine
to see and classify the objects.
Activity Plan
There are 8 levels and you will be allowed not more than an hour to complete all the eight levels. And
it is not mandatory for students to complete all eight levels to understand the concept of computer
vision. Although it is recommended that students complete all eight levels).
Activity Outcome
Gaining a basic overview of AI and computer vision.
Activity details
Level 1:
The first level is talking about a branch of AI called Machine Learning (ML). This level is explaining
about the example of ML in our daily life like email filters, voice recognition, auto-complete text and
computer vision.

3
Level 2 – 4:
Students can proceed through the first four levels on their own or with a partner. In order to program
AI, use the buttons to label an image as either "Fish" or "Not Fish". Each image and its label become
part of the data which is used to train the Computer Vision Model. Once the AI model gets properly
trained, it will classify an image as ‘Fish’ and ‘Not Fish’. Fish will be allowed to live in the ocean but
‘Not-Fish’ will be grabbed and stored in the box, to be taken out of ocean/river later.
At the end of Level-4, we should explain to students how AI identifies objects using the training data,
in this case fish is our training data. This ability of AI to see and identify an object is called ‘Computer
Vision’. Similar to the case of humans where they can see an object and recognise it, an AI application
(using camera) can also see an object and recognise it.
Advance learners may go ahead with the level 5 and beyond.
Quick Question?

Based on the activity you just did, answer the following questions:

Q1: Can we say ‘computer vision’ acts as the eye of a machine / robot?
Q2: In the above activity, why did we have to supply many examples of fish before the AI model
actually started recognizing the fish?
Q3: Can a regular camera see things?
Q4: Let me check your imagination – You have been tasked to design and develop a prototype of a
robot to clean your nearby river/ ponds. Can you use your imagination and write 5 lines about features
of this ‘future’ robot?
With the supporting materials provided in the section above, you must have developed a reasonably
good understanding of Computer Vision. Let us reinforce the learning by quickly going over it again in
brief.
Computer vision is a sub-set of Artificial Intelligence which enables the machines (robot/ any other
device with camera) to see and understand the digital images – photographs or videos. Computer
Vision has made it possible to make good use of the critical capabilities of AI by giving machines the
power of vision. Computer vision enables machines/ robots to inspect objects and accomplish certain
tasks making them useful for both homes and offices.

1.3.1. Computer Vision: How does computer see an image

Consider the image below:

4
Looking at the picture, the human eye can easily tell that a train/engine has crashed through the statin
wall (https://en.wikipedia.org/wiki/Montparnasse_derailment). Do you think the computer also views

the image the same was as humans? No, the computer sees images as a matrix of 2-dimensional array
(or three-dimensional array in case of a colour image).

The above image is a grayscale image, which means each value in the 2D matrix represents the
brightness of the pixels. The number in the matrix ranges between 0 to 255, wherein 0 represents
black and 255 represents white and the values between them is a shade of grey.

In the grayscale image, each pixel represents the brightness or darkness of the pixel, which means the
grayscale image is composed of only one channel*. But colour image is the right mix of three primary
colours (Red, Green and Blue), so a colour image will have three channels*.

1.3.2. Computer Vision: Primary Tasks

1.3.3.
There are primarily four tasks that Computer vision accomplishes:
1. Semantic Segmentation (Image Classification)
2. Classification + Localization
3. Object Detection
4. Instance Segmentation

5
1. Semantic Segmentation
Semantic Segmentation is also called the Image classification. Semantic segmentation is a process in
Computer Vision where an image is classified depending on its visual content. Basically, a set of classes
(objects to identify in images) are defined and a model is trained to recognize them with the help of
labelled example photos. In simple terms it takes an image as an input and outputs a class i.e. a cat,
dog etc. or a probability of classes from which one has the highest chance of being correct.
For human, this ability comes naturally and effortlessly but for machines, it’s a fairly complicated
process.
For example, the cat image shown below is the size of 248x400x3 pixels (297,600 numbers)

Though image classification model takes this image as an input and reports 4 possibilities, we can see
that it indicates the highest probability of a cat. Therefore, the classification model has to be refined
/trained on further to produce a single label ‘cat’ as output.

2. Classification and Localization

Once the object classified and labelled, the localization task is evoked which puts a bounding box
around the object in the picture. The term ‘localization’ refers to where the object is in the image. Say
we have a dog in an image, the algorithm predicts the class and creates a bounding box around the
object in the image.

6
3. Object Detection
When human beings see a video or an image, they immediately identify the objects present in them.
This intelligence can be duplicated using a computer. If we have multiple objects in the image, the
algorithm will identify all of them and localise (put a bounding box around) each one of them. You will
therefore, have multiple bounding boxes and labels around the objects.

4. Instance segmentation
Instance segmentation is that technique of CV which helps in identifying and outlining distinctly each
object of interest appearing in an image. This process helps to create a pixel-wise mask for each object
in the image and provides us a far more granular understanding of the object(s) in the image. As you
can see in the image below, objects belonging to the same class are shown in multiple colours.

1.2 Weather Prediction using AI

Weather forecasting deals with gathering the satellites data, identifying patterns in the observations
made, and then computing the results to get accurate weather predictions. This is done in real-time to
prevent disasters. Artificial Intelligence uses computer-generated mathematical programs and
computer vision technology to identify patterns so that relevant weather predictions can be made.
Scientists are now using AI for weather forecasting to obtain refined and accurate results, fast!

7
1. IBM Global High-resolution Atmospheric Forecasting System (IBM GRAF) is a high-precision global
weather model that updates hourly to provide a clearer picture of weather activity around the globe
2. Panasonic has been working on its weather forecasting model for years. The company makes
TAMDAR, a speciality weather sensor installed on commercial airplanes.

1.5. Price forecast for commodities

What is commodity? Commodities are the natural resources and agriculture products that come from
the earth. Some examples of these goods are wheat, cattle, soybeans, corn, oranges, various metals,
coal, cotton, oil etc.
The AI based commodity forecasting system can produce transformative results, such as:

1. The accuracy level would be much higher than the classical forecasting model

2. AI can work on broad range of data and due to which it can reveal new insights

3. AI model is not rigid like classical model therefore its forecasting is always based on the most recent
input.
1.6. Self-Driving car

Wikipedia defines the self-driving car as, " A self-driving car, also known as an autonomous vehicle
(AV), driverless car, robot car, or robotic car is a vehicle that is capable of sensing its environment and
moving safely with little or no human input.”
Self-driving cars combine a variety of sensors to perceive their surroundings, such
as radar, lidar, sonar, GPS, odometry and inertial measurement units.

1. Computer Vision: Computer vision allows the car to see / sense the surrounding. Basically, it uses:

 ‘Camera’ that captures the picture of its surrounding which then goes to a deep learning
model to processing the image. This helps the car to know when the light is red or where
there is a zebra crossing etc.

 ‘Radar’ – It is a detection system to find out the how far or close the other vehicles on the
road are

 ‘Lidar’ – It is a surveillance method with which the distance to a target is measured. The lidar
(Lidar emits laser rays) is usually placed in a spinning wheel on top of the car so that they can
spin around very fast, looking at the environment around them. Here you can see a Lidar
placed on top of the Google car

8
2. Deep Learning: This is the brain of the car which takes driving decisions on the information
gathered through various sources like computer visions etc.

 Robotics: The self-driven cars have a brain and vision but still its brain needs to connect
with other parts of the car to control and navigate effectively. Robotics helps transmit the
driving decisions (by deep learning) to steering, breaks, throttle etc.
 Navigation: Using GPS, stored maps etc. the car navigates busy roads and hurdles to reach
its destination.

2. Characteristics and Types of AI

 Artificial Intelligence is autonomous and can make independent decisions — it does not require
human inputs, interference or intervention and works silently in the background without the
user’s knowledge. These systems do not depend on human programming, instead they learn on
their own through data experiencing.
 Has the capacity to predict and adapt – Its ability to understand data patterns is being used for
future predictions and decision-making.
 It is continuously learning – It learns from data patterns.
 AI is reactive – It perceives a problem and acts on perception.
 AI is futuristic – Its cutting-edge technology is expected to be used in many more fields in future.

There are many applications and tools, being backed by AI, which has a direct impact on our daily life.
So, it is important for us to understand what kind of systems can be developed, in a broad sense, using
AI.
2.1. Data Driven AI
The recent development in cheap data storage (hard disk etc.), fast processors (CPU, GPU or TPU) and
sophisticated deep learning algorithm has made it possible to extract huge value from data and that
has led to the rise of data centric AI systems. Such AI systems can predict what will happen next based
on what they’ve experienced so far, very efficiently. At times these systems have managed to
outperform humans. Data driven AI systems, get trained with large datasets, before it makes
predictions, forecasts or decisions. However, the success of such systems depends largely on the
availability of correctly labelled large datasets.
Every AI system we know around, are data driven AI system.
2.2. Autonomous System
Autonomous system is a technology which understands the environment and reacts without human
intervention. The autonomous systems are often based on Artificial intelligence. For example, self-
driving car, space probe rover like Mars rover or a floor cleaning robot etc. are autonomous system. An
IoT device like a smart home system is another example of an autonomous system. Because it does
things like gather information about the temperature of a room, then uses that information to
determine whether the AC needs to be switched on or not, then executes a task to achieve the desired
result.
2.3. Recommendation systems
A recommendation system recommends or suggests products, services, information to users based on
analysis of data based on a number of factors such as the history, behaviour, preference, and interest
of the user, etc. It is a data driven AI which needs data for training. For instance, when you watch a
video on YouTube, it suggests many videos that match or suit the videos that you generally search for,
prefer, or have been watching in the past. So, in short it gathers data from the history of your activities
and behaviour. Let us now take another example. Flipkart recommends the types of products that you
normally prefer or buy. All these are classic examples of recommendation, but you must know that
9
Machine Learning (AI) is working behind all this to give you a good user experience.
3. Cognitive Computing (Perception, Learning, Reasoning)

2. The platform (Cognitive computing) uses

1.This is a platform based on Artificial Machine Learning, Reasoning, Natural Language
Intelligence and Signal processing. Processing (NLP) and Computer Vision to
compute results.

3.Cognitive computing improves human decision 4.Cognitive computing tries to mimic the human
making brain

Examples of cognitive computing software: IBM Watson, Deep mind, Microsoft Cognitive service etc.

3.1. Applications of Cognitive Computing

In mainstream usage, Cognitive Computing is used to aid humans in their decision-making process.
Some examples of Cognitive Computing and its applications include the treatment of disease/’ illness
by supporting medical doctors. For example, ‘The IBM Watson for Oncology’ has been used at the
‘Memorial Sloan Kettering Cancer Centre’ to provide oncologists with evidence-based treatment
alternatives for patients having cancer

4. AI and Society
AI is sure changing the world but at the same time there is also a lot of hype and misconceptions about
it. In order for citizens, businesses and the government to take full advantages of AI, it is imperative
that we have realistic view about the same.
AI will impact almost every walk of society, from health, security, culture, education, jobs and
businesses. And as with any change/development, AI also has positive and negative influence on
society and this depends on how we leverage the same.
Some of the key social benefits of AI are outlined below:
1. Healthcare
IBM Watson (An AI Tool by IBM) can predict development of a particular form of cancer up to 12
months before its onset with almost a 90% accuracy.
There are many such developments happening in the field of medical science. To control the outbreak
of CORONA virus in China, the country leaned on Artificial Intelligence (AI), Data Science, to track cases
and fight the pandemic. Our healthcare sectors are moving towards a future where Robots and AI
tools will work alongside doctors.
2. Transportation
Transportation is a field where artificial intelligence along with machine learning has given us major
innovations.
Autonomous vehicles like cars, trucks etc. use advanced AI capabilities that offer features like lane-
changing systems, automated vehicle guidance, automated braking, use of sensors and cameras for
collision avoidance, and analysing information in real time, thus saving human lives by reducing road
accidents.
10
3. Disaster Prediction
AI is considered as one of the best tools for prediction of natural occurrences. There is an AI model
that can almost perfectly predict the weather for the next couple of days, which was unimaginable
before the advent of AI.
4. Agriculture
Farming is a sector faced is full of multiple challenges such like unpredictable as unpredictable
weather, availability of natural resources, growing populations etc. With the help of AI, farmers can
now analyse a variety of factors things in real time such as weather conditions, temperature, water
usage or soil conditions collected from their farm. Real-time data analytics helps farmers to maximize
their crop yields and thus, in turn, their profits too.
Listed below are some of the challenges posed by AI
1. Integrity of AI
AI systems learn by analysing huge volumes of data. What are the consequences of using biased data
(via the training data set)) in favour of a particular class/section of customers or users?
In 2016, the professional networking site LinkedIn was discovered to have a gender bias in its system.
When a search was made for the female name ‘Andrea’, the platform would show
recommendations/results of male users with the name ‘Andrew’ and its variations. However, the site
did not show similar recommendations/results for male names. i.e. A search/query for the name
‘Andrew’ did not result in a prompt asking the users if he/she meant to find ‘Andrea’. The company
said this was due to a gender bias in their training data which they fixed later.
2. Technological Unemployment
Due to heavy automation, (with the advent of AI and robotics) some sets of people will lose their jobs.
These jobs will be replaced by intelligent machines. There will be significant changes in the workforce
and the market — there will be creation of some high skilled jobs however some roles and jobs will
become obsolete.

3. Disproportionate control over data

Data is the fuel to AI; the more data you have, the more intelligent machine you would be able to
develop. Technology giants are investing heavily in AI and data acquisition projects. This gives them an
unfair advantage over their smaller competitors.

4. Privacy
In this digitally connected world, privacy will become next to impossible. Numerous consumer
products, from smart home appliances to computer applications have features that makes them
vulnerable to data exploitation by AI. AI can be utilized to identify, track and monitor individuals across
multiple devices, whether they are at work, home, or at a public location. To complicate things further,
AI does not forget anything. Once AI knows you, it knows you forever!
Let us have a quick quiz now!
Question 1: The size of an image is represented as 600 x 300 x 3. What would be the type of the
image?
a) Jpeg Image
b) Grayscale image

11
c) Colour Image
d) Large image
Question 2: Which of the following have people traditionally done better than computers?
a) Recognizing relative importance
b) Detecting Emotions
c) Resolving Ambiguity
d) All of the above
Question 3: You have been asked to design an AI application which will prepare the ‘minutes of the
meeting’. Which part of AI will you use to develop your solution -
a) Computer Vision
b) Python Programming
c) Chatbot
d) Natural Language Processing (NLP)

12