Technical Seminar Report

on

Machine Learning Techniques in Mechanical

Engineering

Submitted in partial fulfillment of

the requirements for the award of the degree of

Bachelor of Engineering
in
Mechanical Engineering

Submitted by

Seggoju Raja Havish

bearing U.S.N

1SJ11ME090

Department of Mechanical Engineering

S.J.C. Institute of Technology, Chikkaballapur
P.B.No. 20, B.B.Road, Chickballapur - 562 101
 Project Report
on

Design and Development of Automatic Pesticide

Sprayer for Vineyards

Submitted in partial fulfillment of

the requirements for the award of the degree of

Bachelor of Engineering
in
Mechanical Engineering

Prajwal G.V. 1SJ11ME066

Sandesh Chakravarthy 1SJ11ME083
Seggoju Raja Havish 1SJ11ME090
Yashwanth Gowda 1SJ11ME118

Department of Mechanical Engineering

S.J.C. Institute of Technology, Chikkaballapur
P.B.No. 20, B.B.Road, Chickballapur - 562 101
 Department of Mechanical Engineering
S.J.C. Institute of Technology, Chikkaballapur

Certificate

This is to certify that, The Technical seminar on Machine Learning Techniques in Me-
chanical Engineering was presented at mechanical engineering department by Seggoju
Raja Havish bearing U.S.N 1SJ11ME090,in partial fulfillment for the award of B.E degree
in Mechanical Engineering of the Visveswaraya Technological University, Belgaum, during
2014-15. The Seminar report has been approved as it satisfies academic requirement in re-
spect of Technical seminar prescribed for B.E degree.

Maximum 50 Marks
marks obtained

Seminar coordinator H.O.D

Date:
 Contents
1 Introduction 1
1.1 What is Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Types of Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2.1 Supervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2.2 Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2.3 Recommender Systems . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Machine Learning Problems and Algorithms 2

2.1 Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1.1 Cost Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.1.2 Gradient Descent Algorithm . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 Logistic Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.1 Sigmoid Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2.2 Decision Boundary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.3 Simplified Cost Function . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.4 Multiclass Classification . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.2 Artificial Neural Networks (ANNs) . . . . . . . . . . . . . . . . . . . 8
2.3.3 Forward Propagation Intuition . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.4.1 k -means Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 Applied Machine Learning 12

3.1 Misfire Detection in IC Engine using Kstar Algorithm . . . . . . . . . . . . . 12
3.1.1 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.2 Feature Extraction and Selection . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2.1 Results and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.3 Machine Learning for the Improvement of Springback Modeling . . . . . . . 13
3.3.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.3.2 Machine Learning for updating FEM results . . . . . . . . . . . . . . 14
3.3.3 Results and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Conclusion 16
References16
1 Introduction
1.1 What is Machine Learning
It is a scientific discipline which deals with the construction of algorithms which can learn
from data.
What this means is that, given a data set {X 1 , X 2 , X 3 ...X m } where X i = {x1 , x2 , x3 ...xn },
and dependent variables {y 1 , y 2 , y 3 ...y m }, we can construct a hypothesis such that, for a given
X m+1 we can predict the behavior of y m+1 . Here {x1 , x2 , x3 ...xn } are called features of X i .
example :

x1 x2 x3 y
Specimen Length Diameter Maximum Load Maximum Load
in ’mm’ in ’mm’ (As det. by F.E.M.) in ’kN’ (Actual) in ’kN’
X1 Specimen 1 200 20 5.1 5.4
X2 Specimen 2 350 15 4.9 5.2
X3 Specimen 3 180 22 6.2 6.9
.. .. .. .. .. ..
. . . . . .
Xm Specimen ’m’ 350 20 5.4 5.6

Now let’s say we want to know the maximum allowable load on ’Specimen m+1’ where
we know the values of length,diameter and maximum load as determined by F.E.M then we
can use the algorithm to predict the actual maximum load.

1.2 Types of Machine Learning

1.2.1 Supervised Learning
The example shown above is that of supervised learning where a right answer exists, i.e, the
it is possible to comment on the accuracy of the prediction. Types of supervised learning
problems are-

a) Regression: In this type of machine learning problem the output is continuous valued,
like in the case of the above example where the maximum Load can vary continuously.

b) Classification:Here the output is discrete valued, i.e, there exist a finite number of possible
answers.
e.g. the problem of classifying a given picture as that of a man or a horse given a previous
data set.

1.2.2 Unsupervised Learning

In this type of machine learning problem the right answer is not known and the algorithm
just clusters the given data set. The characteristics of the various clusters are not known to
us in advance. The algorithm tries to find a hidden structure in the given data.

1
1.2.3 Recommender Systems
These are information filtering systems that seek to predict the ’rating’ or ’preference’ to-
wards an item. The YoutubeTM ’related videos’ is an example of this type of learning.

2 Machine Learning Problems and Algorithms

2.1 Linear Regression
Taking the same example as above:
m - number of training examples
X - inputs
x - feature
y - outputs
[X (i) , y (i) ] - ith training example.
hθ - hypothesis

Now, for simplicity, let us consider only one feature for the data set; the ratio of the
length to diameter.

Specimen x1 Maximum Load(kN)

Specimen 1 10 5.4
Specimen 2 23.33 5.2
Specimen 3 8.18 6.9
.. .. ..
. . .
Specimen ’m’ 17.5 5.6

Here, the hypothesis hΘ (x1 ) is given as:

hΘ (x1 ) = θ0 + θ1 x1
Here θ0 and θ1 are the parameters and they are chosen such that the value of hΘ (x1 ) is
as close to the value of y as possible.

2.1.1 Cost Function

This function is used to calculate the error of predictions for a given value of theta. The
optimal value of theta is at the minimum value of the cost function. The cost function for
univariate linear regression is given as:
m
X
1
J(θ0, θ1) = 2m [hθ (x(i)) − y (i)]2
i=1

2
2.1.2 Gradient Descent Algorithm
It is an algorithm devised to solve the linear regression problem. It calculates the optimal
value of the parameter set Θ = {θ0 , θ1 , θ2 ...θn }. This algorithm basically minimizes the value
of the cost function J(Θ).
The mathematical expression is given by:
θj := θj − α ∂θ∂ j J(Θ)

m
X
∂ ∂ 1
∂θj J(Θ) = ∂θj [ 2m [hΘ(x(i)) − y (i)]2]
i=1
Consider
 the given data, representing it in matrix format for computational simplicity:
1 x01    
1 x1  1 10 5.4
1
1 23.33 5.2

2
[X] = 1 x1  = 
     
[Θ] = θ θ [y] =

0 1
 .. ..  1 8.18  6.9
. . 
1 17.5 5.6
1 xm 1

The equation is given given by:

hΘ (X) = [X][Θ]

Applying gradient descent to this we get:

3
  m h
X i2 
∂ ∂ 1
∂θj J(Θ) = ∂θj 2m [X][ΘT ] − [y]
i=1

differentiating the equation -

m
X h i
∂ 1
∂θj J(Θ) = m XjT T
[X][Θ ] − [y]
i=1
Putting all these equations together and generalizing for the entire theta matrix, we get -

m  iT
1 X h
Θ=Θ−α X T [X][ΘT ] − [y]
m i=1

Learning rate(α) : This constant represents increment or decrement in period. It

controls the step size to be taken while checking for various values of theta.

From graph 1 we can see that the partial differential term takes care of calculating the
minimum point even if the value overshoots.
The value of α need not be varied manually since the amount of decrement made gradually
decreases along the curve for the same decrement in value of θ, as seen from graph 2.

2.2 Logistic Regression

An example of the classification type machine learning problem. Here we have predefined
sets and we have to classify the given new example into one of them.

4
 Take the same example as we have been using with categories {0, 1}, let us apply a load
on the specimen and if the specimen doesn’t fail it is classified as 0 else as 1.

y ∈ {0, 1}
Let us say we apply a load ’c’, failure occurs when actual maximum load(or allowable
load ) is less than ’c’.

0 if (allowable load) > c
y=
1 if (allowable load) < c, i.e, failure

Here {0, 1} represent the probability of the failure of specimen. We arbitrarily assume
the threshold value of 0.5 in this case. If the probability of failure is grater than the threshold
it is assumed to fail. In this case we can’t use the expression for hΘ as it is, since we need
to calculate the probability of the specimen’s failure.

2.2.1 Sigmoid Function

The sigmoid function gives an estimated probability of y belonging to any of the given
classifications.

1
g(z) = 1+exp(−z)

1
T


hΘ (X) = g XΘ = 1+exp(−XΘT )

Here g(z) is called the sigmoid function. A graph of the sigmoid function is given below.

5
 For a value of z more than 0.5 g(z) is approximately equal to 1, and at -0.5 equal to 0.
E.g. applying the above formulas if we get hΘ (X) = 0.7, it implies there is a 70% chance of
the specimen failing.

2.2.2 Decision Boundary

The decision boundary gives us a better understanding of the problem. It is basically a
graphical representation of the given training examples and there attributes along with the
separation boundary.For given data set with two features {x1 , x2 } we have -

2.2.3 Simplified Cost Function

The cost function calculations used in linear regressions will lead to non-convex curves, the
minima of which are difficult to calculate. For this reason we use a simplified cost function
given by -
m
X  
1 i (i)
J(Θ) = m cost hΘ (x ), y
i=1

6
 

−log(hΘ (X)) if y = 1
cost hΘ (xi ), y (i)
=
−log(1 − hΘ (X)) if y = 0
The cost equation can also be written as -

cost hΘ (xi ), y (i) = −ylog(hΘ (X)) − (1 − y)log(1 − hΘ (X))
m
X
1
−y i log(hΘ (xi )) − (1 − y i )log(1 − hΘ (xi ))


J(Θ) = m
i=1

We use the same gradient descent algorithm to solve for optimum values of Θ.
Differentiating and replacing the gradient descent algorithm we get.
m
X
T
Θ := Θ − α hΘ (xi ) − y i X
i=1

2.2.4 Multiclass Classification

We have seen in the previous example the problem of classifying a given specimen into two
groups, either the specimen fails or not. Many a times we come across machine learning
problems in which there are more than two groups. i,e - y ∈ {0, 1, 2, 3 . . . , k}. In this case
we run the logistic regression problem in the following manner -

1. Make the given data set into k copies

2. In each new training set assign the 0 y 0 value of the group to be determined as 1 and
others as 0.

3. Run the logistic regression algorithm and get the independent values of Θ for each
classification.
Θ ∈ Rk×n
.

4. When a new specimen is given to examine, the values of the sigmoid function for all
0 0
k sets of Θ values are calculated and the set where maximum probability occurs is
assigned to it.

The following graphs illustrate the concept.

7
 The graph on the right represents the given data, the data is then relabeled as shown in
graph 2 to calculate the values of θ for the first group. We repeat the same for each group.

2.3 Neural Networks

2.3.1 Introduction
Neuron is an electrically excitable cell that processes and transmits information through elec-
trical and chemical signals. These signals between neurons occur via synapses, specialized
connections with other cells. Neurons can connect to each other to form neural networks.
Neurons are the core components of the nervous system, which includes the brain and the
spinal cordwhich together comprise the central nervous system (CNS).

We try to mimic this operation of neurons. We

view the brain in general as having only one al-
gorithm - the one present in the neurons, and the
operation of the whole body is based on similar
types of brain cells.
For example in the case of ’Human echolocation’,
the normally visual part of the human brain is uti-
lized for auditory processing.
Following the same pattern of a common algorithm
we develop Artificial Neural Networks.

2.3.2 Artificial Neural Networks (ANNs)

Artificial neural networks (ANNs) are a family of statistical learning algorithms inspired by
biological neural networks (the central nervous systems of animals, in particular the brain)

8
and are used to estimate or approximate functions that can depend on a large number of
inputs and are generally unknown. Artificial neural networks are generally presented as
systems of interconnected ”neurons” which can compute values from inputs, and are capable
of machine learning as well as pattern recognition.
ANNs essentially consist of hidden layers, these are like new features which the network
calculates for itself as per the outputs of the training set. These values of the hidden layer
can later be utilized for predictions.

On the left side is a single neuron element where x0 , x1 and x2 are the inputs or the
features, and using given algorithm the output is calculated.
On the right side is shown a full neural network with two hidden layers.

a21 = g x0 θ10
1 1 1


+ x1 θ11 + x2 θ12
a22 = g x0 θ20
1 1 1


+ x1 θ21 + x2 θ22

..
.

Here,
aji - activation unit where subscript i represents the activation unit number in that layer,
and j represents the layer number.
j
θi1,i2 - weight controlling the function mapping. i1, i2 are the activation unit number and
the wight number respectively and j is the hidden layer number.
g(z) - sigmoid function.

2.3.3 Forward Propagation Intuition

This section will help in better understanding of how Neural networks function using the
logical AND operator.
Let -

9
 hΘ (x) = g (θ0 + θ1 x1 + θ2 x2 )
where -
   
θ0 −30
θ1  =  20 
θ2 20
Putting these two together -

hΘ (x) = g (−30 + 20x1 + 20x2 )

drawing the truth table for this -
x1 x2 z hΘ (x) = g(z)
0 0 30 0
0 1 -10 0
1 0 -10 0
1 1 10 1

This clearly indicates how neural networks can be used to implement logical AND gate.
Similarly logical OR, NOT, XNOR and XOR can also be implemented.

2.4 Unsupervised Learning

As discussed before unsupervised learning refers to learning from unlabeled data sets. The
given data set only consists of features and no 0 y 0 column as in the first example. It is the
job of the algorithm to make useful clusters or groups. Some of the algorithms commonly
used are - k-means algorithm, Principal Component Analysis, Hierarchical clustering, etc.

2.4.1 k -means Clustering

This algorithm aims to distribute the ’n’ readings into ’k’ clusters based on the similarity of
the features. The algorithm consists primarily of two steps -

1. Cluster assignment(assignment step)

2. Reassignment of cluster centroids(update step)

Both these are explained using the image shown, in the first step ’k’ random readings are
marked as cluster centroids and the other (n-k) readings are assigned to one of these clusters
based on their proximity to the centroids (with respect to features).
In the second step the mean average of all the members of a cluster is calculated and
that cluster center is reassigned to it. This is done to all k clusters.
These two steps are repeated until optimal condition is reached.

10
The above example takes six iterations of the algorithm to find the optimal clusters. These
two steps can be mathematically represented as -

Repeat {

1. for i = 0 to m
c(i) =index (from 1 to K) for cluster
centroids closest to x(i) .

minkx(i) − µk k2

2. for k = 1 to K
µk =avg(mean) of points assigned
to cluster.

µk = mean(x1 + x2 + x3 ...)

11
3 Applied Machine Learning
In this section applications of machine learning and its problem solving techniques in me-
chanical engineering are discussed. The two applications discussed are -

1. Misfire detection in I.C. engines using k-star algorithm.

2. Machine learning for improvement of spring-back modeling environment prediction

3.1 Misfire Detection in IC Engine using Kstar Algorithm

In case of misfire, uncombusted fuel accesses the exhaust system, where its combustion in the
catalytic converter can cause a dangerous increase in temperature, damaging the catalytic
converter due to thermal overload. It may also lead to increased emission of hydrocarbons.
Thus it becomes necessary to detect misfire to prevent pollution and wastage of fuel.
Methods using the torsional vibration signal of the crankshaft and the acceleration signal
of the engine head were developed to detect misfire. crankshaft speed for misfire detection.
Some works have already been carried out in this field using features such as cylinder devi-
ation torque, instantaneous angular velocity and instantaneous exhaust manifold pressure.
These methods can be costly and cumbersome.

3.1.1 Experimental setup

The experimental setup comprises mainly of

the spark ignition IC engine test rig with pro-
visions made in order to manually cause mis-
fire in a particular cylinder and the data acqui-
sition system. An accelerometer is attached
on the engine which measures the vibration
signals. The acquired signal is then passed
through an ADC to collect the required data
from which features are extracted. Fig shows
the basic flow of the steps involved in the whole
process.

12
3.2 Feature Extraction and Selection
The vibration signals acquired cannot be used directly. Useful information needs to be ex-
tracted from them which will represent the signal. Here statistical features are extracted. De-
scriptive statistics for a particular signal gives a wide range of parameters namely mean, stan-
dard error, median, mode, standard deviation, sample variance, kurtosis, skewness, range,
minimum, maximum, sum and count.

3.2.1 Results and Conclusion

The diagonal of the confusion matrix shown
indicates the number of correct detections, this
shows that the k-star algorithm can detect
misfires with 100% accuracy and the position
of misfires in a four cylinder engine with 86%
accuracy. This makes it ideal for industry ap-
plications.

3.3 Machine Learning for the Improvement of Springback Mod-

eling
Springback of sheet metal products is a very complex problem. It is the result of the stress
state in the material following the forming process and means a change of shape in the sheet
metal forming product after the withdrawal of the forming forces.
Knowledge in the evaluation of the simulation forming process of conventional mild steel
using the FEM is at a relatively high-level, more difficulties are encountered in the FEM
simulations of new, advanced materials. It is impossible to completely overcome springback
but it can be limited and monitored by setting the optimal forming parameters.

3.3.1 Experimental Setup

The basis of the research was an experiment called the draw/bend test. This is a simplified
method of deep drawing process, where sheet metal is pulled over the radius.

13
 After the experiment progression, springback occurred. It is the result of flexible angular
changes and sidewall curl. Their sum is expressed in corner. For three different materials,
three parameters such as tool radius, restraint force and friction were analyzed.

3.3.2 Machine Learning for updating FEM results

For improving our FEM springback results a Weka workbench was used. The Weka work-
bench is a collection of ML algorithms and data preprocessing tools. In this case, the ML
system based on experiment and simulation draw/bend test, six different algorithms were
used: Linear Regression, Isotonic Regression, Least Medium Square, SMO, Gaussian Pro-
cesses and Multilayer Perceptron.

3.3.3 Results and Conclusion

Figure demonstrates the experimental springback results and the results of FEM simulations
for the HSLA material. The FEM simulation results deviate slightly from the experimental
values. The good correlation between the FEM simulation results and the experiment is
also expressed in ML, since the presence of attribute FEM simulation result significantly
improves the correlation model of the ML algorithms presented overall. Compared to the
solely FEM method an even higher correlation with the FEM combined Linear Regression
and Multilayer Perceptron method was reached.

14
Furthermore, a common ML model was created. More attributes (four material parameters)
and data sets were available for this procedure and despite the consideration of all data
sets they obtained a pretty well learned model. This can be seen in Figure below, where
correlation coefficients for all six ML algorithms are presented. If we take a look at the
results of the correlation coefficients we can see that the last three tested methods (SMO,
Gaussian Processes and Multilayer Perceptron) are more suitable for springback prediction.
One of the reasons is definitely in the nonlinearity of the complex springback phenomenon
and these methods are able to model this kind of phenomena. However, combination with
the FEM raises the correlation coefficient to a higher level compared to the solely FEM
method.

15
4 Conclusion
Machine learning and its various algorithms can provide a alternative to solving the existing
problems in mechanical engineering. The large amounts of data required for application
of M.L are available. In a world where computational intelligence in rapidly growing the
many branches in mechanical engineering can flourish with the support of M.L. This being
a relatively new field offers many research opportunities.

References
[1] Prof. Andrew Ng, Stanford CS dept. http://cs229.stanford.edu/materials.html

[2] Dezelak, M. Pahole, I. Ficko, M. Brezocnik, Advances in Production Engineering &

Management 7 (2012) 1, 17-26

[3] Anish Bahri, V.Sugumaran, S. Babu Devasenapati, 12 Pages, 8 Figures, 4 Tables. In-
ternational Journal of Research in Mechanical Engineering, 2013

[4] Google search and images

[5] Wikipedia

16