Symmetry: Identification of Apple Leaf Diseases Based On Deep Convolutional Neural Networks
Symmetry: Identification of Apple Leaf Diseases Based On Deep Convolutional Neural Networks
Symmetry: Identification of Apple Leaf Diseases Based On Deep Convolutional Neural Networks
Article
Identification of Apple Leaf Diseases Based on Deep
Convolutional Neural Networks
Bin Liu 1,2, *,† ID
, Yun Zhang 1,† , DongJian He 2,3 and Yuxiang Li 4
1 College of Information Engineering, NorthWest A&F University, No. 22, Xinong Road,
Yangling 712100, China; yunzhang@nwsuaf.edu.cn
2 Key Laboratory of Agricultural Internet of Things (NorthWest A&F University), Ministry of Agriculture,
Yangling 712100, China; hdj168@nwsuaf.edu.cn
3 College of Mechanical and Electronic Engineering, NorthWest A&F University, No. 22, Xinong Road,
Yangling 712100, China
4 School of Information Technology, Henan University of Science and Technology, No. 263, Kaiyuan Avenue,
Luoyang 471023, China; liyuxiang@haust.edu.cn
* Correspondence: liubin0929@nwsuaf.edu.cn; Tel.: +86-187-1048-7673
† These authors contributed equally to this work and should be considered co-first authors.
Abstract: Mosaic, Rust, Brown spot, and Alternaria leaf spot are the four common types of apple
leaf diseases. Early diagnosis and accurate identification of apple leaf diseases can control the
spread of infection and ensure the healthy development of the apple industry. The existing research
uses complex image preprocessing and cannot guarantee high recognition rates for apple leaf
diseases. This paper proposes an accurate identifying approach for apple leaf diseases based on deep
convolutional neural networks. It includes generating sufficient pathological images and designing
a novel architecture of a deep convolutional neural network based on AlexNet to detect apple leaf
diseases. Using a dataset of 13,689 images of diseased apple leaves, the proposed deep convolutional
neural network model is trained to identify the four common apple leaf diseases. Under the hold-out
test set, the experimental results show that the proposed disease identification approach based on
the convolutional neural network achieves an overall accuracy of 97.62%, the model parameters
are reduced by 51,206,928 compared with those in the standard AlexNet model, and the accuracy
of the proposed model with generated pathological images obtains an improvement of 10.83%.
This research indicates that the proposed deep learning model provides a better solution in disease
control for apple leaf diseases with high accuracy and a faster convergence rate, and that the image
generation technique proposed in this paper can enhance the robustness of the convolutional neural
network model.
Keywords: apple leaf diseases; deep learning; convolutional neural networks; image processing
1. Introduction
China is a modern agricultural country supplying fruit products, wherein the fruit planting area is
relatively large. Due to its rich nutritional and medicinal value, the apple has become one of China’s four
major fruits. However, diseases in apple leaves cause major production and economic losses, as well as
reductions in both the quality and quantity of the fruit industry output. Apple leaf disease detection has
received increasing attention for the monitoring of large apple orchards.
Traditionally, plant disease severity is scored with visual inspection of plant tissues by trained
experts [1], which leads to expensive cost and low efficiency. With the popularization of digital cameras
and the advance of information technology in agriculture, cultivation and management expert systems
have been widely used, greatly improving the production capacity of plants [2]. However, for the
“expert system”, extraction and expression characteristics of pests and diseases mainly depend on
expert experience, which easily leads to a relative lack of standardization and low recognition rates.
With the popularity of machine learning algorithms in computer vision, in order to improve the
accuracy and rapidity of the diagnosis results, researchers have studied automated plant disease
diagnosis based on traditional machine learning algorithms, such as random forest, k-nearest neighbor,
and Support Vector Machine (SVM) [3–12]. However, because the classification features are selected
and adopted based on human experience, these approaches improved the recognition accuracy, but the
recognition rate is still not high enough and is vulnerable to artificial feature selection. Developed
in recent years, the deep convolutional neutral network approach is an end-to-end pipeline that can
automatically discover the discriminative features for image classification, whose advantages lie in the
use of shared weights to reduce the memory footprint and improve performance, and the direct input
of the image into the model. Until now, the convolutional neural network has been regarded as one of
the best classification approaches for pattern recognition tasks. Inspired by the breakthrough of the
convolutional neutral network in image-based recognition, the use of convolutional neural networks
to identify early disease images has become a new research hotspot in agricultural informatization.
In [13–20], convolutional neural networks (CNNs) are widely studied and used in the field of crop
disease recognition. These studies show that convolutional neural networks have not only reduced the
demand of image preprocessing, but also improved the recognition accuracy.
In this paper, we present a novel identifying approach for apple leaf diseases based on a deep
convolutional neural network. The CNN-based approach faces two difficulties. First of all, apple
pathological images are not sufficient for the training model. Second, determining the best structures
of the network model is fundamentally a more difficult task.
The main contributions of this paper are summarized as follows:
• In order to solve the problem of insufficient apple pathological images, this paper proposes
a training image generation technology based on image processing techniques, which can
enhance the robustness and prevent overfitting of the CNN-based model in the training process.
Natural apple pathological images are first acquired and are then processed in order to generate
sufficient pathological images using digital image processing technologies such as image rotation,
brightness adjustment, and PCA (Principal Component Analysis) jittering to disturb natural
images; these are able to simulate the real environment of image acquisition, and expanding the
pathological images gives an important guarantee of generalization capability of the convolutional
neural network model.
• A convolutional neural network is first employed to diagnose apple leaf diseases; the end-to-end
learning model can automatically discover the discriminative features of the apple pathological
images and identify the four common types of apple leaf diseases with high accuracy. By analyzing
the characteristics of apple leaf diseases, a novel deep convolutional neural network model based
on AlexNet is proposed; the convolution kernel size is adjusted, fully-connected layers are
replaced by a convolutional layer, and GoogLeNet’s Inception is applied to improve the feature
extraction ability.
The experimental results show that the proposed CNN-based model achieves an accuracy of
97.62% on the hold-out test set, which is higher than the other traditional models. Compared with
the standard AlexNet model, the parameters of the proposed model are significantly decreased by
51,206,928, demonstrating the faster convergence rate. Using the dataset of 13,689 synthetic images
of diseased apple leaves, the identification rate increases by 10.83% over that of the original natural
images, proving the better generalization ability and robustness.
The remainder of this paper is organized as follows. In Section 2, related work is introduced and
summarized. In Section 3, based on apple leaf pathological image acquisition and image processing
technology, sufficient training images are generated. Section 4 describes the novel deep convolutional
neural network model. Section 5 analyzes the experimental results provided by the identification
approach to apple leaf diseases based on CNNs. Finally, this paper is concluded in Section 6.
Symmetry 2018, 10, 11 3 of 16
2. Related Work
Plant diseases are a major threat to production and quality, and many researchers have made
various efforts to control these diseases. In the last few years, traditional machine learning algorithms
have been widely used to realize disease detection. In [6], Qin et al. proposed a feasible solution
for lesion image segmentation and image recognition of alfalfa leaf disease. The ReliefF method
was first used to extract a total of 129 features, and then an SVM model was trained with the most
important features. The results indicated that image recognition of the four alfalfa leaf diseases can
be implemented and obtained an average accuracy of 94.74%. In [7], Rothe et al. presented a pattern
recognition system for identifying and classifying three cotton leaf diseases. Using the captured dataset
of natural images, an active contour model was used for image segmentation and Hu’s moments
were extracted as features for the training of an adaptive neuro-fuzzy inference system. The pattern
recognition system achieved an average accuracy of 85%. In [8], Islam et al. presented an approach that
integrated image processing and machine learning to allow the diagnosis of diseases from leaf images.
This automated method classifies diseases on potato plants from ‘Plant Village’, which is a publicly
available plant image database. The segmentation approach and utilization of an SVM demonstrated
disease classification in over 300 images, and obtained an average accuracy of 95%. In [9], Gupta
proposed an autonomously modified SVM-CS (Cuckoo Search) model to identify the healthy portion
and disease. Using a dataset of diseases containing plant leaves suffering from Alternaria Alternata,
Cercospora Leaf Spot, Anthracnose, and Bacterial Blight, along with healthy leaf images, the proposed
model was trained and optimized using the concept of a cuckoo search. However, identification and
classification approaches of these studies are semiautomatic and complex, and deal with a series of
image processing technologies. At the same time, it is very difficult to accurately detect the specific
disease images without extracting and designing the appropriate classification features depending
heavily on expert experience.
Recently, several researchers have studied plant disease identification based on deep learning
approaches. In [16], Lu et al. proposed a novel identification approach for rice diseases based on deep
convolutional neural networks. Using a dataset of 500 natural images of diseased and healthy rice
leaves and stems, CNNs were trained to identify 10 common rice diseases. The experimental results
showed that the proposed model achieved an average accuracy of 95.48%. In [17], Tan et al. presented
an approach based on CNN to recognize apple pathologic images, and employed a self-adaptive
momentum rule to update CNN parameters. The results demonstrated that the recognition accuracy
of the proposal was up to 96.08%, with a fairly quick convergence. In [18], a novel cucumber leaf
disease detection system was presented based on convolutional neural networks. Under the fourfold
cross-validation strategy, the proposed CNN-based system achieved an average accuracy of 94.9%
in classifying cucumbers into two typical disease classes and a healthy class. The experimental
results indicate that a CNN-based model can automatically extract the requisite classification features
and obtain the optimal performance. In [14], Sladojevic et al. proposed a novel approach based
on deep convolutional networks to detect plant disease. By discriminating the plant leaves from
their surroundings, 13 common different types of plant diseases were recognized by the proposed
CNN-based model. The experimental results showed that the proposed CNN-based model can reach
a good recognition performance, and obtained an average accuracy of 96.3%. In [19], Mohanty et al.
developed a CNN-based model to detect 26 diseases and 14 crop species. Using a public dataset of
54,306 images of diseased and healthy plant leaves, the proposed model was trained and achieved
an accuracy of 99.35%. These studies show that convolution neural networks have been widely applied
to the field of crop and plant disease recognition, and have obtained good results. However, on the
one hand, these studies only apply the CNN-based models to identify crop and plant diseases without
improving the model. On the other hand, so far, the CNN-based model has not been applied to the
identification of apple leaf diseases; a novel CNN-based model developed by our research group is
applied to detect apple leaf diseases in this paper.
Symmetry 2018, 10, 11 4 of 16
Symmetry 2018, 10, 11 4 of 16
3.
3. Generating
Generating Apple
Apple Pathological
Pathological Training
TrainingImages
Images
x0 = r cosα
(1)
Symmetry 2018, 10, 11 y0 = r sin α 5 of 16
The horizontal mirror symmetry takes takes a vertical line in an image as the axis, and all pixels of the
image are exchanged. Assume that wwrepresents thethe
represents width,
width,and
andthat anan
that arbitrary point’s
arbitrary coordinate
point’s is
coordinate
0 ) ; after mirror symmetry, the point’s coordinate is (w− x0 , y0 ) .
(isx0 ,(yx0 ), ;yafter mirror symmetry, the point’s coordinate is ( w − x ,
0 0y ) .
0
As shown in Figure 2, a pathological image is rotated and mirrored to generate four pathological
As shown in Figure 2, a pathological image is rotated and mirrored to generate four pathological
images, in which the angle of rotation consists of 90 ◦ , 180 ◦ and 270 ◦ , and mirror symmetry includes
images, in which the angle of rotation consists of 90 °, 180 °and 270 °, and mirror symmetry includes
horizontal symmetry.
horizontal symmetry.
For the
For the alteration
alteration of
of image
image brightness,
brightness, thethe RGB
RGB value
value of
of pixels
pixels needs
needs to
to be
be increased
increased or
or decreased
decreased
randomly. Assume
randomly. AssumethatthatV0Vrepresents
0 representsthe the original
original RGBRGB value,
value, V adjusted
V is the is the adjusted value,
value, and and d
d represents
the brightness transformation factor. The formula is as follows:
represents the brightness transformation factor. The formula is as follows:
VV==VV00×
×(1
(1++dd) ). . (4)
(4)
For the
For the contrast
contrast of
of the
the image,
image, the
the larger
larger RGB
RGB value
value is
is increased
increased and
and the
the smaller
smaller RGB
RGB value
value is
is
reduced, based on the median value of the brightness. The formula is as follows:
reduced, based on the median value of the brightness. The formula is as follows:
V = i +(V −i)×(1+ d) (5)
V = i + (V00− i ) × (1 + d) (5)
where
i represents
where i represents thethe median
median value
value of theofbrightness,
the brightness, and
and the theparameters
other other parameters
have thehave
samethe same
meaning
meaning
as as in Equation
in Equation (4). (4).
In addition to the direction disturbance and
In addition to the direction disturbance and light
light disturbance,
disturbance, Gaussian
Gaussian noise
noise and
and PCA
PCA jittering
jittering
are also employed on the original apple leaf pathological
are also employed on the original apple leaf pathological images.images.
The original images are disturbed by Gaussian noise, which can simulate the possible noise
caused by equipment in the image acquisition process. First, random numbers are generated consistent
Symmetry 2018, 10, 11 6 of 16
Symmetry 2018, 10, 11 6 of 16
The original images are disturbed by Gaussian noise, which can simulate the possible noise
The2018,
Symmetry
caused original
by 10, 11 images
equipment in are
thedisturbed by Gaussian
image acquisition noise, First,
process. whichrandom
can simulate the
numbers possible
are noise
6 of 16
generated
caused bywith
consistent equipment in the
a Gaussian image acquisition
distribution. Then, the process. random numbers First, random are added numbers
to theare generated
original pixel
consistent
values of with
the a
image,Gaussian
which distribution.
finally Then,
compresses the
the random
sums to numbers
the [0, 255] are added
interval. to the original pixel
with a Gaussian distribution. Then, the random numbers are added to the original pixel values of the
values
PCAof the image,
jittering waswhich finallyby
proposed compresses the sums is tousedthe [0, to255] interval.
image, which finally compresses the Alex
sumsettoal.the [22],
[0, and
255] interval. reduce overfitting. In this paper, it
PCA jittering was proposed by Alex et al. [22], and is used to reduce
R
= [reduce
I xy to G
I xyBoverfitting.
B In this paper, it
]TT , the following
I xyR , I xyG ,overfitting.
is applied to expand
PCA jittering wasthe dataset. by
proposed ToAlex
eachet RGB image
al. [22], and pixel
is used quantity
In this paper, it is
is applied to expand the dataset. To each RGB image pixel I xy =R[ I xyG, I xy B, I xyT] , the following quantity
is added:to expand the dataset. To each RGB image pixel Ixy = [ Ixy , Ixy , Ixy ] , the following quantity is
applied
is added:
added:
[P1 ,P2 ,P3 ][a1λ1, a2λ2 , a3λ3 ]TTT (6)
[P[P
1 ,P,P
2 ,P,P
1 ][1aλλ1 ,,aa2 λλ2 ,, aa3λλ3]]
3 ][ a
2 3 1 1 2 2 3 3
(6)
(6)
where PiPandi and λ itheare
λi are iththe ith feature
feature vectorvector and eigenvalue
and eigenvalue of the of
3 ×the 3 × 3 covariance
3 covariance matrixmatrix
of RGBof pixel
RGB
where
values,
P i and
respectively,
λ i are the ith feature
is theαrandom
and αiand vector and
variable. eigenvalue of the 3 × 3 covariance matrix of RGB
pixel values, respectively, i is the random variable.
pixelThe lightrespectively,
values, disturbanceand α i is theinrandom
is illustrated Figure 3, with the six pathological images generated by
variable.
The light disturbance is illustrated in Figure 3, with the six pathological images generated by
adjusting the
The light brightness, contrast,
disturbancecontrast, and
is illustratedsharpness.
in Figure 3,Figure 4 visualizes
with 4the the Gaussian
six pathological imagesnoise and PCA
generated by
adjusting the brightness, and sharpness. Figure visualizes the Gaussian noise and PCA
jittering
adjusting against
the the pathological
brightness, contrast,image.
and sharpness. Figure 4 visualizes the Gaussian noise and PCA
jittering against the pathological image.
jittering against the pathological image.
different2018,
Symmetry from10, the
11first convolutional layer’s kernel size of 11 × 11 × 3 in the standard AlexNet.7 of The
16
second convolutional layer filters the noise with 256 kernels of size 5 × 5 × 48; response-normalization
layers follow the first two convolutional layers, which are themselves followed by max-pooling layers.
first convolutional layer’s kernel size of 11 × 11 × 3 in the standard AlexNet. The second convolutional
The third convolutional layer has 384 kernels with a size of 3 × 3 × 256 connected to the (normalized,
layer filters the noise with 256 kernels of size 5 × 5 × 48; response-normalization layers follow the first
pooled) outputs of the second convolutional layer. The fourth layer is filtered with 384 kernels of size
two convolutional layers, which are themselves followed by max-pooling layers. The third convolutional
3 × 3 × 192, and the fifth layer has 256 kernels with a size of 2 × 2 × 192 to improve the ability to extract
layer has 384 kernels with a size of 3 × 3 × 256 connected to the (normalized, pooled) outputs of the
small features, which is also different from the standard AlexNet, and is then followed by a max-
second convolutional layer. The fourth layer is filtered with 384 kernels of size 3 × 3 × 192, and the fifth
pooling layer.
layer has 256 kernels with a size of 2 × 2 × 192 to improve the ability to extract small features, which is
After AlexNet Precursor, an architecture named Cascade Inception is designed including two
also different from the standard AlexNet, and is then followed by a max-pooling layer.
max-pooling layers and two Inception structures. The first max-pooling layer is applied to filter the
After AlexNet Precursor, an architecture named Cascade Inception is designed including two
noise of feature maps generated by AlexNet Precursor, and the two Inceptions then extract the
max-pooling layers and two Inception structures. The first max-pooling layer is applied to filter the
optimal discrimination features from multidimension analysis. Feature maps before the first
noise of feature maps generated by AlexNet Precursor, and the two Inceptions then extract the optimal
Inception are input into the second Inception’s concatenation layer, which prevents some of the
discrimination features from multidimension analysis. Feature maps before the first Inception are
features being filtered by these two Inceptions. Meanwhile, the sixth convolutional layer followed by
input into the second Inception’s concatenation layer, which prevents some of the features being
the Cascade Inception has 4096 kernels with a size of 1 × 1 × 736, which replaces the first two fully
filtered by these two Inceptions. Meanwhile, the sixth convolutional layer followed by the Cascade
connected layers of the standard AlexNet. The fully connected layer is adjusted to predict four classes
Inception has 4096 kernels with a size of 1 × 1 × 736, which replaces the first two fully connected
of apple leaf diseases, and the final layer is a four-way Softmax layer.
layers of the standard AlexNet. The fully connected layer is adjusted to predict four classes of apple
leaf diseases, and the final layer is a four-way Softmax layer.
Table 1. Related parameters of the convolutional neural network (CNN)-based model.
More
More specifically,
specifically, the
the convolution
convolution layer,
layer, pooling
pooling layer,
layer, activation
activation function,
function, and
and Softmax
Softmax layer
layer in
in
the novel CNN-based model are described below.
the novel CNN-based model are described below.
where λ means the λth layer, kij represents the convolutional kernel, b j is the bias, and M j is a set of
input feature maps [16].
s j = maxαi (8)
i ∈R j
where R j represents pooling region j in feature map a, and i is the index of each element within it;
s denotes the pooled feature maps [25].
1 m k
m i∑
J (θ) = − [ ∑ 1{y(i) = j} log p(y(i) = j| x (i) ; θ)]. (10)
=1 j =1
n o
The training database is denoted ( x (1) , y(1) ), ( x (2) , y(2) ), . . . , ( x (m) , y(m) ) , yi ∈ {1, 2 . . . , k}.
In Softmax regression, the possibility of classifying x into category j is
θTj x (i)
(i ) (i ) e
p(y = j| x ; θ) = T (i )
. (11)
∑kl=1 eθl x
4.5. GoogLeNet’s
4.5. GoogLeNet’s Inception
Inception
AA special
special structure
structure named
named Inception
Inception is is the main feature of GoogLeNet; it it keeps
keeps the sparse
network structure,
network structure, andand utilizes an intensive matrix of high-performance computing. As shown
As shown in
Figure 6,
Figure 6,the
theInception
Inceptionconsists
consistsofofparallel
parallel 1 ×1 1,×31,× 3,
3× and3, 5and 5 × 5 convolutional
× 5 convolutional layers
layers as wellas
as well as
a max-
a max-pooling
pooling layer tolayer to aextract
extract varietya of
variety of features
features in parallel.in parallel.
Then, 1 ×Then, 1 × 1 convolution
1 convolution layers are
layers are added for
added for dimensionality
dimensionality reduction.reduction.
Finally, aFinally, a filter concatenation
filter concatenation layer concatenates
layer concatenates simplysimply the output
the output of all
of all parallel
these these parallel
layerslayers
[23]. [23].
Figure
Figure 6. GoogLeNet’s
GoogLeNet’s Inception.
Inception.
4.6.
4.6. Nesterov’s
Nesterov’s Accelerated
Accelerated Gradient
Gradient (NAG)
(NAG)
The
The training
training process
process of
of convolutional
convolutional neural
neural networks
networks includes
includes two
two stages
stages of
of aa feedforward
feedforward pass
pass
and
and aa backpropagation
backpropagation pass.
pass. In
In the
the backpropagation
backpropagation pass pass stage,
stage, the
the error
error is
is passed
passed from
from higher
higher layers
layers
to
to lower
lower layers.
layers.
Stochastic GradientDescent
Stochastic Gradient Descent (SGD)
(SGD) is used
is used to update
to update the weight
the weight for convolutional
for convolutional neural
neural networks.
networks.
However, However,
SGD may SGD lead tomay thelead to optimum”
“local the “local problem.
optimum” To problem. To problem,
solve this solve thisNesterov’s
problem,
Nesterov’s Accelerated Gradient (NAG) is applied to train the proposed CNN-based
Accelerated Gradient (NAG) is applied to train the proposed CNN-based model. As a convex model. As a
convex optimization
optimization algorithm,
algorithm, NAG has NAG has arate
a higher higher rate of convergence.
of convergence. The updated Theweights
updatedare weights are
calculated
calculated based on the last iteration, as shown in the Equations
based on the last iteration, as shown in the Equations (13) and (14): (13) and (14):
d i = β d i −1 + α g ( θ − β d i −1 ) (13)
di = βdi−1 + αg(θ − βdi−1 ) (13)
This paper uses the four common types of apple leaf diseases to evaluate the novel CNN-based
model. These apple pathological images were collected in Qingyang County, Gansu Province, China
and Baishui County, Shanxi Province, China. After application of image processing techniques,
the generated pathological images constituted a dataset of 13,689 images of diseased apple leaves;
the numbers of various pathological images in the training and test sets are presented in Table 3.
the above
Symmetrysection
2018, 10, showed,
11 the characteristics of Mosaic and Brown spot are very different11from of 16 the
others, and recognition rates of 100.00% and 99.29 were achieved for Mosaic and Brown spot,
respectively.
However,However, Alternaria
Alternaria leaf leaf spot similar
spot is extremely is extremely
to Rustsimilar to Rust
in geometric in geometric
features, features,
which leads which
to their
leadslower
to their lower rates.
recognition recognition
As shown rates. As shown
in Figure in Figure
7, pathological 7, pathological
features in the originalfeatures inextracted
image are the original
imageby the proposed model with GoogLeNet Inception, which improves the automatic feature extraction the
are extracted by the proposed model with GoogLeNet Inception, which improves
automatic feature extraction
in a multidimensional space.inHence,
a multidimensional space. Hence,
the proposed CNN-based the aproposed
model has CNN-based
better identification model
ability
with regards to apple leaf diseases.
has a better identification ability with regards to apple leaf diseases.
(a) (b)
Figure 7. Activation
Figure visualization:
7. Activation visualization:(a)
(a)original
original image; and(b)
image; and (b)the
thelearned
learned weights
weights by the
by the firstfirst
layer.layer.
Table
Table4.
4. Recognition performance.
Recognition performance.
Table
Table5.5.Confusion matrixfor
Confusion matrix forour
ourwork.
work.
Predicted
Predicted
ClassClass
Alternaria Leaf
Alternaria Leaf SpotSpot Mosaic
Mosaic Rust Rust Brown
Brown Spot Spot Accuracy (%)
Accuracy (%)
Alternaria
Alternaria
Leaf Spot
689
689 3 3 58 58 0 0 91.87% 91.87%
Leaf Spot
Ground Ground
Truth Mosaic
Mosaic 0 763 0 0 100%
Rust 3 0 0 763 437 0 0 0 99.32% 100%
Truth Brown Spot 5 1 0 842 99.29%
Rust 3 0 437 0 99.32%
Brown Spot 5 1 0 842 99.29%
In addition, in this experiment, the five CNN-based models were selected to research the
variation of accuracy
In addition, in thiswith the trainingthe
experiment, epochs.
five As shown in Figure
CNN-based models8, thewere
four classical
selectedconvolutional
to research the
variation of accuracy with the training epochs. As shown in Figure 8, the four classicalepochs
neural networks and the proposed model begin to converge after a certain number of and
convolutional
finally achieve their optimal recognition performance. On the whole, the training processes of
neural networks and the proposed model begin to converge after a certain number of epochs and
GoogLeNet, VGGNet-16, and AlexNet are basically stable after 10 epochs, and other models have
finally achieve their optimal recognition performance. On the whole, the training processes of
a satisfactory convergence after 15 epochs. Because of the use of transfer learning, VGGNet-16 has
GoogLeNet, VGGNet-16, and AlexNet are basically stable after 10 epochs, and other models have a
a faster convergence speed than other CNN-based models, and achieves an accuracy of 96.32%.
satisfactory
Because convergence
GoogLeNet uses after 15 epochs.
Inception Because of the
structures—which use
have of transfer
a strong abilitylearning, VGGNet-16
for feature learning—tohas a
faster convergence speed than other CNN-based models, and achieves an accuracy
extract the features of apple leaf diseases, the convergence point of GoogLeNet occurs at 10 epochs. of 96.32%.
Because GoogLeNet
Compared to otheruses Inception
neutral structures—which
networks, have a strong
AlexNet uses a traditional networkability for feature
structures, whichlearning—to
results
extract the features
in slower of apple leaf
convergence—the diseases,
starting the
point of convergence
convergence ispoint of GoogLeNet
at about 16 epochs. As occurs at 10 epochs.
for ResNet-20,
the strategy
Compared of batch
to other normalization
neutral networks,improves
AlexNetitsuses
convergence rate, and
a traditional the model
network reaches convergence
structures, which results in
at 20
slower epochs. In our work,
convergence—the Inception
starting pointstructures, removingisthe
of convergence at partial
about fully connected
16 epochs. layers,
As for and the the
ResNet-20,
NAGofoptimization
strategy algorithm are
batch normalization used for the
improves proposed model;
its convergence compared
rate, and thewithmodelthe standard AlexNet,
reaches convergence
the proposed model improves the convergence speed of the network model, begins to converge at
at 20 epochs. In our work, Inception structures, removing the partial fully connected layers, and the
about 14 epochs, and provides higher recognition accuracy for apple leaf diseases.
NAG optimization algorithm are used for the proposed model; compared with the standard AlexNet,
the proposed model improves the convergence speed of the network model, begins to converge at
about 14 epochs, and provides higher recognition accuracy for apple leaf diseases.
Symmetry 2018, 10, 11 12 of 16
Symmetry 2018, 10, 11 12 of 16
Figure
Figure 8.
8. Convergence comparison.
In addition, to prevent overfitting in this paper, various methods were performed. First, various
In addition, to prevent overfitting in this paper, various methods were performed. First, various
digital image processing technologies such as image rotation, mirror symmetry, brightness
digital image processing technologies such as image rotation, mirror symmetry, brightness adjustment,
adjustment, and PCA jittering, were applied to the natural training images to simulate the real
and PCA jittering, were applied to the natural training images to simulate the real acquisition
acquisition environment and increase the diversity and quantity of the apple pathological training
environment and increase the diversity and quantity of the apple pathological training images,
images, which can prevent the overfitting problem and make the proposed model generalize better
which can prevent the overfitting problem and make the proposed model generalize better during
during the training process. Second, the response-normalization layers were used in the proposed
the training process. Second, the response-normalization layers were used in the proposed model to
model to achieve local normalization, which is thought of as an effective way to prevent the
achieve local normalization, which is thought of as an effective way to prevent the overfitting problem.
overfitting problem. Third, by replacing some of the fully connected layers with convolution layers,
Third, by replacing some of the fully connected layers with convolution layers, the proposed model has
the proposed model has fewer training parameters than the standard CNN-based model, and this
fewer training parameters than the standard CNN-based model, and this scheme aids the generation
scheme aids the generation of the model.
of the model.
5.3.
5.3. Computational
Computational Resources
Resources
In computationaltheory,
In computational theory, thethe simplest
simplest computational
computational resources
resources are computation
are computation time, thetime, the
number
number of parameters necessary to solve a problem, and memory space [28]. In
of parameters necessary to solve a problem, and memory space [28]. In this section, the computational this section, the
computational resource
resource comparisons ofcomparisons of fournetwork
four classic neural classic neural
modelsnetwork
and the models
proposedand the proposed
model model
are analyzed in
are analyzed in Table 6. Compared with other learning models, although
Table 6. Compared with other learning models, although the proposed model is trained with batchthe proposed model is
trained
size 128,with batch
it takes thesize 128,
least it takes
video the least
memory spacevideo
for memory
training. space for training.
The standard The standard
AlexNet AlexNet
has the minimum
has the minimum training time among all the CNN-based models. Compared
training time among all the CNN-based models. Compared with AlexNet, the proposed model not with AlexNet, the
proposed model not only has a similar training time, but also achieves a higher
only has a similar training time, but also achieves a higher accuracy of recognition. As for ResNet-20, accuracy of
recognition.
it has the leastAslearned
for ResNet-20,
weights,itbut hastakes
the least
up a learned weights,
great deal but takes
of memory spaceup a great
and takes deal of memory
the longest time
space and takes the longest time to train parameters. Overall, the proposed
to train parameters. Overall, the proposed model uses less computational resources to build the model uses less
model
computational
and acquires the best accuracy in identifying apple leaf diseases, which allows it to meet the needsleaf
resources to build the model and acquires the best accuracy in identifying apple of
diseases, which
real production. allows it to meet the needs of real production.
Table 6. Computational
Table 6. Resource Comparison.
Computational Resource Comparison.
Figure 9. Contrasts
Contrasts of optimization algorithm Stochastic Gradient Descent (SGD) and Nesterov’s
Nesterov's
Accelerated Gradient (NAG).
At
At the
the same
same time,
time, as
as shown
shown inin Figure
Figure 9,
9, the
the result
result shows
shows that
that the
the training
training process
process of
of the
the proposed
proposed
model
model almost converged after 25 epochs, and finally achieved an accuracy of 97.62%. The reason for
almost converged after 25 epochs, and finally achieved an accuracy of 97.62%. The reason for
this
this phenomenon
phenomenon is is that
that the
the learning
learning rate
rate decreases
decreases gradually
gradually to
to almost
almost the
the invariant,
invariant, which
which greatly
greatly
reduces the updated
reduces the updated amplitude
amplitude of of parameters.
parameters. Furthermore,
Furthermore, the
the learned
learned weights
weights of the CNN-based
of the CNN-based
model were updated to almost the state of convergence. After this, the learned weights only had a
minor update. As a result, the training process was basically stable after 25 epochs.
model were updated to almost the state of convergence. After this, the learned weights only had
a minor update. As a result, the training process was basically stable after 25 epochs.
Figure 10.
Figure 10. The influence
influence of
of expanded
expanded dataset.
dataset.
From the known results, as shown in Figure 10, this phenomenon is mainly due to the following
From the known results, as shown in Figure 10, this phenomenon is mainly due to the following
reasons: (1) the expanded dataset generated by various digital image processing technologies gives
reasons: (1) the expanded dataset generated by various digital image processing technologies gives
the proposed CNN-based model more chances to learn appropriate layered features; (2) the diversity
the proposed CNN-based model more chances to learn appropriate layered features; (2) the diversity
of images in the expanded image dataset helps to fully train the learned weights in the CNN-based
of images in the expanded image dataset helps to fully train the learned weights in the CNN-based
model, while the smaller image dataset lacks diversity and is going to cause the overfitting problem;
model, while the smaller image dataset lacks diversity and is going to cause the overfitting problem;
and (3) the preprocessing of the images simulates the real acquisition environment of the apple
and (3) the preprocessing of the images simulates the real acquisition environment of the apple
pathological images and, as a consequence, the CNN-based model has better identification ability for
pathological images and, as a consequence, the CNN-based model has better identification ability for
natural apple pathological images obtained from the apple orchard. The experimental result shows
natural apple pathological images obtained from the apple orchard. The experimental result shows
that the expanding dataset contributes to enhancing the generalization ability of the proposed model.
that the expanding dataset contributes to enhancing the generalization ability of the proposed model.
6. Conclusions
6. Conclusions
This paper
This paperhashasproposed
proposed a novel
a novel deepdeep convolutional
convolutional neuralneural
network network model
model to to accurately
accurately identify
identify apple leaf diseases, which can automatically discover the discriminative
apple leaf diseases, which can automatically discover the discriminative features of leaf diseases features of leaf
diseases
and and
enable anenable an end-to-end
end-to-end learninglearning
pipelinepipeline
with highwith high accuracy.
accuracy. In orderIntoorder to provide
provide adequateadequate
apple
apple pathological images, firstly, a total of 13,689 images were generated by image
pathological images, firstly, a total of 13,689 images were generated by image processing technologies, processing
technologies,
such such
as direction as direction
disturbance, disturbance,
light disturbance, light
anddisturbance,
PCA jittering. and PCA jittering.
Furthermore, Furthermore,
a novel structure ofa
anovel
deepstructure of a deep
convolutional neuralconvolutional
network based neural network
on the AlexNet based
modelon was
the AlexNet
designedmodel was designed
by removing partial
by removing partial full connected layers, adding pooling layers, introducing
full connected layers, adding pooling layers, introducing the GoogLeNet Inception structure the GoogLeNet
into
Inception
the proposedstructure
networkintomodel,
the proposed network
and applying themodel, and applying
NAG algorithm the NAGnetwork
to optimize algorithm to optimize
parameters to
network parameters to accurately identify
accurately identify the apple leaf diseases. the apple leaf diseases.
The novel
The novel deep
deep convolutional
convolutional network
network model
model was
was implemented
implemented in in the
the Caffe
Caffe framework
framework on on the
the
GPU platform.
GPU platform. Using
Using aa dataset
dataset of of 13,689
13,689 images
images of of diseased
diseased leaves,
leaves, the
the proposed
proposed model
model was
was trained
trained
to detect apple leaf diseases. The results demonstrated are satisfactory, and the proposed model can
obtain a recognition accuracy of 97.62%, which is higher than the recognition abilities of other models.
Compared with the standard AlexNet model, the proposed model reduces the number of parameters
greatly, has a faster convergence rate, and the accuracy of the proposed model with supplemented
images is increased by 10.83% compared with the original set of diseased leaf images. The results
indicated that the proposed CNN-based model can accurately identify the four common types of
Symmetry 2018, 10, 11 15 of 16
to detect apple leaf diseases. The results demonstrated are satisfactory, and the proposed model can
obtain a recognition accuracy of 97.62%, which is higher than the recognition abilities of other models.
Compared with the standard AlexNet model, the proposed model reduces the number of parameters
greatly, has a faster convergence rate, and the accuracy of the proposed model with supplemented
images is increased by 10.83% compared with the original set of diseased leaf images. The results
indicated that the proposed CNN-based model can accurately identify the four common types of apple
leaf diseases with high accuracy, and provides a feasible solution for identification and recognition of
apple leaf diseases.
In addition, due to the restriction of biological growth laws and the current season in which
the apple leaves have fallen, other diseases of apple leaves are difficult to collect. In future work,
for the sake of detecting apple leaf diseases in real time, other deep neural network models, such as
Faster RCNN (Regions with Convolutional Neural Network), YOLO (You Only Look Once), and SSD
(Single Shot MultiBox Detector), are planned to be applied. Furthermore, more types of apple leaf
diseases and thousands of high-quality natural images of apple leaf diseases still need to be gathered
in the plantation in order to identify more diseases in a timely and accurate manner.
Acknowledgments: We are grateful for anonymous reviewers’ hard work and comments that allowed us to
improve the quality of this paper. This work is supported by National Natural Science Foundation of China
through Grant No. 61602388, by the Natural Science Basic Research Plan in Shaanxi Province of China under
Grant No. 2017JM6059, by the China Postdoctoral Science Foundation under Grant No. 2017M613216, by the
Postdoctoral Science Foundation of Shaanxi Province of China under Grant No. 2016BSHEDZZ121, and the
Fundamental Research Funds for the Central Universities under Grants No. 2452015194 and No. 2452016081.
Author Contributions: Bin Liu contributed significantly to proposing the idea, manuscript preparation and
revision, and providing the research project. Yun Zhang contributed significantly to conducting the experiment,
and manuscript preparation and revision. Dongjian He and Yuxiang Li helped perform the analysis with
constructive discussions.
Conflicts of Interest: We declare that we have no financial or personal relationships with other people or
organizations that can inappropriately influence our work; there is no professional or other personal interest of
any nature or kind in any product, service, and/or company that could be construed as influencing the position
presented in, or the review of the manuscript entitled, “Identification of Apple Leaf Diseases Based on Deep
Convolutional Neural Networks”.
References
1. Dutot, M.L.; Nelson, M.; Tyson, R.C. Predicting the spread of postharvest disease in stored fruit, with application
to apples. Postharvest Biol. Technol. 2013, 85, 45–56. [CrossRef]
2. Zhao, P.; Liu, G.; Li, M.Z. Management information system for apple diseases and insect pests based on GIS.
Trans. Chin. Soc. Agric. Eng. 2006, 22, 150–154.
3. Es-Saady, Y.; Massi, I.E.; Yassa, M.E.; Mammass, D.; Benazoun, A. Automatic recognition of plant leaves
diseases based on serial combination of two SVM classifiers. In Proceedings of the 2nd International
Conference on Electrical and Information Technologies, Tangiers, Morocco, 4–7 May 2016; pp. 561–566.
4. Padol, P.B.; Yadav, A.A. SVM classifier based grape leaf disease detection. In Proceedings of the 2016
Advances in Signal Processing, Pune, India, 9–11 June 2016; pp. 175–179.
5. Sannakki, S.S.; Rajpurohit, V.S.; Nargund, V.B.; Kumar, A.; Yallur, P.S. Diagnosis and classification of grape
leaf diseases using neural networks. In Proceedings of the 4th International Conference on Computing,
Tiruchengode, India, 4–6 July 2013; pp. 1–5.
6. Qin, F.; Liu, D.X.; Sun, B.D.; Ruan, L.; Ma, Z.; Wang, H. Identification of alfalfa leaf diseases using image
recognition technology. PLoS ONE 2016, 11, e0168274. [CrossRef] [PubMed]
7. Rothe, P.R.; Kshirsagar, R.V. Cotton leaf disease identification using pattern recognition techniques.
In Proceedings of the 2015 International Conference on Pervasive Computing, Pune, India, 8–10 January 2015;
pp. 1–6.
8. Islam, M.; Dinh, A.; Wahid, K.; Bhowmik, P. Detection of potato diseases using image segmentation and
multiclass support vector machine. In Proceedings of the 30th IEEE Canadian Conference on Electrical and
Computer Engineering, Windsor, ON, Canada, 30 April–3 May 2017; pp. 1–4.
Symmetry 2018, 10, 11 16 of 16
9. Gupta, T. Plant leaf disease analysis using image processing technique with modified SVM-CS classifier.
Int. J. Eng. Manag. Technol. 2017, 5, 11–17.
10. Dhakate, M.; Ingole, A.B. Diagnosis of pomegranate plant diseases using neural network. In Proceedings
of the 5th National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics,
Patna, India, 16–19 December 2015; pp. 1–4.
11. Gavhale, M.K.R.; Gawande, U. An overview of the research on plant leaves disease detection using image
processing techniques. J. Comput. Eng. 2014, 16, 10–16.
12. Wang, G.; Sun, Y.; Wang, J.X. Automatic image-based plant disease severity estimation using deep learning.
Comput. Intell. Neurosci. 2017, 2017, 1–8. [CrossRef] [PubMed]
13. Mohanty, S.P.; Hughes, D.; Salathe, M. Inference of Plant Diseases from Leaf Images through Deep Learning. arXiv.
2016. Available online: https://www.semanticscholar.org/paper/Inference-of-Plant-Diseases-from-Leaf-Images-
throu-Mohanty-Hughes/62163ff3cb2fbbf5361e340f042b6c288d3b8e6a (accessed on 28 December 2017).
14. Sladojevic, S.; Arsenovic, M.; Anderla, A.; Culibrk, D.; Stefanovic, D. Deep neural networks based recognition
of plant diseases by leaf image classification. Comput. Intell. Neurosci. 2016, 2016. [CrossRef] [PubMed]
15. Hanson, A.M.J.; Joy, A.; Francis, J. Plant leaf disease detection using deep learning and convolutional neural
network. Int. J. Eng. Sci. Comput. 2017, 7, 5324–5328.
16. Lu, Y.; Yi, S.J.; Zeng, N.Y.; Liu, Y.; Zhang, Y. Identification of rice diseases using deep convolutional neural
networks. Neurocomputing 2017, 267, 378–384. [CrossRef]
17. Tan, W.X.; Zhao, C.J.; Wu, H.R. CNN intelligent early warning for apple skin lesion image acquired by
infrared video sensors. High Technol. Lett. 2016, 22, 67–74.
18. Kawasaki, Y.; Uga, H.; Kagiwada, S.; Iyatomi, H. Basic study of automated diagnosis of viral plant diseases
using convolutional neural networks. In Proceedings of the 12th International Symposium on Visual
Computing, Las Vegas, NV, USA, 12–14 December 2015; pp. 638–645.
19. Mohanty, S.P.; Hughes, D.P.; Marcel, S. Using deep learning for image-based plant disease detection.
Front. Plant Sci. 2016, 7, 1419. [CrossRef] [PubMed]
20. Fuentes, A.; Yoon, S.; Kim, S.C.; Park, D.S. A robust deep-learning-based detector for real-time tomato plant
diseases and pests recognition. Sensors 2017, 17, 2022. [CrossRef] [PubMed]
21. Heisel, S.; Kovačević, T.; Briesen, H.; Schembecker, G.; Wohlgemuth, K. Variable selection and training set
design for particle classification using a linear and a non-linear classifier. Chem. Eng. Sci. 2017, 173, 131–144.
[CrossRef]
22. Krizhevsky, A.; Sutskever, I.; Hinton, G.E. ImageNet classification with deep convolutional neural
networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems,
Lake Tahoe, NV, USA, 3–6 December 2012; pp. 1097–1105.
23. Szegedy, C.; Liu, W.; Jia, Y.Q.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, R.
Going deeper with convolutions. In Proceedings of the 2014 IEEE Conference on Computer Vision and
Pattern Recognition, Columbus, OH, USA, 24–27 June 2014; pp. 1–9.
24. Giusti, A.; Dan, C.C.; Masci, J.; Gambardella, L.M.; Schmidhuber, J. Fast image scanning with deep
max-pooling convolutional neural networks. In Proceedings of the 20th IEEE International Conference on
Image Processing, Melbourne, Australia, 15–18 September 2013; pp. 4034–4038.
25. Zeiler, M.D.; Fergus, R. Stochastic pooling for regularization of deep convolutional neural networks.
arXiv, 2013.
26. Ruder, S. An overview of gradient descent optimization algorithms. arXiv. 2016. Available online:
https://arxiv.org/abs/1609.04747 (accessed on 28 December 2017).
27. Bahrampour, S.; Ramakrishnan, N.; Schott, L.; Shah, M. Comparative study of caffe, neon, theano, and torch
for deep learning. In Proceedings of the 2016 International Conference on Learning Representations,
San Juan, PR, USA, 2–5 May 2016; pp. 1–11.
28. Liu, B.; He, J.R.; Geng, Y.J.; Huang, L.; Li, S. Toward emotion-aware computing: A loop selection approach
based on machine learning for speculative multithreading. IEEE Access 2017, 5, 3675–3686. [CrossRef]
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).