Improvement in The Speed of Training A Neural Network Using Sampled Training

Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.
com Volume 2, Issue 4, April 2013 ISSN 2319 - 4847
Improvement in the Speed of Training a Neural Network Using Sampled Training

Advait S. Bhatt1, Harikrishna B. Jethva2
1 PG Student, 2Assosciate Professor, Department of Computer Engineering, L.D. College of Engineering, Gujarat Technological University, Ahmedabad
ABSTRACT
Today the most important use of neural network is in classifying the data. The neural network is trained for classification by giving some input-output pairs which is also known as supervised learning. As training a network is a part of the whole classification process, the time required for training should be as small as possible. In this paper, we propose a scheme where sampling of the training set is done for reducing the time required for training the network and hence the overall time required for classification is reduced.
Keywords: Supervised Learning, Learning rate, Classification, Training Time, Neural Network, Sampled Training.
1. INTRODUCTION
Classification is basically a set of activities responsible for deriving a model that categorizes and describes classes of data and concepts, whose sole purpose is to determine and predict the classes of objects who have no label as [1]. The model derived can be shown in various forms, such as classification (IF-THEN) rules, decision trees, mathematical formulae, or neural network. A neural network, when used for classification, is typically a collection of neuron-like processing units with weighted connections between the units. Fig 1 shown below is the example of a Feed-Forward Artificial Neural Network. Every neural network has one input and one output unit along with zero or more number of hidden units. The input unit receives the input, the output unit generates the classification result and the hidden unit performs the processing. There are some very important components those make up an artificial neuron. These include weighting factor, summation function, transfer function, error function, error and back-propagated value and learning function. The most important among these is the learning function. Its purpose is to modify the variable connection weights when inputs are given to each processing element based on some neural based algorithm.
Figure 1 A Feed-Forward Neural Network There are a large number of neural network algorithms that are implemented for different applications. In each type of algorithms, some type of learning technique is always used. The time required for training a network should be very less so that the actual data is classified in less time. Further this paper is divided into following sections, section II is about training a neural network. Section III is about the proposed scheme of sampled training in a neural network, section IV is about the advantages of the proposed scheme and section V is the conclusion so derived after the study of the two schemes.
2. TRAINING A NEURAL NETWORK

2.1 Process of Classification Every classification technique basically consists of two phases: the learning or the training phase and the other is the output or the testing phase. In the learning phase, the training data whose data objects have a known class label are given as input to the model so as to train the model. Once the model has been trained, the original data set that is to be classified is given as input to the model. The model based on information learned from the training data classifies the test data into appropriate class labels. 2.2 Batch Mode Training (Existing Method)
Volume 2, Issue 4, April 2013
Page 241
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com Volume 2, Issue 4, April 2013 ISSN 2319 - 4847
In a feed-forward neural network, training of the network is done by providing the input-output pairs to the network and then generating the model which can then be used for classification of the test data. In case of the existing system, the whole of the training set is given as input for training the network. So the total time required for performing the classification includes the time required for training and the time required for classifying the test data based on the model generated. It has found that for better error gradient estimations, more number of input vectors for each gradient calculation should be used [2]. But the down side of using more input vectors per weight update is that the gradient has to be calculated for each input vector used [3]. In the training process it is the gradient calculations that take the longest. Using multiple input vectors quickly lengthens the training times to intolerable levels.
Figure 2 Flow-Chart of Batch Training in Neural Networks Figure 2 shows the flowchart of the batch mode training scheme that is being used in the existing system. As it can seen from the figure above, in batch mode of training a neural network, the complete training set is used for training. After the training has been accomplished and the model generated, the test data is given as input for classifying the unknown class labels. This is how the existing system works. But the problem with this training is that the time required to train a neural network is more as the whole training set is used [4]. So, in order to reduce the training time, we propose a new scheme where the training set is sampled into smaller chunks.
3. SAMPLED TRAINING (PROPOSED SCHEME)

In order to reduce the overall time of classifying the test data, the time required for training the network should be reduced greatly. In this scheme, we sample the whole training set into fixed size data sets and then give them as input to the network for training. As shown in the figure below, the training set is sampled into smaller training sets having fixed number of tuples. Once the network is trained, test data is given for classification. If the performance of the model generated is as per the user specified levels, the network is trained and if not then the remaining samples of the training set are given as input further training of the network. This process continues until the samples are available for training or until the user specified accuracy is obtained.
Figure 3 Flow-Chart of Sampled Training in Neural Networks
Page 242
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com Volume 2, Issue 4, April 2013 ISSN 2319 - 4847
With re-sampling taking place, the training set is reduced and so is the training time. As the training time is reduced, the overall time of classification is also reduced [5].
4. ADVANTAGES OF SAMPLED TRAINING

Lesser Weight Updates As discussed earlier that gradient calculation requires maximum time in the training process. So to reduce the training time, the number of gradient calculations to be performed should be decreased. In sampled training, as the number of training samples used for training is less, the gradient calculations are reduced and so is the number of weight updates resulting in speed improvement in training a neural network. Reduction in Training Time As the numbers of training samples that are used for training are lesser than whole of the training set, the amount of time required to train a neural network with sampled training is less. More Accurate Classification Model As the accuracy of the trained model is compared every time after the test data is classified the accuracy of the classification model so generated is always above than the user-specified accuracy level.
5. CONCLUSION
In Feed-Forward neural networks used with Gradient Descent Technique, training time is the most important parameter to be considered. As the gradient calculations require or consume more time than actual training. So the technique of using the training set should be checked to reduce the training time. In batch mode training, the numbers of weight updates are more with increase in the training data which in turn results in the increase of gradient calculations. Due to more gradient calculations, the training time also increases. In sampled training, as the original training data is sampled into fixed size samples and then given as input to the network for training, the number of weight updates is less as compared to number of weight updates in batch mode training. And so, as a result gradient calculation also reduces which results in lesser training times.
6. ACKNOWLEDGEMENTS
Advait Bhatt wishes to thank Prof. H.B.Jethva, for his guidance and help for doing this work. He also acknowledges Prof. D. A. Parikh, Head of computer department, and to all staff of computer department for full support for completion of this work. Prof. Harikrishna Jethva wishes to acknowledge his family and staff of computer department at L.D.College of engineering.
References
[1] Jiaweu Han and Micheline Kamber, Data Mining Concepts and Techniques , second edition, Elsevier press [2] Nortje William Daniel, Comparison of Bayesian Learning and Conjugate Gradient Descent Training Of Neural Networks, University of Pretoria, October 2001. [3] Er. Shegal Parveen, Dr. Gupta Sangeeta and Prof. Kumar Dharminder, Minimization of Error in Training A Neural Network Using Gradient Descent Technique, International Journal of Technical Research (IJTR), Vol-1, Issue-1, Mar-Apr 2012. [4] Gong Liang, Liu Chengliang and Yuan Fuqing, Training Feed-forward Neural Network Using the Gradient Descent Method With The Optimal Step Size, Journal of Computational Information System 8:4 (2012). [5] Nandy Sudarshan, Sarkar Prartha Prathim and Das Achintya, An Improved Gauss-Newtons Method Based BackPropagation Algorithm For Fast Convergence, International Journal for Computer Applications, Vol 39-No. 8, February 2012. AUTHOR Advait S. Bhatt, pursuing his Master Degree in Computer Science & Technology from Gujarat Technological University (L D College of Engg., Ahmedabad), received his Bachelor Degree in Information Technology from Dharmsinh Desai University, Nadiad in 2010. His area of interest include Database Systems, Data Mining, C/C++/Java/Advanced java Programming. Prof. Harikrishna B. Jethva, received his post graduate degree in Computer engineering from Dharmsinh Desai University in 2009 and Bachelor Degree in Computer Engineering from Saurashtra University, Rajkot in 2001. He has worked as a Assistant Professor for 10 Years and presently working as a Associate Professor in L.D.College of Engineering. His area of interest is in Neural Network, Theory of Computation, Computer Network, Compiler Design, Soft Computing and Distributed Computing.
Page 243

Improvement in The Speed of Training A Neural Network Using Sampled Training

Uploaded by

Copyright:

Available Formats

Improvement in The Speed of Training A Neural Network Using Sampled Training

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Improvement in The Speed of Training A Neural Network Using Sampled Training

Uploaded by

Copyright:

Available Formats

Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.

com Volume 2, Issue 4, April 2013 ISSN 2319 - 4847

Improvement in the Speed of Training a Neural Network Using Sampled Training

2. TRAINING A NEURAL NETWORK

Volume 2, Issue 4, April 2013

3. SAMPLED TRAINING (PROPOSED SCHEME)

Figure 3 Flow-Chart of Sampled Training in Neural Networks

Volume 2, Issue 4, April 2013

4. ADVANTAGES OF SAMPLED TRAINING

Volume 2, Issue 4, April 2013

You might also like