Keywords
Multilayer Feedforward Neural Network
Recurrent Neural Network
Radial Basis Function
Training
Advantages and Disadvantages.
How to Cite
Abstract
In this paper, an overview of the artificial neural networks is presented. Their main and popular types such as the multilayer feedforward neural network (MLFFNN), the recurrent neural network (RNN), and the radial basis function (RBF) are investigated. Furthermore, the main advantages and disadvantages of each type are included as well as the training process.References
Haykin S. Neural Networks and Learning Machines, Third Edit. Pearson, 2009.
Sharkawy A-N, Aspragathos N. Human-Robot Collision Detection Based on Neural Networks. Int J Mech Eng Robot Res, 2018;. 7(2): 150-157.
Sharkawy A-N, Koustoumpardis PN, Aspragathos N. Manipulator Collision Detection and Collided Link Identification based on Neural Networks. in Advances in Service and Industrial Robotics RAAD 2018 Mechanisms and Machine Science, A. Nikos, K. Panagiotis, and M. Vassilis, Eds. Springer, Cham, 2018; pp. 3-12.
Lu S, Chung JH, Velinsky SA. Human-Robot Collision Detection and Identification Based on Wrist and Base Force / Torque Sensors. in Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005; no. April, 3796-3801.
Ito M, Noda K, Hoshino Y, Tani J. Dynamic and interactive generation of object handling behaviors by a small humanoid robot using a dynamic neural network model. Neural Networks 2006; 19(3): 323-337,
Sharkawy A-N, Koustoumpardis PN, Aspragathos N. Variable Admittance Control for Human - Robot Collaboration based on Online Neural Network Training. in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2018): 2018.
Sharkawy A-N, Koustoumpardis PN, Aspragathos N. A Neural Network based Approach for Variable Admittance Control in Human- Robot Cooperation : Online Adjustment of the Virtual Inertia. Intell Serv Robot 1-37: 2020.
Passricha V, Aggarwal RK. Convolutional Neural Networks for Raw Speech Recognition. in From Natural to Artificial Intelligence - Algorithms and Applications, 2018; 21-40.
Palaz D, Magimai-Doss M, Collobert R. Convolutional Neural Networks-Based Continuous Speech Recognition Using Raw Speech Signal. in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015; 4295-4299.
Fadzil MHA, Bakar HA. Human face recognition using neural networks. in Proceedings of 1st International Conference on Image Processing, 1994.
Zhao ZQ, Huang DS, Sun BY. Human face recognition based on multi-features using neural networks committee. Pattern Recognit Lett. 2004; 25(12): 1351-1358,
Fukuoka Y. Artificial Neural Networks in Medical Diagnosis. in Computational Intelligence Processing in Medical Diagnosis Studies in Fuzziness and Soft Computing, S. M., T. HN., J. A., J. A., J. S., and J. L.C., Eds. 2002; 197-228.
Moreno-Escobar JA, Gallegos-Funes FJ, Ponomaryov VI. Rank M-type radial basis functions network for medical image processing applications. in Image Processing: Algorithms and Systems V, 2007; 6497: 1-12.
Er O, Yumusak N, Temurtas F. Chest diseases diagnosis using artificial neural networks. Expert Syst Appl. 2010; 37(12): 7648-7655.
Sukthomya W, Tannock J. The training of neural networks to model manufacturing processes. J Intell Manuf. 2005; 16(1): 39-51.
Abdelhameed MM, Tolbah FA. Design and implementation of a flexible manufacturing control system using neural network. Int J Flex Manuf Syst. 2002; 14(3): 263-279.
Falat L, Pancikova L. Quantitative Modelling in Economics with Advanced Artificial Neural Networks. Procedia Econ Financ. 2015; 34: 194-201.
Badea LM. (Stroie), Predicting Consumer Behavior with Artificial Neural Networks. Procedia Econ Financ. 2014; 15(14): 238-246.
Du K-L, Swamy MNS. Neural networks in a softcomputing framework. London: Springer, 2006.
Most T. Approximation of complex nonlinear functions by means of neural networks. in 2nd Weimar Optimization and Stochastic Days 2005; 2005.
Nielsen MA. Neural Networks and Deep Learning. Determination Press, 2015.
Ferrari S, Stengel RF. Smooth Function Approximation Using Neural Networks. IEEE Trans Neural Networks 2005; 16(1): 24-38.
Hornik K, Stinchcombe M, White H. Universal Approximation of an Unknown Mapping and Its Derivatives Using Multilayer Feedforward Networks. Neurul Networks 1990; 3: 551-560.
Vemuri AT, Polycarpou MM. Neural-Network-Based Robust Fault Diagnosis in Robotic Systems. IEEE Trans Neural Networks 1997; 8(6): 1410-1420.
Sharkawy AN, Koustoumpardis PN, Aspragathos N. Neural Network Design for Manipulator Collision Detection Based only on the Joint Position Sensors. Robotica 2019; 1-19.
Chester DL. Why Two Hidden Layers are Better than One. in International Joint Conference on Neural Networks, 1990; 265-268.
Thomas AJ, Walters SD, Petridis M, Gheytassi SM, Morgan RE. Accelerated optimal topology search for two-hidden-layer feedforward neural networks. in Engineering Applications of Neural Networks EANN 2016 Communications in Computer and Information Science, 2016; 629: J. C. and I. L., Eds. Springer, Cham. 253-266.
Feedforward Neural Networks and Multilayer Perceptrons.”
Fung CC, Iyer V, Brown W, Wong KW. Comparing the Performance of Different Neural Networks Architectures for the Prediction of Mineral Prospectivity. in 2005 International Conference on Machine Learning and Cybernetics, 2005; 394-398.
Chiang YM, Chang LC, Chang FJ. Comparison of staticfeedforward and dynamic-feedback neural networks for rainfall-runoff modeling. J Hydrol. 2004; 290(3-4): 297-311.
Schmidhuber J. Deep learning in neural networks : An overview. Neural Networks 2015; 61: 85-117.
Jeatrakul P, Wong KW. Comparing the performance of different neural networks for binary classification problems. in 2009 8th International Symposium on Natural Language Processing, SNLP ’09, 2009; 111-115.
Chen SC, Lin SW, Tseng TY, Lin HC. Optimization of backpropagation network using simulated annealing approach. in 2006 IEEE International Conference on Systems, Man and Cybernetics, 2006; 2819-2824.
Sassi MA, Otis MJD, Campeau-Lecours A. Active stability observer using artificial neural network for intuitive physical human-robot interaction. Int J Adv Robot Syst. 2017; 14(4): 1- 16.
De Momi E, Kranendonk L, Valenti M, Enayati N, Ferrigno G. A Neural Network-Based Approach for Trajectory Planning in Robot-Human Handover Tasks. Front Robot AI. 2016; 3(June, 1-10.
Rad AB, Bui TW, Li V, Wong YK. A New On-Line PID Tuning Method Using Neural Networks. IFAC Proc Vol IFAC Work Digit Control Past, Present Futur PID Control, 2000; 33(4): 443-448.
Elbelady SA, Fawaz HE, Aziz A. MA. Online Self Tuning PID Control Using Neural Network for Tracking Control of a Pneumatic Cylinder Using Pulse Width Modulation Piloted Digital Valves. Int J Mech Mechatronics Eng IJMME-IJENS 2016; 16(3) 123-136.
Hernández-Alvarado R, García-Valdovinos LG, SalgadoJiménez T, Gómez-Espinosa A, Fonseca-Navarro F. Neural Network-Based Self-Tuning PID Control for Underwater Vehicles. Sensors, 2016; 16(9): 1429, 1-18.
Xie T, Yu H, Wilamowski B. Comparison between traditional neural networks and radial basis function networks. in Proceedings - ISIE 2011: 2011 IEEE International Symposium on Industrial Electronics, 2011; 1194-1199.
Anderson D, McNeill G. Artificial neural neworks technology: A DACS state-of-the-art report. Utica, New York, 1992.
Du K, Swamy MNS. Neural Networks and Statistical Learning. Springer, 2014.
Marquardt DW. An Algorithm for Least-Squares Estimation of Nonlinear Parameters. J Soc Ind Appl Math. 1963; 11(2): 431- 441.
Hagan MT, Menhaj MB. Training Feedforward Networks with the Marquardt Algorithm. IEEE Trans Neural Networks 1994; 5(6): 2-6.
Rumelhart DE, Hinton GE, Williams RJ. Learning internal representations by error propagation. in Parallel Distributed Processing: Exploration of the Microstructure of Cognition, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, 1986; 318-362.
Rojas R. Neural Networks - A Systematic Introduction. Berlin: Springer-Verlag, 1996.
Singh A, Yang L, Levine S. GPLAC: Generalizing VisionBased Robotic Skills Using Weakly Labeled Images. in Proceedings of the IEEE International Conference on Computer Vision, 2017; 5852-5861.
Sharkawy AN, Koustoumpardis PN, Aspragathos N. Humanrobot collisions detection for safe human-robot interaction using one multi-input-output neural network. Soft Comput, 2020; 24(9): 6687-6719.
Jordan MI. Serial Order: A Parallel Distributed Processing Approach. San Diego; La Jolla, CA 92093, 1986.
Elman JL. Finding structure in time. Cogn Sci. 1990; 14(2): 179-211.
Wu W, An SY, Guan P, Huang DS, Sen Zhou B. Time series analysis of human brucellosis in mainland China by using Elman and Jordan recurrent neural networks. BMC Infect Dis. 2019; 19(1) 1-11.
Du K-L, Swamy MNS. Neural Networks and Statistical Learning. Springer, 2013.
Maass W, Joshi P, Sontag ED. Computational aspects of feedback in neural circuits. PLoS Comput Biol. 2007; 3(1): 0015-0034.
Siegelmann HT, Sontag ED. Turing computability with neural nets. Appl Math Lett. 1991; 4(6): 77-80.
Zhao X, Chumkamon S, Duan S, Rojas J, Pan J. Collaborative Human-Robot Motion Generation using LSTMRNN Collaborative Human-Robot Motion Generation using LSTM-RNN. in 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), 2018.
Torkar C, Yahyanejad S, Pichler H, Hofbaur M, Rinner B. RNN-based Human Pose Prediction for Human-Robot Interaction. in Proceedings of the ARW & OAGM Workshop 2019, 2019; 76-80.
Yamada T, Murata S, Arie H, Ogata T. Dynamical Integration of Language and Behavior in a Recurrent Neural Network for Human - Robot. Front Neurorobot. 2016; 10: 1-17.
Schydlo P, Rakovic M, Jamone L, Santos-Victor J. Anticipation in Human-Robot Cooperation : A Recurrent Neural Network Approach for Multiple Action Sequences Prediction. in IEEE International Conference on Robotics and Automation (ICRA), 2018; 5909-5914.
Pascanu R, Mikolov T, Bengio Y. On the difficulty of training recurrent neural networks Razvan. in Proceedings of the 30th International Conference on Machine Learning, 2013.
Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Networks 1994; 5(2): 157-166.
Venkatesan P, Anitha S. Application of a radial basis function neural network for diagnosis of diabetes mellitus. Curr Sci. 2006; 91(9): 1195-1199.
Baxt WG. Use of an Artificial Neural Network for the Diagnosis of Myocardial Infarction. Ann Intern Med. 1991; 843-848.
Buchman TG, Kubos KL, Seidler AJ, Siegforth MJ. A comparison of statistical and connectionist models for the prediction of chronicity in a surgical intensive care unit. Crit Care Med. 1994; 22(5): 750-762.
Selker HP, Griffith JL, Patil S, Long WJ, D’Agostino RB. A comparison of performance of mathematical predictive methods for medical diagnosis: identifying acute cardiac ischemia among emergency department patients. J Investig Med. 1995; 43(5):468-476.
Lang E, Pitts L, Damron S, Rutledge R. Outcome after severe head injury: An analysis of prediction based upon comparison of neural network versus logistic regression analysis. Neurol Res. 1997; 19(3): 274-280.
Lapuerta P, Rajan S, Bonacini M. Neural Networks As Predictors of Outcomes in Alcoholic Patients With Severe Liver Disease. Hepatology 1997; 25(2): 302-306.
Lippmann RP, Shahian DM. Coronary artery bypass risk prediction using neural networks. Ann Thorac Surg. 1997; 63(6): 1635-1643.
Kurban T, Beşdok E. A comparison of RBF neural network training algorithms for inertial sensor based terrain classification. Sensors, 2009; 9(8): 6312-6329.
Wang X, Ding Y, Shao H. The improved radial basis function neural network and its application. Artif Life Robot. 1998; 2(1): 8-11.
Song Y, Ren Y. A Predictive Model of Nonlinear System Based on Generalized Regression Neural Network. in 2005 International Conference on Neural Networks and Brain, 2005; 2009-2012.
Specht DF. A General Regression Neural Network. IEEE Trans Neural Networks, 1991; 2(6): 568-576.
Frost F, Karri V. Performance comparison of BP and GRNN models of the neural network paradigm using a practical industrial application. in ICONIP’99 ANZIIS’99 & ANNES’99 & ACNN’99 6th International Conference on Neural Information Processing Proceedings (Cat No99EX378), 1999; 1069- 1074.
Ooi SY, Teoh ABJ, Ong TS. Compatibility of biometric Strengthening with probabilistic neural network. in IEEEInternational Symposium on Biometrics and Security Technologies, ISBAST’08, 2008; 1-6.
Gorunescu F, Gorunescu M, El-darzi E. An Evolutionary Computational Approach to Probabilistic Neural Network with Application to Hepatic Cancer Diagnosis. in 18th IEEE Symposium on Computer-Based Medical Systems, 2005; 461-466.
Wu SG, Bao FS, Xu EY, Wang Y-X, Chang Y-F, Xiang Q-L. A Leaf Recognition Algorithm for Plant Classification Using Probabilistic Neural Network. in 2007 IEEE International Symposium on Signal Processing and Information Technolog, 2007; 11-16.
Kraipeerapun P. Neural network classification based on quantification of uncertainty. Murdoch University, 2009.
All the published articles are licensed under the terms of the Creative Commons Attribution Non-Commercial License (CC BY-NC 4.0) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.