Abstract
In this article, we propose some methods for deriving symbolic interpretation of data in the form of rule based learning systems by using Support Vector Machines (SVM). First, Radial Basis Function Neural Networks (RBFNN) learning techniques are explored, as is usual in the literature, since the local nature of this paradigm makes it a suitable platform for performing rule extraction. By using support vectors from a learned SVM it is possible in our approach to use any standard Radial Basis Function (RBF) learning technique for the rule extraction, whilst avoiding the overlapping between classes problem. We will show that merging node centers and support vectors explanation rules can be obtained in the form of ellipsoids and hyper-rectangles. Next, in a dual form, following the framework developed for RBFNN, we construct an algorithm for SVM. Taking SVM as the main paradigm, geometry in the input space is defined from a combination of support vectors and prototype vectors obtained from any clustering algorithm. Finally, randomness associated with clustering algorithms or RBF learning is avoided by using only a learned SVM to define the geometry of the studied region. The results obtained from a certain number of experiments on benchmarks in different domains are also given, leading to a conclusion on the viability of our proposal.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Abbreviations
- RBF:
-
Radial Basis Function
- RBFNN:
-
Radial Basis Function Neural Network
- SVM:
-
Support Vector Machine
References
Andrews R., Diederich J. and Tickle A. (1995). A survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems 8(6):373–389
Blake, C. and Merz, C.: UCI repository of machine learning databases (1998).
Cherkassky V. and Mulier F. (1998). Learning from Data. Wiley, New York
Cortes C. and Vapnik V. (1995). ‘Support vector networks’. Machine Learning 20:273–297
Craven M. and Shavlik J. (1997). Using neural networks for data mining. Future Generation Computer Systems 13:211–229
Cristianini N. and Shawe-Taylor J. (2000). An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press, Cambridge
Duda R., Hart P. and Stork D. (2001). Pattern Recognition, 2nd edn. Wiley, New York Inc,
Fu, X. and Wang, L.: Rule extraction by genetic algorithms based on a simplified RBF neural network. In: Proceedings of the Congress on Evolutionary Computation, vol. 2, pp. 753–758, 2001.
Guyon, I., Martíc, N. and Vapnik, V.: Advances in Knowledge Discovery and Data Mining, Chapt. Discovery Information Patterns and Data Cleaning. MIT Press, 1996.
Huber, K. and Berthold, M.: Building precise classifiers with automatic rule extraction. In: Proceedings of the IEEE International Conference on Neural Networks, vol. 3. pp. 1263–1268, 1995.
Kaufmand, L. and Rousseeuw, P.: Finding Groups in Data. An Introduction to Cluster Analysis. Wiley, New York Inc, 1990.
Kecman V. (2001). Learning and Soft Computing Support Vector Machines, Neural Networks and Fuzzy Logic Models. MIT Press, Cambridge
Ma, J. and Zhao, Y.: OSU support vector machines toolbox, version 3.0’. http:// www.csie.ntu.edu.tw/~cjlin/libsvm, 2004.
McGarry, K., Wermter, S. and MacIntyre, J.: Knowledge extraction from local networks. In: Proceedings of the International Joint Conference on Neural Networks. pp. 765–770, 2001.
Mitra S., Pal S. and Mitra P. (2002). Data mining in soft computing framework: a survey. IEEE Transactions on Neural Networks 13(1):3–14
Moody J. and Darken C. (1989). Fast learning in networks of locally-tuned processing units. Neural Computation 1(2): 281–294
Nabney, I. and Bishop, C.: Netlab neural networks software. http://www.ncrg. aston.ac.uk/netlab, 2002.
Núñez, H., Angulo, C. and Catal_A.: Rule extraction from support vector machines. In: Verleysen, M., (ed.), ESANN 2002, European Symposium on Artificial Neural Networks. Bruges, Belgium, pp. 107–112, d-facto publications, 2002.
Núñez, H., Angulo, C. and Catal_A.: Hybrid architecture based on support vector machines. In: Proceedings of the IWANN 2003, Vol. 2686 of Lecture Notes in Computer Science. Menorca, Spain, pp. 646–653, 2003.
Orr, M.: Radial basis function networks. http://www.anc.ed.ac.uk/~mjo/rbf.html, 1999.
Schölkopf, B.: The Kernel trick for distances. In: Neural Information Processing Systems, NIPS, vol. 13, 2000.
Seitono R., Leow W. and Zurada J. (2002). Extraction rules from artificial neural networks for nonlinear regression. IEEE Transactions on Neural Networks 13(3):564–577
Vapnik, V.: (1998) Statistical Learning Theory. New York, Wiley Inc,
Zhou Z.-H. (2004) Rule extraction: using neural networks or for neural networks?. Journal of Computer Science and Technology 19(2):249–253
Zhou Z.-H., Jiang Y. and Chen S.-F. (2003). Extracting symbolic rules from trained neural network ensembles. AI Communications 16(1):3–15
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License ( https://creativecommons.org/licenses/by-nc/2.0 ), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Núñez, H., Angulo, C. & Català, A. Rule-Based Learning Systems for Support Vector Machines. Neural Process Lett 24, 1–18 (2006). https://doi.org/10.1007/s11063-006-9007-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-006-9007-8