Abstract
We consider the problem and issues of classifier fusion and discuss how they should be reflected in the fusion system architecture. We adopt the Bayesian viewpoint and show how this leads to classifier output moderation to compensate for sampling problems. We then discuss how the moderated outputs should be combined to reflect the prior distribution of models underlying the classifier designs.We then elaborate how the final stage of fusion should combine the complementary measurement information that might be available to different experts. This process is embodied in an overall architecture which shows why the fusion of raw expert outputs is a nonlinear function of the expert outputs and how this function can be realised as a sequence of relatively simple processes.
Chapter PDF
Similar content being viewed by others
Keywords
- Support Vector Machine
- Feature Selection
- Machine Intelligence
- Output Moderation
- Discriminatory Information
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Devijver, P.A., Kittler, J.: Pattern Recognition:A Statistical Approach. Prentice-Hall, Englewood Cliffs, N.J. (1982).
Titterington, D., Smith, A., Makov, U.: Statistical Analysis of Finite Mixture Distributions. John Wiley and Sons, Chichester (1985).
Akaike, H.: A New Look at Statistical Model Identification. IEEE Trans. Automatic Control 19 (1994) 716–723.
Schwarz, G.: Estimating the Dimension of a Model. The Annals of Statistics 6 (1978) 461–464.
Sardo, L., Kittler, J.: Minimum Complexity Estimator for RBF Networks Architecture Selection. Proc. International Conference on Neural Networks,Washington (1996) 137–142.
Sardo, L., Kittler, J.: Model ComplexityValidation for PDF Estimation Using Gaussian Mixtures. Proc. 14th International Conference on Pattern Recognition, Brisbane (1998) 195–197.
Quinlen, R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, Calif. (1993).
Vapnik, V.N.: The Nature of Statistical Learning Theory. JohnWiley, NewYork (1998).
Pudil, P., Novovičová, J., Kittler, J.: Floating Search Methods in Feature Selection. Pattern Recognition Letters 15 (1994) 1119–1125.
Pudil, P., Novovičová, J., Choakjarerwanit, N., Kittler, J.: Feature selection based on the approximation of class densities by finite mixtures of special type. Pattern Recognition 28 (1995) 1389–1397.
Pudil, P., Novovičová, J.: Novel Methods for Subset Selection with Respect to Problem Knowledge. IEEE Transactions on Intelligent Systems — Special Issue on Feature Transformation and Subset Selection (1998) 66–74.
Jain, A.K., Zongker, D.: Feature Selection: Evaluation, Application and Small Sample Performance. IEEE Transactions on PAMI 19 (1997) 153–158.
Novovičová, J., Pudil, P., Kittler, J.: Divergence based feature selection for multimodal class densities. IEEE Transactions on PAMI, 18 (1996) 218–223.
Somol, P., Pudil, P., Novovičová J., Paclik, P.: Adaptive floating search methods in feature selection. Pattern Recognition Letters 20 (1999) 1157–1163.
Somol, P., Pudil, P.: Oscillating Search Algorithms For Feature Selection. Proc. 15th IAPR International Conference on Pattern Recognition, Barcelona (2000).
Somol, P., Pudil, P., Ferri, F.J., Kittler, J.: Fast Branch and Bound Algorithm For Feature Selection. Proc 4thWorld Multiconference on Systemics, Cybernetics and Informatics, Orlando, Florida (2000).
Mayer, H.A., Somol, P., Pudil, P., Grim, J., Huber R., Schwaiger, R.: A Comparison of Deterministic and Non-Deterministic Feature Selection Algorithms for k-NN, Gaussian, and Neural Classifiers. Proc. 4thWorld Multiconference on Systemics, Cybernetics and Informatics, Orlando, Florida (2000).
Ferri, F.J., Kadirkamanathan, V., Kittler, J.: Feature Subset Search Using Genetic Algorithms. Proc IEE Workshop on Natural Algorithms in Signal Processing (1993) 23-1–23-7.
Mayer, H.A., Somol, P., Huber, R., Pudil, P.: Improving Statistical Measures of Feature Subsets by Conventional and Evolutionary Approaches. Proc. 3rd IAPR International Workshop on Statistical Techniques in Pattern Recognition, Alicante, (2000).
Alkoot F.M., Kittler, J.: Multiple Expert System Design by Combined Feature Selection and Probability Level Fusion. Proc. Conf. Fusion 2000, Paris (2000).
Alkoot F.M., Kittler, J.: Feature Selection for an Ensemble of Classifiers. Proc. 4th World Multiconference on Systemics, Cybernetics and Informatics, Orlando, Florida (2000).
Kittler, J., Hancock, E.R.: Combining Evidence in Probabilistic Relaxation. International Journal of Pattern Recognition and Artificial Intelligence, 3 (1989) 29–51.
Christmas, W.J., Kittler, J., Petrou, M.: Structural Matching in ComputerVision Using Probabilistic Relaxation. IEEETransPattern Analysis andMachine Intelligence, 17 (1995) 749–764.
Kittler, J.: Probabilistic Relaxation and the Hough Transform. Pattern Recognition 33 (2000) 705–714.
Shanmugan, K.S., Breipohl, A.M.: Random Signals: Detection, Estimation and DataAnalysis. Wiley, NewYork (1988).
Bedworth, M.: High level data fusion. PhDThesis, Aston University, United Kingdom (1999).
Jonsson, K., Kittler, J., Li Y.P., Matas, J.: Support Vector Machine for Face Authentication. In Proceeding of BMVC’99 (1999) 543–553.
Jonsson, K., Kittler, J., Matas, J.: Learning Support Vectors for Face Authentication: Sensitivity to Mis-Registrations. Proceeding of ACCV’00, Taipei (2000) 806–811.
Murphy, P.: Repository of machine learning databases and domain theories. ftp://ftp.ics.uci.edu/pub/machine-learning-databases (1999).
Jain, A.K., Duin, R.P.W., Mao, J.: Statistical pattern recognition: A review. IEEE Trans. on Pattern Analysis and Machine Intelligence PAMI-22 (2000) 4–37.
Bishop, C.J., Neural networks for pattern recognition. Clarendon Press, Oxford (1995).
Breiman, L., Friedman, J.H., Olsen, R.A., Stone, C.J.: Classification and Regression Trees, Wadsworth, California (1984).
Christmas, W.J., Kittler, J., Petrou, M.: Analytical Approaches to Neural Network Design. in Multiple Paradigms, Comparative Studies and Hybrid Systems, eds E S Gelsema, and L N Kanal, North Holland (1994) 325–335.
Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On Combining Classifiers. IEEE Trans. Pattern Analysis and Machine Intelligence 20 (1998) 226–239.
Kittler, J.: Combining Classifiers:A Theoretical Framework. Pattern Analysis and Applications 1 (1998) 18–27.
Fairhurst, M.C., Abdel Wahab, H.M.S: An interactive two-level architecture for a memory network pattern classifier. Pattern Recognition Letters 11 (1990) 537–540.
Ho, T.K., Hull, J.J., Srihari, S.N.: Decision combination in multiple classifier systems. IEEE Trans. on Pattern Analysis and Machine Intelligence 16 (1994) 66–75.
Wolpert, D.H.: Stacked generalization. Neural Networks 5 (1992) 241–260.
Xu, L., Krzyzak, A., Suen, C.Y.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Trans. SMC 22 (1992) 418–435.
Kittler, J., Matas, J., Jonsson, K., Ramos Sánchez, M.U.: Combining evidence in personal identity verification systems. Pattern Recognition Letters 18 (1997) 845–852.
Tumer, K., Ghosh, J.: Analysis of Decision Boundaries in Linearly Combined Neural Classifiers. Pattern Recognition, 29 (1996) 341–348.
Woods, K.S., Bowyer, K., Kergelmeyer, W.P.: Combination of multiple classifiers using local accuracy estimates. Proc. of CVPR96 (1996), 391–396.
Kittler, J., Hojjatoleslami, A., Windeatt, T.: Strategies for combining classifiers employing shared and distinct pattern representations. Pattern Recognition Letters 18 (1997) 1373–1377.
Huang, T.S., Suen, C.Y.: Combination of multiple experts for the recognition of unconstrained handwritten numerals. IEEE Trans Pattern Analysis and Machine Intelligence 17 (1995) 90–94.
Alkoot, F.M., Kittler, J.: Improving the performance of the product fusion strategy. Proc. 15th IAPR International Conference on Pattern Recognition, Barcelona (2000).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kittler, J. (2000). A Framework for Classifier Fusion: Is It Still Needed?. In: Ferri, F.J., Iñesta, J.M., Amin, A., Pudil, P. (eds) Advances in Pattern Recognition. SSPR /SPR 2000. Lecture Notes in Computer Science, vol 1876. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44522-6_5
Download citation
DOI: https://doi.org/10.1007/3-540-44522-6_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67946-2
Online ISBN: 978-3-540-44522-7
eBook Packages: Springer Book Archive
