Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

A Discriminant Model for the Pattern Recognition of Linearly Independent Samples

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Fisher’s linear discriminant analysis (FLDA) has been used in many pattern recognition applications. However, this method cannot be applied for solving the pattern recognition problems when the within-class scatter matrix is singular, i.e., when the so-called small sample size problem occurs. Many FLDA variants that have been proposed in the past to circumvent this problem either suffer from excessive computational load when patterns have a large dimension or lose some useful discriminant information. In this paper, a new systematic framework for the pattern recognition of datasets with linearly independent samples is developed. Within this framework, a discriminant model, in which the samples of the individual classes of a dataset lie on parallel hyperplanes and project to single distinct points of a discriminant subspace of the underlying input space, is shown to exist. Based on this model, three algorithms that do not encounter the adverse effects of the small sample size (SSS) problem are developed to obtain such a discriminant subspace for a given dataset with linearly independent samples. A kernelized algorithm is also developed for the discriminant analysis of datasets for which the samples are not linearly independent. Simulation results are provided to examine the validity of the proposed discriminant model and to demonstrate the effectiveness, both in terms of complexity and classification accuracy, of the linear and nonlinear algorithms designed based on the proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Y. Bing, J. Lianfu, C. Ping, A new LDA-based method for face recognition, in Proc. 16th Int. Conf. Pattern Recognition, vol. 1 (2002), pp. 168–171

    Google Scholar 

  2. C. Burges, A Tutorial on support vector machine for pattern recognition. Data Min. Knowl. Discov. 2, 121–167 (1998)

    Article  Google Scholar 

  3. H. Cevikalp, M. Neamtu, M. Wilkes, Discriminative common vector method with kernels. IEEE Trans. Neural Netw. 17(6), 1550–156 (2006)

    Article  Google Scholar 

  4. D.-G. Chen, Q. He, X.Z. Wang, On linear separability of data sets in feature space. Neurocomputing 70(13–15), 2441–8 (2007)

    Article  Google Scholar 

  5. D.-Q. Dai, P.C. Yuen, Regularized discriminant analysis and its application to face recognition. Pattern Recognit. 36, 845–847 (2003)

    Article  MATH  Google Scholar 

  6. K.-L. Du, M.N.S. Swamy, Neural Networks in a Softcomputing Framework (Springer, London, 2006)

    MATH  Google Scholar 

  7. K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd edn. (Academic Press, San Diego, 1990)

    Google Scholar 

  8. H. Gao, W.D. James, Why direct LDA is not equivalent to LDA. Pattern Recognit. 39, 1002–1996 (2006)

    Article  MATH  Google Scholar 

  9. P. Howland, M. Jeon, H. Park, Structure preserving dimension reduction for clustered text data based on the generalized singular value decomposition. SIAM J. Matrix Anal. Appl. 25(1), 165–179 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  10. R. Huang, Q. Liu, H. Lu, S. Ma, Solving the small size problem of LDA, in Proc. 16th Int. Conf. Pattern Recognition, vol. 3 (2002), pp. 29–32

    Google Scholar 

  11. C. Lee, D. Landgrebe, Analyzing high-dimensional multispectral data. IEEE Trans. Geosci. Remote Sens. 31(7), 792–800 (1993)

    Article  Google Scholar 

  12. J. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, J. Wang, An efficient kernel discriminant analysis method. Pattern Recognit. 38(10), 1788–1790 (2005)

    Article  MATH  Google Scholar 

  13. C.H. Park, H. Park, A comparison of generalized linear discriminant analysis algorithms. Pattern Recognit. 41, 1083–1097 (2008)

    Article  MATH  Google Scholar 

  14. B. Schölkopf, A.J. Smola, Learning with Kernels—Support Vector Machines, Regularization, Optimization, and Beyond (MIT, Cambridge, 2002)

    Google Scholar 

  15. G. Strang, Linear Algebra and Its Application, 3rd edn. (MIT, Cambridge, 1998)

    Google Scholar 

  16. D.L. Swets, J. Weng, Using discriminant eigenfeatures for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 18(8), 831–836 (1996)

    Article  Google Scholar 

  17. Wei Wu, M.O. Ahmad, S. Samadi, Discriminant analysis based on modified generalized singular value decomposition and its numerical error analysis. IET Comput. Vis. 3(3), 159–173 (2009)

    Article  MathSciNet  Google Scholar 

  18. J. Yang, A.F. Frangi, J. Yang, D. Zhang, Z. Jin, KPCA plus LDA: a complete kernel fisher discriminant framework for feature extraction and recognition. IEEE Trans. Pattern Anal. Mach. Intell. 27(2), 230–244 (2005)

    Article  Google Scholar 

  19. J. Yang, D. Zhang, J.-Y. Yang, A generalised K-L expansion method which can deal with small sample size and high-dimensional problems. Pattern Anal. Appl. 6, 47–54 (2003)

    Article  MATH  Google Scholar 

  20. J. Ye, R. Janardan, C.H. Park, H. Park, An optimization criterion for generalized discriminant analysis on under sampled problems. IEEE Trans. Pattern Anal. Mach. Intell. 26(8), 982–994 (2004)

    Article  Google Scholar 

  21. J. Ye, T. Xiong, Q. Li, R. Janardan, J. Bi, V. Cherkassky, C. Kambhamettu, Efficient model selection for regularized linear discriminant analysis, in Proc. International Conference on Information and Knowledge Management (2006), pp. 5–11

    Google Scholar 

  22. H. Yu, J. Yang, A direct LDA algorithm for high-dimensional data with application to face recognition. Pattern Recognit. 34, 2067–2070 (2001)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Research Institute of Tsinghua University, Shen Zhen, Guang Dong, 518000, China

    Wei Wu

  2. Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, H3G 1M8, Canada

    M. Omair Ahmad

Authors
  1. Wei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Omair Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Omair Ahmad.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wu, W., Ahmad, M.O. A Discriminant Model for the Pattern Recognition of Linearly Independent Samples. Circuits Syst Signal Process 31, 669–687 (2012). https://doi.org/10.1007/s00034-011-9337-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-011-9337-2

Keywords

Search

Navigation