Abstract
Learning an informative dictionary is a critical challenge in sparse representation and low-rank modeling. The quality of dictionary usually affects the performance of learning models significantly. In this chapter, we propose a novel low-rank dictionary learning method, which learns a discriminative dictionary with low-rank constraints. We learn a sub-dictionary for each class separately, and the overall representation ability of the dictionary is also considered. In particular, the Fisher criterion is incorporated in our model to improve the discriminability of dictionary, which maximizes the ratio of the between-class scatter to within-class scatter. In practice, training samples may contain noisy information, which would undermine the quality of the dictionary. Inspired by the recent advances in low-rank matrix recovery, we enforce a low-rank constraint on the sub-dictionary for each class to tackle this problem. Our model is formulated as an \(l_1\) regularized rank-minimization problem, which can be solved by the iterative projection method (IPM) and inexact augmented Lagrange multiplier (ALM) algorithms. The proposed discriminative dictionary learning with low-rank regularization (\(D^2L^2R^2\)) method is evaluated on four public face and digit image datasets, in comparison with existing representative dictionary learning and image classification methods. The experimental results demonstrate that our method outperforms related methods in various settings.
This chapter is reprinted with permission from Elsevier. “Learning Low-Rank and Discriminative Dictionary for Image Classification”, Image and Vision Computing, 2014. © [2014] Elsevier.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The machine used installs 24 GB RAM and Intel Xeon W3350 CPU.
References
M. Aharon, M. Elad, A. Bruckstein, K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)
T. Bai, Y. Li, Robust visual tracking using flexible structured sparse representation. IEEE Trans. Ind. Inf. 10(1), 538–547 (2014)
S.R. Becker, E.J. Candès, M.C. Grant, Templates for convex cone problems with applications to sparse signal recovery. Math. Program. Comput. 3(3), 165–218 (2011)
P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces versus fisherfaces: recognition using class specific linear projection. IEEE TPAMI 19(7), 711–720 (1997)
Y. Bengio, A.C. Courville, P. Vincent, Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)
D.P. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods (Academic Press, New York, 1982)
E. Candès, B. Recht, Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)
E.J. Candès, X.D. Li, Y. Ma, J. Wright, Robust principal component analysis? J. ACM 58(3), 11 (2011)
E.J. Candès, B. Recht, Exact matrix completion via convex optimization. Found. Comput. Math. 9(6), 717–772 (2009)
C.F. Chen, C.P. Wei, Y.F. Wang, Low-rank matrix recovery with structural incoherence for robust face recognition, in CVPR 2012, pp. 2618–2625 (2012)
S. Chen, S.A. Billings, W. Luo, Orthogonal least squares methods and their application to non-linear system identification. Int. J. Control 50(5), 1873–1896 (1989)
G. Davis, S. Mallat, M. Avellaneda, Adaptive greedy approximations. Constr. Approximation 13(1), 57–98 (1997)
D.L. Donoho, A. Maleki, A. Montanari, Message-passing algorithms for compressed sensing. Proc. Nat. Acad. Sci. 106(45), 18914–18919 (2009)
D.L. Donoho, Y. Tsaig, Fast solution ofwhen the solution may be sparse. IEEE Trans. Inf. Theory 54(11), 4789 (2008)
R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, 2nd edn. (Wiley, New York, 2001)
B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, Least angle regression. Ann. Stat. 32(2), 407–499 (2004)
M. Elad, B. Matalon, M. Zibulevsky, Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization. Appl. Comput. Harmonic Anal. 23(3), 346–367 (2007)
K. Frisch, The logarithmic potential method of convex programming. Memorandum May 13, 1955 (1955)
S. Gao, I.W.-H. Tsang, Y. Ma, Learning category-specific dictionary and shared dictionary for fine-grained image categorization. IEEE Trans. Image Process. 23(2), 623–634 (2014)
H. Guo, Z. Jiang, L. Davis, Discriminative dictionary learning with pairwise constraints, in ACCV 2012, vol. 7724 (Springer, Berlin, 2013), pp. 328–342
G. Irie, D. Liu, Z. Li, S.-F. Chang, A bayesian approach to multimodal visual dictionary learning, in CVPR, pp. 329–336 (2013)
I.H. Jhuo, D. Liu, D.T. Lee, S.F. Chang, Robust visual domain adaptation with low-rank reconstruction, in CVPR 2012 (2012)
Z. Jiang, Z. Lin, L. Davis, Learning a discriminative dictionary for sparse coding via label consistent k-svd, in CVPR 2011, pp. 1697–1704 (2011)
N. Karmarkar, A new polynomial-time algorithm for linear programming, in Proceedings of the 16th Annual ACM Symposium on Theory of Computing 1984, (ACM, 1984), pp. 302–311
R. Keshavan, A. Montanari, S. Oh, Matrix completion from noisy entries. J. Mach. Learn. Res. 11, 2057–2078 (2010)
K. Kreutz-Delgado, J.F. Murray, B.D. Rao, K. Engan, T.-W. Lee, T.J. Sejnowski, Dictionary learning algorithms for sparse representation. Neural Comput. 15(2), 349–396 (2003)
Y. LeCun, L. Bottou, Y. Bengio, P. Haaffner, Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
H. Lee, A. Battle, R. Raina, A.Y. Ng, Efficient sparse coding algorithms, in NIPS 2007, ed. by B. Schölkopf, J. Platt, T. Hoffman (MIT Press, Cambridge, 2007), pp. 801–808
K.-C. Lee, J. Ho, D. Kriegman, Acquiring linear subspaces for face recognition under variable lighting. IEEE TPAMI 27(5), 684–698 (2005)
M.S. Lewicki, B.A. Olshausen, Probabilistic framework for the adaptation and comparison of image codes. JOSA A 16(7), 1587–1601 (1999)
L. Li, S. Li, Y. Fu, Discriminative dictionary learning with low-rank regularization for face recognition, in FG 2013 (2013)
L. Li, S. Li, Y. Fu, Learning low-rank and discriminative dictionary for image classification. Image Vis. Comput. (2014)
Z. Lin, M. Chen, Y. Ma, The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv preprint arXiv:1009.5055 (2010)
G.C. Liu, Z.C. Lin, Y. Yu, Robust subspace segmentation by low-rank representation, in ICML 2010, pp. 663–670 (2010)
G.C. Liu, S.C. Yan, Latent low-rank representation for subspace segmentation and feature extraction, in ICCV 2011 (2011)
R.S. Liu, Z.C. Lin, F.D. Torre, Z.X. Su, Fixed-rank representation for unsupervised visual learning, in CVPR 2012 (2012)
L. Ma, C. Wang, B. Xiao, W. Zhou, Sparse representation for face recognition based on discriminative low-rank dictionary learning, in CVPR 2012, pp. 2586–2593 (2012)
D.M. Malioutov, M. Cetin, A.S. Willsky, Homotopy continuation for sparse signal representation, in ICASSP 2005, IEEE, vol. 5, pp. 728–733 (2005)
S.G. Mallat, Z. Zhang, Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41(12), 3397–3415 (1993)
X. Mo, V. Monga, R. Bala, Z. Fan, Adaptive sparse representations for video anomaly detection. IEEE Trans. Circuits Syst. Video Techn. 24(4), 631–645 (2014)
I. Naseem, R. Togneri, M. Bennamoun, Linear regression for face recognition. IEEE TPAMI 32(11), 2106–2112 (2010)
Y. Nesterov, A method of solving a convex programming problem with convergence rate o (1/k2). Sov. Math. Dokl. 27, 372–376 (1983)
M.R. Osborne, B. Presnell, B.A. Turlach, A new approach to variable selection in least squares problems. IMA J. Numer. Anal. 20(3), 389–403 (2000)
T. Poggio, T. Serre, et al., Learning a dictionary of shape-components in visual cortex: comparison with neurons, humans and machines. Ph.D. thesis, Massachusetts Institute of Technology (2006)
L. Rosasco, S. Mosci, S. Santoro, A. Verri, S. Villa, Iterative projection methods for structured sparsity regularization. Technical report, Technical Report MIT-CSAIL-TR-2009-050, MIT (2009)
F. Samaria, A. Harter, Parameterisation of a stochastic model for human face identification, in WACV 1994, pp. 138–142 (1994)
M. Shao, C. Castillo, Z. Gu, Y. Fu, Low-rank transfer subspace learning, in ICDM 2012 (2012)
T. Sim, S. Baker, M. Bsat, The cmu pose, illumination, and expression (pie) database, in FG 2002 (2002)
A. Wagner, J. Wright, A. Ganesh, Z. Zhou, H. Mobahi, Y. Ma, Toward a practical face recognition system: Robust alignment and illumination by sparse representation. IEEE TPAMI 34(2), 372–386 (2012)
J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, Y. Gong, Locality-constrained linear coding for image classification, in CVPR 2010 (2010)
N. Wang, J. Wang, D.-Y. Yeung, Online robust non-negative dictionary learning for visual tracking, in ICCV, pp. 657–664 (2013)
J. Wright, A. Yang, A. Ganesh, S. Sastry, Y. Ma, Robust face recognition via sparse representation. IEEE TPAMI 31(2), 210–227 (2009)
Y. Xie, C. Huang, T. Song, J. Ma, J. Jing, Object co-detection via low-rank and sparse representation dictionary learning, in VCIP, pp. 1–6 (2013)
A.Y. Yang, S. Iyengar, S. Sastry, R. Bajcsy, P. Kuryloski, R. Jafari, Distributed segmentation and classification of human actions using a wearable motion sensor network, in CVPRW 2008, pp. 1–8 (2008)
A.Y. Yang, Z. Zhou, A. Ganesh, S.S. Sastry, Y. Ma, Fast l1-minimization algorithms for robust face recognition. arXiv preprint arXiv:1007.3753 (2010)
J. Yang, J. Wright, T.S. Huang, Y. Ma, Image super-resolution as sparse representation of raw image patches, in CVPR (2008)
J. Yang, K. Yu, Y. Gong, T.S. Huang, Linear spatial pyramid matching using sparse coding for image classification, in CVPR, pp. 1794–1801 (2009)
M. Yang, L. Zhang, X.C. Feng, D. Zhang, Fisher discrimination dictionary learning for sparse representation, in ICCV 2011, pp. 543–550 (2011)
H. Zhang, J. Yang, Y. Zhang, N.M. Nasrabadi, T.S. Huang, Close the loop: joint blind image restoration and recognition with sparse representation prior, in ICCV 2011, pp. 770–777 (2011)
Q. Zhang, B. Li, Discriminative k-svd for dictionary learning in face recognition, in CVPR 2010, pp. 2691–2698 (2010)
Y. Zhang, Z. Jiang, L.S. Davis, Learning structured low-rank representations for image classification, in CVPR, pp. 676–683 (2013)
Z.D. Zhang, A. Ganesh, X. Liang, Y. Ma, Tilt: transform invariant low-rank textures. Int. J. Comput. Vis. 99(1), 1–24 (2012)
G.Y. Zhu, S.C. Yan, Y. Ma, Image tag refinement towards low-rank, content-tag prior and error sparsity, in ACM Multimedia 2010, pp. 461–470 (2010)
L.S. Zhuang, H.Y. Gao, Z.C. Lin, Y. Ma, X. Zhang, N.H. Yu, Non-negative low rank and sparse graph for semi-supervised learning, in CVPR 2012, pp. 2328–2335 (2012)
Acknowledgments
This research is supported in part by the NSF CNS award 1314484, Office of Naval Research award N00014-12-1-1028, Air Force Office of Scientific Research award FA9550-12-1-0201, and U.S. Army Research Office under grant number W911NF-13-1-0160.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Li, S., Li, L., Fu, Y. (2014). Low-Rank and Sparse Dictionary Learning. In: Fu, Y. (eds) Low-Rank and Sparse Modeling for Visual Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-12000-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-12000-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11999-1
Online ISBN: 978-3-319-12000-3
eBook Packages: Computer ScienceComputer Science (R0)