Abstract
A series of recent results shows that if a signal admits a sufficiently sparse representation (in terms of the number of nonzero coefficients) in an “incoherent” dictionary, this solution is unique and can be recovered as the unique solution of a linear programming problem. We generalize these results to a large class of sparsity measures which includes the ℓp-sparsity measures for 0 ≤ p ≤ 1. We give sufficient conditions on a signal such that the simple solution of a linear programming problem simultaneously solves all the non-convex (and generally hard combinatorial) problems of sparsest representation w.r.t. arbitrary admissible sparsity measures. Our results should have a practical impact on source separation methods based on sparse decompositions, since they indicate that a large class of sparse priors can be efficiently replaced with a Laplacian prior without changing the resulting solution.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Benaroya, L., Gribonval, R., Bimbot, F.: Représentations parcimonieuses pour la séparation de sources avec un seul capteur. In: GRETSI 2001, Toulouse, France, Article _ 434 (2001)
Bertsekas, D.: Non-Linear Programming, 2nd edn. Athena Scientific, Belmont (1995)
Donoho, D., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via _1 minimization. Proc. Nat. Aca. Sci. 100(5), 2197–2202 (2003)
Donoho, D., Elad, M., Temlyakov, V.: Stable recovery of sparse overcomplete representations in the presence of noise. Working draft (February 2004)
Donoho, D., Huo, X.: Uncertainty principles and ideal atomic decompositions. IEEE Trans. Inform. Theory 47(7), 2845–2862 (2001)
Elad, M., Bruckstein, A.: A generalized uncertainty principle and sparse representations in pairs of bases. IEEE Trans. Inform. Theory 48(9), 2558–2567 (2002)
Feuer, A., Nemirovsky, A.: On sparse representations in pairs of bases. IEEE Trans. Inform. Theory 49(6), 1579–1581 (2003)
Fuchs, J.-J.: On sparse representations in arbitrary redundant bases. Technical report, IRISA, to appear in IEEE Trans. Inform. Theory (December 2003)
Gilbert, A., Muthukrishnan, S., Strauss, M.: Approximation of functions over redundant dictionaries using coherence. In: The 14th ACM-SIAM Symposium on Discrete Algorithms, SODA 2003 (January 2003)
Gilbert, A., Muthukrishnan, S., Strauss, M., Tropp, J.: Improved sparse approximation over quasi-incoherent dictionaries. In: Int. Conf. on Image Proc. (ICIP 2003), Barcelona, Spain (September 2003)
Gribonval, R., Nielsen, M.: Approximation with highly redundant dictionaries. In: Unser, M., Aldroubi, A., Laine, A.F. (eds.) Proc. SPIE 2003, San Diego, CA, August 2003. Wavelets: Applications in Signal and Image Processing X, vol. 5207, pp. 216–227 (2003)
Gribonval, R., Nielsen, M.: Highly sparse representations from dictionaries are unique and independent of the sparseness measure. Technical Report R-2003-16, Dept. of Math. Sciences, Aalborg University (October 2003)
Gribonval, R., Nielsen, M.: Sparse decompositions in unions of bases. IEEE Trans. Inform. Theory 49(12), 3320–3325 (2003)
Gribonval, R., Vandergheynst, P.: Exponential convergence of Matching Pursuit in quasi-incoherent dictionaries. Technical report 1619, IRISA (2004)
Shrijver, A.: Theory of Linear and Integer Programming. John Wiley, Chichester (1998)
Tropp, J.: Greed is good: Algorithmic results for sparse approximation. Technical report, Texas Institute for Computational Engineering and Sciences (2003)
Tropp, J.: Just relax: Convex programming methods for subset selection and sparse approximation. Technical Report ICES Report 04-04, UT-Austin (February 2004)
Zibulevsky, M., Pearlmutter, B.: Blind source separation by sparse decomposition in a signal dictionary. Neural Computations 13(4), 863–882 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gribonval, R., Nielsen, M. (2004). On the Strong Uniqueness of Highly Sparse Representations from Redundant Dictionaries. In: Puntonet, C.G., Prieto, A. (eds) Independent Component Analysis and Blind Signal Separation. ICA 2004. Lecture Notes in Computer Science, vol 3195. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30110-3_26
Download citation
DOI: https://doi.org/10.1007/978-3-540-30110-3_26
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23056-4
Online ISBN: 978-3-540-30110-3
eBook Packages: Springer Book Archive