Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Testing that a Local Optimum of the Likelihood is Globally Optimum Using Reparameterized Embeddings

Applications to Wavefront Sensing

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Many mathematical imaging problems are posed as non-convex optimization problems. When numerically tractable global optimization procedures are not available, one is often interested in testing ex post facto whether or not a locally convergent algorithm has found the globally optimal solution. When the problem is formulated in terms of maximizing the likelihood function under a statistical model for the measurements, one can construct a statistical test that a local maximum is in fact the global maximum. A one-sided test is proposed for the case that the statistical model is a member of the generalized location family of probability distributions, a condition often satisfied in imaging and other inverse problems. We propose a general method for improving the accuracy of the test by reparameterizing the likelihood function to embed its domain into a higher-dimensional parameter space. We show that the proposed global maximum testing method results in improved accuracy and reduced computation for a physically motivated joint-inverse problem arising in camera-blur estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Alberge, F., Nikolova, M., Duhamel, P.: Blind identification/equalization using deterministic maximum likelihood and a partial prior on the input. IEEE Trans. Signal Process. 54(2), 724–737 (2006)

    MATH  Google Scholar 

  2. Andrieu, C., Doucet, A.: Simulated annealing for maximum a posteriori parameter estimation of hidden Markov models. IEEE Trans. Inform. Theory 46(3), 994–1004 (2000)

    MathSciNet  MATH  Google Scholar 

  3. Andrieu, C., Doucet, A., Fitzgerald, W.J.: An introduction to Monte Carlo methods for Bayesian data analysis. In: Mees, A.I. (ed.) Nonlinear Dynamics and Statistics, pp. 169–217. Springer, Berlin (2001)

    Google Scholar 

  4. Barakat, R., Sandler, B.H.: Determination of the wave-front aberration function from measured values of the point-spread function: a two-dimensional phase retrieval problem. JOSA A 9(10), 1715–1723 (1992)

    Google Scholar 

  5. Biernacki, C.: Testing for a global maximum of the likelihood. J. Comput. Graph. Stat. 14(3), 657–674 (2005)

    MathSciNet  Google Scholar 

  6. Blatt, D., Hero, A.O.: On tests for global maximum of the log-likelihood function. IEEE Trans. Inform. Theory 53(7), 2510–2525 (2007)

    MathSciNet  MATH  Google Scholar 

  7. Cox, D., Hinkley, D.: Theoretical Statistics. Chapman and Hall, London (1974)

    MATH  Google Scholar 

  8. Cox, D.R.: Tests of separate families of hypotheses. Proc. Fourth Berkeley Symp. Math. Stat. Probab. 1, 105–123 (1961)

    MathSciNet  MATH  Google Scholar 

  9. Cox, D.R.: Further results on tests of separate families of hypotheses. J. Royal Stat. Soc. Series B (Methodol.) 24(2), 406–424 (1962)

    MathSciNet  MATH  Google Scholar 

  10. Cramér, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)

    MATH  Google Scholar 

  11. Durand, S., Nikolova, M.: Stability of the minimizers of least squares with a non-convex regularization. part i: local behavior. Appl. Math. Optim. 53(2), 185–208 (2006)

    MathSciNet  MATH  Google Scholar 

  12. Durand, S., Nikolova, M.: Stability of the minimizers of least squares with a non-convex regularization. part ii: global behavior. Appl. Math. Optim. 53(3), 259–277 (2006)

    MathSciNet  MATH  Google Scholar 

  13. Fisher, R.A.: Theory of statistical estimation. In: Mathematical Proceedings of the Cambridge Philosophical Society, vol. 22, pp. 700–725. Cambridge University Press, Cambridge (1925)

  14. Gan, L., Jiang, J.: A test for global maximum. J. Am. Stat. Assoc. 94(447), 847–854 (1999)

    MathSciNet  MATH  Google Scholar 

  15. Gerwe, D.R., Johnson, M.M., Calef, B.: Local minima analysis of phase diverse phase retrieval using maximum likelihood. In: The Advanced Maui Optical and Space Surveillance Technical Conference (2008)

  16. Goodman, J.W.: Introduction to Fourier Optics. McGraw-Hill, New York (1996)

    Google Scholar 

  17. Isernia, T., Leone, G., Pierri, R.: Phase retrieval of radiated fields. Inverse Problems 11(1), 183 (1995)

    MathSciNet  MATH  Google Scholar 

  18. Kotz, S., Balakrishnan, N., Johnson, N.L.: Continuous Multivariate Distributions, Volume 1: Models and Applications, vol. 1. John Wiley & Sons, New Jersey (2004)

    MATH  Google Scholar 

  19. Kullback, S.: Information Theory and Statistics. Courier Corporation (1997)

  20. Le Cam, L.: Asymptotic Methods in Statistical Decision Theory. Springer Science & Business Media, Berlin (2012)

    Google Scholar 

  21. LeBlanc, J.W., Thelen, B.J., Hero, A.O.: Joint camera blur and pose estimation from aliased data. J. Opt. Soc. Am. A 35(4), 639–651 (2018)

    Google Scholar 

  22. Lehmann, E., Casella, G.: Theory of Point Estimation. Springer, Berlin (1998)

    MATH  Google Scholar 

  23. Liu, C., Rubin, D.B., Wu, Y.N.: Parameter expansion to accelerate em: the px-em algorithm. Biometrika 85(4), 755–770 (1998)

    MathSciNet  MATH  Google Scholar 

  24. Martial, G.: Strehl ratio and aberration balancing. J. Opt. Soc. Am. A, Opt. Image Sci. (USA) 8(1), 164–70 (1991)

    Google Scholar 

  25. Moretta, R., Pierri, R.: The “traps” issue in a non linear inverse problem: the phase retrieval in circular case. In: 2019 PhotonIcs and Electromagnetics Research Symposium-Spring (PIERS-Spring), pp. 552–559. IEEE (2019)

  26. Nikolova, M.: Estimées localement fortement homogenes. Comptes Rendus de l’Acad. des Sci.-Series I-Math. 325(6), 665–670 (1997)

    MathSciNet  MATH  Google Scholar 

  27. Nikolova, M.: Markovian reconstruction using a GNC approach. IEEE Trans. Image Process. 8(9), 1204–1220 (1999)

    Google Scholar 

  28. Nikolova, M.: Model distortions in Bayesian map reconstruction. Inverse Problems Imaging 1(2), 399 (2007)

    MathSciNet  MATH  Google Scholar 

  29. Nikolova, M., Hero, A.: Segmentation of a road from a vehicle-mounted radar and accuracy of the estimation. In: Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No. 00TH8511), pp. 284–289. IEEE (2000)

  30. Nikolova, M., Idier, J., Mohammad-Djafari, A.: Inversion of large-support ill-posed linear operators using a piecewise Gaussian MRF. IEEE Trans. Image Process. 7(4), 571–585 (1998)

    MathSciNet  MATH  Google Scholar 

  31. Nocedal, J., Wright, S.: Numerical Optimization. Springer, Berlin (1999)

    MATH  Google Scholar 

  32. Noll, R.J.: Zernike polynomials and atmospheric turbulence. J. Opt. Soc. Am. 66(3), 207–211 (1976)

    Google Scholar 

  33. Osher, S., Fedkiw, R., Piechor, K.: Level set methods and dynamic implicit surfaces. Appl. Mech. Rev. 57(3), B15–B15 (2004)

    Google Scholar 

  34. Rao, R.C.: Large sample tests of statistical hypotheses concerning several parameters with applications to problems of estimation. In: Mathematical Proceedings of the Cambridge Philosophical Society, vol. 44, pp. 50–57. Cambridge University Press, Cambridge (1948)

  35. Sethian, J.A.: A fast marching level set method for monotonically advancing fronts. Proc. Natl. Acad. Sci. 93(4), 1591–1595 (1996)

    MathSciNet  MATH  Google Scholar 

  36. Sharman, K., McClurkin, G.: Genetic algorithms for maximum likelihood parameter estimation. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2716–2719. New York (1989)

  37. Silvey, S.D.: The Lagrangian multiplier test. Ann. Math. Stat. 30(2), 389–407 (1959)

    MathSciNet  MATH  Google Scholar 

  38. Stigler, S.M., et al.: The epic story of maximum likelihood. Stat. Sci. 22(4), 598–620 (2007)

    MathSciNet  MATH  Google Scholar 

  39. Vapnik, V.: The support vector method of function estimation. In: Suykens, J.A.K., Vandewalle, J. (eds.) Nonlinear Modeling, pp. 55–85. Springer, Berlin (1998)

    Google Scholar 

  40. Wald, A.: Tests of statistical hypotheses concerning several parameters when the number of observations is large. Trans. Am. Math. Soc. 54(3), 426–482 (1943)

    MathSciNet  MATH  Google Scholar 

  41. Wald, A.: Note on the consistency of the maximum likelihood estimate. Ann. Math. Stat. 20(4), 595–601 (1949)

    MathSciNet  MATH  Google Scholar 

  42. Wang, S.Q., He, J.H.: Nonlinear oscillator with discontinuity by parameter-expansion method. Chaos, Solitons Fractals 35(4), 688–691 (2008)

    MathSciNet  MATH  Google Scholar 

  43. White, H.: Maximum likelihood estimation of misspecified models. Econom. J. Econom. Soc. 50, 1–25 (1982)

    MathSciNet  MATH  Google Scholar 

  44. Wilks, S.S.: The large-sample distribution of the likelihood ratio for testing composite hypotheses. Ann. Math. Stat. 9(1), 60–62 (1938)

    MATH  Google Scholar 

  45. Zernike, vF: Beugungstheorie des schneidenver-fahrens und seiner verbesserten form, der phasenkontrastmethode. Physica 1(7), 689–704 (1934)

    MATH  Google Scholar 

Download references

Acknowledgements

This work was partially supported by ARO grant W911NF-15-1-0479 and a DOE NNSA grant to the University of Michigan Consortium on Verification Technology.

Author information

Authors and Affiliations

  1. University of Michigan, Ann Arbor, MI, 48109, USA

    Joel W. LeBlanc, Brian J. Thelen & Alfred O. Hero

  2. Michigan Tech Research Institute, Ann Arbor, MI, 48105, USA

    Joel W. LeBlanc & Brian J. Thelen

Authors
  1. Joel W. LeBlanc
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brian J. Thelen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alfred O. Hero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joel W. LeBlanc.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

LeBlanc, J.W., Thelen, B.J. & Hero, A.O. Testing that a Local Optimum of the Likelihood is Globally Optimum Using Reparameterized Embeddings. J Math Imaging Vis 62, 858–871 (2020). https://doi.org/10.1007/s10851-020-00979-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-020-00979-0

Keywords

Search

Navigation