research-article

ROPTLIB: An Object-Oriented C++ Library for Optimization on Riemannian Manifolds

Authors:

Kyle A. Gallivan,

Paul HandAuthors Info & Claims

ACM Transactions on Mathematical Software (TOMS), Volume 44, Issue 4

Article No.: 43, Pages 1 - 21

https://doi.org/10.1145/3218822

Published: 31 July 2018 Publication History

Abstract

Riemannian optimization is the task of finding an optimum of a real-valued function defined on a Riemannian manifold. Riemannian optimization has been a topic of much interest over the past few years due to many applications including computer vision, signal processing, and numerical linear algebra. The substantial background required to successfully design and apply Riemannian optimization algorithms is a significant impediment for many potential users. Therefore, multiple packages, such as Manopt (in Matlab) and Pymanopt (in Python), have been developed. This article describes ROPTLIB, a C++ library for Riemannian optimization. Unlike prior packages, ROPTLIB simultaneously achieves the following goals: (i) it has user-friendly interfaces in Matlab, Julia, and C++; (ii) users do not need to implement manifold- and algorithm-related objects; (iii) it provides efficient computational time due to its C++ core; (iv) it implements state-of-the-art generic Riemannian optimization algorithms, including quasi-Newton algorithms; and (v) it is based on object-oriented programming, allowing users to rapidly add new algorithms and manifolds.

References

[1]

T. Abrudan, J. Eriksson, and V. Koivunen. 2008. Steepest descent algorithms for optimization under unitary matrix constraint. IEEE Transactions on Signal Processing 56, 3 (Mar. 2008), 1134--1147.

Digital Library

[2]

T. Abrudan, J. Eriksson, and V. Koivunen. 2009. Conjugate gradient algorithm for optimization under unitary matrix constraint. Signal Processing (Elsevier) 89, 9 (Sept. 2009), 1704--1714.

Digital Library

[3]

T. E. Abrudan. 2007. Matlab codes for optimization under unitary matrix constraint. Retrieved July 2016 from http://legacy.spa.aalto.fi/sig-legacy/unitary_optimization/index.html.

[4]

P.-A. Absil, C. G. Baker, and K. A. Gallivan. 2007. Trust-region methods on Riemannian manifolds. Foundations of Computational Mathematics 7, 3 (2007), 303--330. http://link.springer.com/article/10.1007/s10208-005-0179-9.

[5]

P.-A. Absil and P.-Y. Gousenbourger. 2015. Differentiable Piecewise-Bezier Surfaces on Riemannian Manifolds. Technical Report. ICTEAM Institute, Universite Catholiqué de Louvain, Louvain-La-Neuve, Belgium.

[6]

P.-A. Absil, R. Mahony, and R. Sepulchre. 2008. Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ. http://sites.uclouvain.be/absil/amsbook/.

Digital Library

[7]

K. P. Adragni and S. Wu. 2013. Grassmann Manifold Optimization. Retrieved July 2016 from https://cran.r-project.org/web/packages/GrassmannOptim/index.html.

[8]

Angelos Barmpoutis, Baba C. Vemuri, Timothy M. Shepherd, and John R. Forder. 2007. Tensor splines for interpolation and approximation of DT-MRI with applications to segmentation of isolated rat hippocampi. IEEE Transactions on Medical Imaging 26, 11 (2007), 1537--1546.

[9]

N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre. 2014. Manopt, a Matlab Toolbox for optimization on manifolds. Journal of Machine Learning Research 15 (2014), 1455--1459. http://www.manopt.org.

Digital Library

[10]

Léopold Cambier and P.-A Absil. 2016. Robust low-rank matrix completion by Riemannian optimization. Siam Journal on Scientific Computing 38, 5 (2016) S440--S460.

Digital Library

[11]

E. J. Candès, Y. C. Eldar, T. Strohmer, and V. Voroninski. 2013. Phase retrieval via matrix completion. SIAM Journal on Imaging Sciences 6, 1 (2013), 199--225. arXiv:1109.0573v2.

[12]

A. Cherian and S. Sra. 2015. Riemannian dictionary learning and sparse coding for positive definite matrices. CoRR abs/1507.02772 (2015). http://arxiv.org/abs/1507.02772.

[13]

J. E. Dennis and R. B. Schnabel. 1983. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Springer, New Jersey.

Digital Library

[14]

M. Ehler. 2013. A C++ library for optimization on Riemannian manifolds. Retrieved July 2016 from http://homepage.univie.ac.at/manuel.graef/quadrature.php.

[15]

M. T. Harandi, C. Sanderson, R. Hartley, and B. C. Lovell. 2012. Sparse coding and dictionary learning for symmetric positive definite matrices: A kernel approach. In Proceedings of the European Conference on Computer Vision (2012).

[16]

S. Hosseini, Wen Huang, and R. Yousefpour. 2016. Line Search Algorithms for Locally Lipschitz Functions on Riemannian Manifolds. Technical Report INS Preprint No. 1626. Institutfür Numerische Simulation.

[17]

S. Hosseini and S. Sra. 2014. Geometric Optimization Toolbox. Retrieved July 2016 from http://suvrit.de/work/soft/gopt.html.

[18]

S. Hosseini and A. Uschmajew. 2016. A Riemannian gradient sampling algorithm for nonsmooth optimization on manifolds. Institut für Numerische Simulation, INS Preprint No. 1607.

[19]

W. Huang. 2013. Optimization Algorithms on Riemannian Manifolds with Applications. Ph.D. dissertation. Florida State University, Department of Mathematics.

[20]

W. Huang, P.-A. Absil, and K. A. Gallivan. 2015a. A Riemannian symmetric rank-one trust-region method. Mathematical Programming 150, 2 (Feb. 2015), 179--216. http://www.optimization-online.org/DB_FILE/2013/06/3905.pdfhttp://link.springer.com/10.1007/s10107-014-0765-1.

Digital Library

[21]

W. Huang, P.-A. Absil, and K. A. Gallivan. 2016a. A Riemannian BFGS method for nonconvex optimization problems. Lecture Notes in Computational Science and Engineering 112 (2016), 627--634.

[22]

W. Huang, P.-A. Absil, and K. A. Gallivan. 2016b. Intrinsic representation of tangent vectors and vector transport on matrix manifolds. Numerische Mathematik 136, 2 (2016), 523--543.

Digital Library

[23]

W. Huang, P.-A. Absil, K. A. Gallivan, and Paul Hand. 2016c. Riemannian Manifold Optimization Library. Retrieved November 2016 from http://www.math.fsu.edu/ whuang2/Indices/index_ROPTLIB.html.

[24]

W. Huang, P.-A. Absil, K. A. Gallivan, and Paul Hand. 2017. Riemannian Manifold Optimization Library for Reproducing Experiments. Retrieved November 2016 from https://www.math.fsu.edu/whuang2/papers/ROPTLIB.htm.

[25]

W. Huang, K. A. Gallivan, and P.-A. Absil. 2015b. A Broyden class of quasi-Newton methods for Riemannian optimization. SIAM Journal on Optimization 25, 3 (2015), 1660--1685.

[26]

W. Huang, K. A. Gallivan, Anuj Srivastava, and P.-A. Absil. 2015c. Riemannian optimization for registration of curves in elastic shape analysis. Journal of Mathematical Imaging and Vision 54, 3 (2015), 320--343.

Digital Library

[27]

W. Huang, K. A. Gallivan, and X. Zhang. 2017. Solving phaselift by low rank Riemannian optimization methods for complex semidefinite constraints. SIAM Journal on Scientific Computing 39, 5 (2017), B840--B859.

[28]

Sadeep Jayasumana, Richard Hartley, Mathieu Salzmann, Hongdong Li, and Mehrtash Harandi. 2013. Kernel methods on the Riemannian manifold of symmetric positive definite matrices. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 73--80.

Digital Library

[29]

B. Jeuris, R. Vandebril, and B. Vandereycken. 2012. A survey and comparison of contemporary algorithms for computing the matrix geometric mean. Electronic Transactions on Numerical Analysis 39 (2012), 379--402.

[30]

H. Kasai and B. Mishra. 2015. Riemannian preconditioning for tensor completion. Retrieved from http://arxiv.org/abs/1506.02159 arXiv: 1506.02159.

[31]

P. Li, Q. Wang, W. Zuo, and L. Zhang. 2013. Log-Euclidean kernels for sparse representation and dictionary learning. Proceedings of the IEEE International Conference on Computer Vision (2013), 1601--1608.

Digital Library

[32]

S. B. Lippman, J. Lajoie, and B. E. Moo. 2012. C++ Primer (5th ed.). Addison-Wesley Professional.

Digital Library

[33]

Melissa Marchand, Wen Huang, Arnaud Browet, Paul Van Dooren, and Kyle A. Gallivan. 2016a. A Riemannian optimization approach for role model extraction. In Proceedings of the 22nd International Symposium on Mathematical Theory of Networks and Systems. 58--64.

[34]

Melissa Marchand, Wen Huang, Kyle Gallivan, and Bradley Marchand. 2016b. Multi-input multi-output waveform optimization for synthetic aperture sonar. In Proc. SPIE 9823 (2016). 98231X--98231X--12.

[35]

S. Martin, A. M. Raim, W. Huang, and K. P. Adragni. 2016. ManifoldOptim: An R interface to the ROPTLIB library for Riemannian manifold optimization. 1--25. http://arxiv.org/abs/1612.03930.

[36]

J. C. Meza, R. A. Oliva, P. D. Hough, and P. J. Williams. 2007. OPT++: An objective-oriented toolkit for nonlinear optimization. ACM Transactions on Mathematical Software 33, 2 (2007), Article 12.

Digital Library

[37]

B. Mishra. 2014. A Riemannian Approach to Large-Sscale Constrained Least-Squares with Symmetries. Ph.D. dissertation. University of Liege.

[38]

H. Mittelmann. 2010. Decision tree for optimization software. Technical Report, School of Mathematical and Statistical Sciences, Arizona State University. http://plato.asu.edu/guide.html.

[39]

J. Nocedal and S. J. Wright. 2006. Numerical Optimization (2nd ed.). Springer.

[40]

X. Pennec, P. Fillard, and N. Ayache. 2006a. A Riemannian framework for tensor computing. International Journal of Computer Vision 66, 5255 (2006), 41--66.

Digital Library

[41]

X. Pennec, P. Fillard, and N. Ayache. 2006b. A Riemannian framework for tensor computing. International Journal of Computer Vision 66, 1 (2006), 41--66.

Digital Library

[42]

R. E. Perez, P. W. Jansen, and J. R. R. A. Martins. 2012. PyOpt: A python-based object-oriented framework for nonlinear constrained optimization. Structural and Multidisciplinary Optimization 45, 1 (2012), 101--118.

Digital Library

[43]

A. Rathore, W. Huang, and Absil P.-A. 2015. Riemannian Optimization Package. Retrieved July 2016 from https://sourceforge.net/projects/rieoptpack/.

[44]

W. Ring and B. Wirth. 2012. Optimization methods on Riemannian manifolds and their application to shape space. SIAM Journal on Optimization 22, 2 (Jan. 2012), 596--627. http://epubs.siam.org/doi/abs/10.1137/11082885X

[45]

H. Sato. 2016. A Dai--Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions. Computational Optimization and Applications 64, 1 (May 2016), 101--118.

Digital Library

[46]

H. Sato and T. Iwai. 2013. A Riemannian optimization approach to the matrix singular value decomposition. SIAM Journal on Optimization 23, 1 (2013), 188--212.

[47]

R. Sivalingam, D. Boley, V. Morellas, and N. Papanikolopoulos. 2014. Tensor sparse coding for positive definite matrices. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 3 (2014), 592--605.

Digital Library

[48]

F. J. Theis, T. P. Cason, and P.-A. Absil. 2009. Soft dimension reduction for ICA by joint diagonalization on the Stiefel manifold. In Proceedings of the 8th International Conference on Independent Component Analysis and Signal Separation, 354--361. http://link.springer.com/chapter/10.1007/978-3-642-00599-2_45.

Digital Library

[49]

J. Townsend, N. Koep, and S. Weichwald. 2016. Pymanopt: A python toolbox for optimization on manifolds using automatic differentiation. Journal of Machine Learning Research 17, 137 (2016), 1--5. http://jmlr.org/papers/v17/16-177.html.

Digital Library

[50]

B. Vandereycken. 2013. Low-rank matrix completion by Riemannian optimization—extended version. SIAM Journal on Optimization 23, 2 (2013), 1214--1236. http://arxiv.org/abs/1209.3834.

[51]

I. Waldspurger, A. D’Aspremont, and S. Mallat. 2015. Phase recovery, MaxCut and complex semidefinite programming. Mathematical Programming 149, 1 (Feb. 2015), 47--81.

Digital Library

[52]

Z. Wen and W. Yin. 2012. Optimization with Orthogonality Constraints. Retrieved July 2016 from http://optman.blogs.rice.edu/.

[53]

Y. You, W. Huang, K. A. Gallivan, and P. A. Absil. 2015. A Riemannian approach for computing geodesics in elastic shape analysis. In Proceedings of the 2015 IEEE Global Conference on Signal and Information Processing (GlobalSIP). 727--731.

[54]

Rohollah Yousefpour. 2016. Combination of steepest descent and BFGS methods for nonconvex nonsmooth optimization. Numerical Algorithms 72, 1 (May 2016), 57--90.

Digital Library

[55]

X. Yuan, W. Huang, P.-A. Absil, and K. A. Gallivan. 2016. A Riemannian limited-memory BFGS algorithm for computing the matrix geometric mean. Procedia Computer Science 80 (2016), 1--11.

Digital Library

[56]

Xinru Yuan, Wen Huang, P.-A. Absil, and K. A. Gallivan. 2017. A Riemannian Quasi-Newton Method for Computing the Karcher Mean of Symmetric Positive Definite Matrices. Technical Report FSU17-02. Florida State University. www.math.fsu.edu/ whuang2/papers/RMKMSPDM.htm.

Cited By

Fei YLiu YJia CLi ZWei XChen M(2025)A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian ManifoldACM Computing Surveys10.1145/370849857:5(1-37)Online publication date: 24-Jan-2025
https://dl.acm.org/doi/10.1145/3708498
Li CGuo GYi PHong Y(2024)Distributed Pose-Graph Optimization With Multi-Level Partitioning for Multi-Robot SLAMIEEE Robotics and Automation Letters10.1109/LRA.2024.33825319:6(4926-4933)Online publication date: Jun-2024
https://doi.org/10.1109/LRA.2024.3382531
Sato H(2023)Differential Geometric Approach to Optimization Problems and Its Applications to Robotics最適化問題に対する微分幾何学的アプローチとロボット工学への応用Journal of the Robotics Society of Japan10.7210/jrsj.41.53041:6(530-535)Online publication date: 2023
https://doi.org/10.7210/jrsj.41.530
Show More Cited By

Index Terms

ROPTLIB: An Object-Oriented C++ Library for Optimization on Riemannian Manifolds
1. Mathematics of computing
  1. Mathematical analysis
    1. Mathematical optimization
      1. Continuous optimization

Recommendations

Pymanopt: a python toolbox for optimization on manifolds using automatic differentiation

Optimization on manifolds is a class of methods for optimization of an objective function, subject to constraints which are smooth, in the sense that the set of points which satisfy the constraints admits the structure of a differentiable manifold. While ...
Kernel Methods on Riemannian Manifolds with Gaussian RBF Kernels
In this paper, we develop an approach to exploiting kernel methods with manifold-valued data. In many computer vision problems, the data can be naturally represented as points on a Riemannian manifold. Due to the non-Euclidean geometry of Riemannian ...
Kernel Methods on the Riemannian Manifold of Symmetric Positive Definite Matrices
CVPR '13: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition

Symmetric Positive Definite (SPD) matrices have become popular to encode image information. Accounting for the geometry of the Riemannian manifold of SPD matrices has proven key to the success of many algorithms. However, most existing methods only ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Mathematical Software

ACM Transactions on Mathematical Software Volume 44, Issue 4

December 2018

305 pages

ISSN:0098-3500

EISSN:1557-7295

DOI:10.1145/3233179

Editors:
Zhaojun Bai
University of California at Davis, USA
,
Wolfgang Bangerth
Colorado State University, USA

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 31 July 2018

Accepted: 01 May 2018

Revised: 01 April 2018

Received: 01 October 2016

Published in TOMS Volume 44, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Interuniversity Attraction Poles Programme initiated
Belgian Science Policy Office
FNRS

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
383
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)1

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fei YLiu YJia CLi ZWei XChen M(2025)A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian ManifoldACM Computing Surveys10.1145/370849857:5(1-37)Online publication date: 24-Jan-2025
https://dl.acm.org/doi/10.1145/3708498
Li CGuo GYi PHong Y(2024)Distributed Pose-Graph Optimization With Multi-Level Partitioning for Multi-Robot SLAMIEEE Robotics and Automation Letters10.1109/LRA.2024.33825319:6(4926-4933)Online publication date: Jun-2024
https://doi.org/10.1109/LRA.2024.3382531
Sato H(2023)Differential Geometric Approach to Optimization Problems and Its Applications to Robotics最適化問題に対する微分幾何学的アプローチとロボット工学への応用Journal of the Robotics Society of Japan10.7210/jrsj.41.53041:6(530-535)Online publication date: 2023
https://doi.org/10.7210/jrsj.41.530
Binatlı OGönen M(2023)MOKPE: drug–target interaction prediction via manifold optimization based kernel preserving embeddingBMC Bioinformatics10.1186/s12859-023-05401-124:1Online publication date: 5-Jul-2023
https://doi.org/10.1186/s12859-023-05401-1
Kimiaei MNeumaier AFaramarzi P(2023)New subspace method for unconstrained derivative-free optimizationACM Transactions on Mathematical Software10.1145/3618297Online publication date: 2-Sep-2023
https://dl.acm.org/doi/10.1145/3618297
Axen SBaran MBergmann RRzecki K(2023)Manifolds.jl: An Extensible Julia Framework for Data Analysis on ManifoldsACM Transactions on Mathematical Software10.1145/361829649:4(1-23)Online publication date: 15-Dec-2023
https://dl.acm.org/doi/10.1145/3618296
Han AMishra BJawanpuria PKumar PGao J(2023)Riemannian Hamiltonian Methods for Min-Max Optimization on ManifoldsSIAM Journal on Optimization10.1137/22M149268433:3(1797-1827)Online publication date: 2-Aug-2023
https://doi.org/10.1137/22M1492684
Guo GYi P(2023)An Asynchronous Parallel Algorithm Framework for Decentralized Pose Graph Optimization2023 3rd International Conference on Computer, Control and Robotics (ICCCR)10.1109/ICCCR56747.2023.10193900(158-163)Online publication date: 24-Mar-2023
https://doi.org/10.1109/ICCCR56747.2023.10193900
Peng QHuang W(2023)An image inpainting algorithm using exemplar matching and low-rank sparse priorInverse Problems10.1088/1361-6420/ad0c4240:1(015002)Online publication date: 27-Nov-2023
https://doi.org/10.1088/1361-6420/ad0c42
Arbel JGirard SNguyen HUsseglio-Carleve A(2023)Multivariate expectile-based distribution: Properties, Bayesian inference, and applicationsJournal of Statistical Planning and Inference10.1016/j.jspi.2022.12.001225(146-170)Online publication date: Jul-2023
https://doi.org/10.1016/j.jspi.2022.12.001
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents