-
Bandicoot: C++ Library for GPU Linear Algebra and Scientific Computing
Authors:
Ryan R. Curtin,
Marcus Edel,
Conrad Sanderson
Abstract:
This report provides an introduction to the Bandicoot C++ library for linear algebra and scientific computing on GPUs, overviewing its user interface and performance characteristics, as well as the technical details of its internal design. Bandicoot is the GPU-enabled counterpart to the well-known Armadillo C++ linear algebra library, aiming to allow users to take advantage of GPU-accelerated comp…
▽ More
This report provides an introduction to the Bandicoot C++ library for linear algebra and scientific computing on GPUs, overviewing its user interface and performance characteristics, as well as the technical details of its internal design. Bandicoot is the GPU-enabled counterpart to the well-known Armadillo C++ linear algebra library, aiming to allow users to take advantage of GPU-accelerated computation for their existing codebases without significant changes. Adapting the same internal template meta-programming techniques that Armadillo uses, Bandicoot is able to provide compile-time optimisation of mathematical expressions within user code. The library is ready for production use and is available at https://coot.sourceforge.io. Bandicoot is distributed under the Apache 2.0 License.
△ Less
Submitted 6 August, 2023;
originally announced August 2023.
-
mlpack 4: a fast, header-only C++ machine learning library
Authors:
Ryan R. Curtin,
Marcus Edel,
Omar Shrit,
Shubham Agrawal,
Suryoday Basak,
James J. Balamuta,
Ryan Birmingham,
Kartik Dutt,
Dirk Eddelbuettel,
Rishabh Garg,
Shikhar Jaiswal,
Aakash Kaushik,
Sangyeon Kim,
Anjishnu Mukherjee,
Nanubala Gnana Sai,
Nippun Sharma,
Yashwant Singh Parihar,
Roshan Swain,
Conrad Sanderson
Abstract:
For over 15 years, the mlpack machine learning library has served as a "swiss army knife" for C++-based machine learning. Its efficient implementations of common and cutting-edge machine learning algorithms have been used in a wide variety of scientific and industrial applications. This paper overviews mlpack 4, a significant upgrade over its predecessor. The library has been significantly refacto…
▽ More
For over 15 years, the mlpack machine learning library has served as a "swiss army knife" for C++-based machine learning. Its efficient implementations of common and cutting-edge machine learning algorithms have been used in a wide variety of scientific and industrial applications. This paper overviews mlpack 4, a significant upgrade over its predecessor. The library has been significantly refactored and redesigned to facilitate an easier prototyping-to-deployment pipeline, including bindings to other languages (Python, Julia, R, Go, and the command line) that allow prototyping to be seamlessly performed in environments other than C++. mlpack is open-source software, distributed under the permissive 3-clause BSD license; it can be obtained at https://mlpack.org
△ Less
Submitted 1 February, 2023;
originally announced February 2023.
-
The ensmallen library for flexible numerical optimization
Authors:
Ryan R. Curtin,
Marcus Edel,
Rahul Ganesh Prabhu,
Suryoday Basak,
Zhihao Lou,
Conrad Sanderson
Abstract:
We overview the ensmallen numerical optimization library, which provides a flexible C++ framework for mathematical optimization of user-supplied objective functions. Many types of objective functions are supported, including general, differentiable, separable, constrained, and categorical. A diverse set of pre-built optimizers is provided, including Quasi-Newton optimizers and many variants of Sto…
▽ More
We overview the ensmallen numerical optimization library, which provides a flexible C++ framework for mathematical optimization of user-supplied objective functions. Many types of objective functions are supported, including general, differentiable, separable, constrained, and categorical. A diverse set of pre-built optimizers is provided, including Quasi-Newton optimizers and many variants of Stochastic Gradient Descent. The underlying framework facilitates the implementation of new optimizers. Optimization of an objective function typically requires supplying only one or two C++ functions. Custom behavior can be easily specified via callback functions. Empirical comparisons show that ensmallen outperforms other frameworks while providing more functionality. The library is available at https://ensmallen.org and is distributed under the permissive BSD license.
△ Less
Submitted 9 February, 2024; v1 submitted 29 August, 2021;
originally announced August 2021.
-
Flexible numerical optimization with ensmallen
Authors:
Ryan R. Curtin,
Marcus Edel,
Rahul Ganesh Prabhu,
Suryoday Basak,
Zhihao Lou,
Conrad Sanderson
Abstract:
This report provides an introduction to the ensmallen numerical optimization library, as well as a deep dive into the technical details of how it works. The library provides a fast and flexible C++ framework for mathematical optimization of arbitrary user-supplied functions. A large set of pre-built optimizers is provided, including many variants of Stochastic Gradient Descent and Quasi-Newton opt…
▽ More
This report provides an introduction to the ensmallen numerical optimization library, as well as a deep dive into the technical details of how it works. The library provides a fast and flexible C++ framework for mathematical optimization of arbitrary user-supplied functions. A large set of pre-built optimizers is provided, including many variants of Stochastic Gradient Descent and Quasi-Newton optimizers. Several types of objective functions are supported, including differentiable, separable, constrained, and categorical objective functions. Implementation of a new optimizer requires only one method, while a new objective function requires typically only one or two C++ methods. Through internal use of C++ template metaprogramming, ensmallen provides support for arbitrary user-supplied callbacks and automatic inference of unsupplied methods without any runtime overhead. Empirical comparisons show that ensmallen outperforms other optimization frameworks (such as Julia and SciPy), sometimes by large margins. The library is available at https://ensmallen.org and is distributed under the permissive BSD license.
△ Less
Submitted 15 November, 2023; v1 submitted 9 March, 2020;
originally announced March 2020.
-
ensmallen: a flexible C++ library for efficient function optimization
Authors:
Shikhar Bhardwaj,
Ryan R. Curtin,
Marcus Edel,
Yannis Mentekidis,
Conrad Sanderson
Abstract:
We present ensmallen, a fast and flexible C++ library for mathematical optimization of arbitrary user-supplied functions, which can be applied to many machine learning problems. Several types of optimizations are supported, including differentiable, separable, constrained, and categorical objective functions. The library provides many pre-built optimizers (including numerous variants of SGD and Qu…
▽ More
We present ensmallen, a fast and flexible C++ library for mathematical optimization of arbitrary user-supplied functions, which can be applied to many machine learning problems. Several types of optimizations are supported, including differentiable, separable, constrained, and categorical objective functions. The library provides many pre-built optimizers (including numerous variants of SGD and Quasi-Newton optimizers) as well as a flexible framework for implementing new optimizers and objective functions. Implementation of a new optimizer requires only one method and a new objective function requires typically one or two C++ functions. This can aid in the quick implementation and prototyping of new machine learning algorithms. Due to the use of C++ template metaprogramming, ensmallen is able to support compiler optimizations that provide fast runtimes. Empirical comparisons show that ensmallen is able to outperform other optimization frameworks (like Julia and SciPy), sometimes by large margins. The library is distributed under the BSD license and is ready for use in production environments.
△ Less
Submitted 9 December, 2018; v1 submitted 22 October, 2018;
originally announced October 2018.
-
A generic and fast C++ optimization framework
Authors:
Ryan R. Curtin,
Shikhar Bhardwaj,
Marcus Edel,
Yannis Mentekidis
Abstract:
The development of the mlpack C++ machine learning library (http://www.mlpack.org/) has required the design and implementation of a flexible, robust optimization system that is able to solve the types of arbitrary optimization problems that may arise all throughout machine learning problems. In this paper, we present the generic optimization framework that we have designed for mlpack. A key priori…
▽ More
The development of the mlpack C++ machine learning library (http://www.mlpack.org/) has required the design and implementation of a flexible, robust optimization system that is able to solve the types of arbitrary optimization problems that may arise all throughout machine learning problems. In this paper, we present the generic optimization framework that we have designed for mlpack. A key priority in the design was ease of implementation of both new optimizers and new objective functions to be optimized; therefore, implementation of a new optimizer requires only one method and implementation of a new objective function requires at most four functions. This leads to simple and intuitive code, which, for fast prototyping and experimentation, is of paramount importance. When compared to optimization frameworks of other libraries, we find that mlpack's supports more types of objective functions, is able to make optimizations that other frameworks do not, and seamlessly supports user-defined objective functions and optimizers.
△ Less
Submitted 17 November, 2017;
originally announced November 2017.
-
Designing and building the mlpack open-source machine learning library
Authors:
Ryan R. Curtin,
Marcus Edel
Abstract:
mlpack is an open-source C++ machine learning library with an emphasis on speed and flexibility. Since its original inception in 2007, it has grown to be a large project implementing a wide variety of machine learning algorithms, from standard techniques such as decision trees and logistic regression to modern techniques such as deep neural networks as well as other recently-published cutting-edge…
▽ More
mlpack is an open-source C++ machine learning library with an emphasis on speed and flexibility. Since its original inception in 2007, it has grown to be a large project implementing a wide variety of machine learning algorithms, from standard techniques such as decision trees and logistic regression to modern techniques such as deep neural networks as well as other recently-published cutting-edge techniques not found in any other library. mlpack is quite fast, with benchmarks showing mlpack outperforming other libraries' implementations of the same methods. mlpack has an active community, with contributors from around the world---including some from PUST. This short paper describes the goals and design of mlpack, discusses how the open-source community functions, and shows an example usage of mlpack for a simple data science problem.
△ Less
Submitted 30 August, 2017; v1 submitted 17 August, 2017;
originally announced August 2017.
