Fine-tuning LLMs with LoRA and Hessian-free optimizers

Stochastic Second-Order Methods in JAX

On the New method of Hessian-free second-order optimization

Implementation of Adaptive Hessian-free optimization.

Implementation of numerical optimization algorithms for logistic regression problem.

Tensorflow implement of Meta Learning with Hessian Free Approach in Deep Neural Nets Training

Matrix-multiplication-only KFAC; Code for ICML 2023 paper on Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning

Course project for CS771A: Introduction to Machine Learning

PyTorch implementation of the Hessian-free optimizer

PyTorch implementation of Hessian Free optimisation

ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning

PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks