OptimizationODE.jl

OptimizationODE.jl provides ODE-based optimization methods as a solver plugin for SciML's Optimization.jl. It wraps various ODE solvers to perform gradient-based optimization using continuous-time dynamics.

Installation

using Pkg
Pkg.add("OptimizationODE")

Usage

using OptimizationODE, OptimizationBase, ADTypes, SciMLBase

function f(x, p)
    return sum(abs2, x)
end

function g!(g, x, p)
    @. g = 2 * x
end

x0 = [2.0, -3.0]
p = []

f_manual = OptimizationFunction(f, SciMLBase.NoAD(); grad = g!)
prob_manual = OptimizationProblem(f_manual, x0)

opt = ODEGradientDescent(dt=0.01)
sol = solve(prob_manual, opt; maxiters=50_000)

@show sol.u
@show sol.objective

Local Gradient-based Optimizers

All provided optimizers are gradient-based local optimizers that solve optimization problems by integrating gradient-based ODEs to convergence:

ODEGradientDescent(dt=...) — performs basic gradient descent using the explicit Euler method. This is a simple and efficient method suitable for small-scale or well-conditioned problems.
RKChebyshevDescent() — uses the ROCK2 solver, a stabilized explicit Runge-Kutta method suitable for stiff problems. It allows larger step sizes while maintaining stability.
RKAccelerated() — leverages the Tsit5 method, a 5th-order Runge-Kutta solver that achieves faster convergence for smooth problems by improving integration accuracy.
HighOrderDescent() — applies Vern7, a high-order (7th-order) explicit Runge-Kutta method for even more accurate integration. This can be beneficial for problems requiring high precision.

You can also define a custom optimizer using the generic ODEOptimizer(solver; dt=nothing) constructor by supplying any ODE solver supported by OrdinaryDiffEq.jl.

OptimizationODE.ODEOptimizer — Type

ODEOptimizer(solver)

Optimization algorithm that solves the gradient flow steady-state problem with the supplied ODE solver.

source

OptimizationODE.ODEGradientDescent — Function

ODEGradientDescent()

ODE-based gradient descent optimizer using the explicit Euler method.

source

OptimizationODE.RKChebyshevDescent — Function

RKChebyshevDescent()

ODE-based gradient descent optimizer using the stabilized ROCK2 Runge-Kutta method.

source

OptimizationODE.RKAccelerated — Function

RKAccelerated()

ODE-based optimizer using the Tsit5 Runge-Kutta method for accelerated gradient flow integration.

source

OptimizationODE.HighOrderDescent — Function

HighOrderDescent()

ODE-based optimizer using the high-order Vern7 Runge-Kutta method.

source

DAE-based Optimizers

Warn

DAE-based optimizers are still experimental and a research project. Use with caution.

In addition to ODE-based optimizers, OptimizationODE.jl provides optimizers for differential-algebraic equation (DAE) constrained problems:

DAEMassMatrix() — uses the Rodas5P solver (from OrdinaryDiffEq.jl) for DAE problems with a mass matrix formulation.
DAEOptimizer(IDA()) — uses the IDA solver (from Sundials.jl) for DAE problems with index variable support (requires using Sundials)

You can also define a custom optimizer using the generic ODEOptimizer(solver) or DAEOptimizer(solver) constructor by supplying any ODE or DAE solver supported by OrdinaryDiffEq.jl or Sundials.jl.

OptimizationODE.DAEOptimizer — Type

DAEOptimizer(solver)

Optimization algorithm for constrained problems represented as differential algebraic equation steady-state systems.

source

OptimizationODE.DAEMassMatrix — Function

DAEMassMatrix()

DAE-based optimizer using a Rodas5P mass-matrix formulation.

source

Interface Details

All optimizers require gradient information (either via automatic differentiation or manually provided grad!). The optimization is performed by integrating the ODE defined by the negative gradient until a steady state is reached.