SimpleNonlinearSolve.jl

SimpleNewtonRaphson(autodiff)
SimpleNewtonRaphson(; autodiff = nothing)

A low-overhead implementation of Newton-Raphson. This method is non-allocating on scalar and static array problems.

Keyword Arguments

• autodiff: determines the backend used for the Jacobian. Defaults to nothing (i.e. automatic backend selection). Valid choices include jacobian backends from DifferentiationInterface.jl.

SimpleBroyden(; linesearch = Val(false), alpha = nothing)

A low-overhead implementation of Broyden. This method is non-allocating on scalar and static array problems.

Keyword Arguments

• linesearch: If linesearch is Val(true), then we use the LiFukushimaLineSearch line search else no line search is used. For advanced customization of the line search, use Broyden from NonlinearSolve.jl.
• alpha: Scale the initial jacobian initialization with alpha. If it is nothing, we will compute the scaling using 2 * norm(fu) / max(norm(u), true).

SimpleHalley(autodiff)
SimpleHalley(; autodiff = nothing)

A low-overhead implementation of Halley's Method.

Keyword Arguments

• autodiff: determines the backend used for the Jacobian. Defaults to nothing (i.e. automatic backend selection). Valid choices include jacobian backends from DifferentiationInterface.jl.

SimpleKlement()

A low-overhead implementation of Klement [4]. This method is non-allocating on scalar and static array problems.

SimpleTrustRegion(;
    autodiff = AutoForwardDiff(), max_trust_radius = 0.0,
    initial_trust_radius = 0.0, step_threshold = nothing,
    shrink_threshold = nothing, expand_threshold = nothing,
    shrink_factor = 0.25, expand_factor = 2.0, max_shrink_times::Int = 32,
    nlsolve_update_rule = Val(false)
)

A low-overhead implementation of a trust-region solver. This method is non-allocating on scalar and static array problems.

Keyword Arguments

• autodiff: determines the backend used for the Jacobian. Defaults to nothing (i.e. automatic backend selection). Valid choices include jacobian backends from DifferentiationInterface.jl.
• max_trust_radius: the maximum radius of the trust region. Defaults to max(norm(f(u0)), maximum(u0) - minimum(u0)).
• initial_trust_radius: the initial trust region radius. Defaults to max_trust_radius / 11.
• step_threshold: the threshold for taking a step. In every iteration, the threshold is compared with a value r, which is the actual reduction in the objective function divided by the predicted reduction. If step_threshold > r the model is not a good approximation, and the step is rejected. Defaults to 0.0001. For more details, see Rahpeymaii, F.
• shrink_threshold: the threshold for shrinking the trust region radius. In every iteration, the threshold is compared with a value r which is the actual reduction in the objective function divided by the predicted reduction. If shrink_threshold > r the trust region radius is shrunk by shrink_factor. Defaults to 0.25. For more details, see Rahpeymaii, F.
• expand_threshold: the threshold for expanding the trust region radius. If a step is taken, i.e step_threshold < r (with r defined in shrink_threshold), a check is also made to see if expand_threshold < r. If that is true, the trust region radius is expanded by expand_factor. Defaults to 0.75.
• shrink_factor: the factor to shrink the trust region radius with if shrink_threshold > r (with r defined in shrink_threshold). Defaults to 0.25.
• expand_factor: the factor to expand the trust region radius with if expand_threshold < r (with r defined in shrink_threshold). Defaults to 2.0.
• max_shrink_times: the maximum number of times to shrink the trust region radius in a row, max_shrink_times is exceeded, the algorithm returns. Defaults to 32.
• nlsolve_update_rule: If set to Val(true), updates the trust region radius using the update rule from NLSolve.jl. Defaults to Val(false). If set to Val(true), few of the radius update parameters – step_threshold = 0.05, expand_threshold = 0.9, and shrink_factor = 0.5 – have different defaults.

SimpleDFSane(;
    σ_min::Real = 1e-10, σ_max::Real = 1e10, σ_1::Real = 1.0,
    M::Union{Int, Val} = Val(10), γ::Real = 1e-4, τ_min::Real = 0.1, τ_max::Real = 0.5,
    nexp::Int = 2, η_strategy::Function = (f_1, k, x, F) -> f_1 ./ k^2
)

A low-overhead implementation of the df-sane method for solving large-scale nonlinear systems of equations. For in depth information about all the parameters and the algorithm, see La Cruz et al. [2].

Keyword Arguments

• σ_min: the minimum value of the spectral coefficient σ_k which is related to the step size in the algorithm. Defaults to 1e-10.
• σ_max: the maximum value of the spectral coefficient σ_k which is related to the step size in the algorithm. Defaults to 1e10.
• σ_1: the initial value of the spectral coefficient σ_k which is related to the step size in the algorithm.. Defaults to 1.0.
• M: The monotonicity of the algorithm is determined by a this positive integer. A value of 1 for M would result in strict monotonicity in the decrease of the L2-norm of the function f. However, higher values allow for more flexibility in this reduction. Despite this, the algorithm still ensures global convergence through the use of a non-monotone line-search algorithm that adheres to the Grippo-Lampariello-Lucidi condition. Values in the range of 5 to 20 are usually sufficient, but some cases may call for a higher value of M. The default setting is 10.
• γ: a parameter that influences if a proposed step will be accepted. Higher value of γ will make the algorithm more restrictive in accepting steps. Defaults to 1e-4.
• τ_min: if a step is rejected the new step size will get multiplied by factor, and this parameter is the minimum value of that factor. Defaults to 0.1.
• τ_max: if a step is rejected the new step size will get multiplied by factor, and this parameter is the maximum value of that factor. Defaults to 0.5.
• nexp: the exponent of the loss, i.e. $f_k=||F(x_k)||^{nexp}$. The paper uses nexp ∈ {1,2}. Defaults to 2.
• η_strategy: function to determine the parameter η_k, which enables growth of $||F||^2$. Called as η_k = η_strategy(f_1, k, x, F) with f_1 initialized as $f_1=||F(x_1)||^{nexp}$, k is the iteration number, x is the current x-value and F the current residual. Should satisfy $η_k > 0$ and $∑ₖ ηₖ < ∞$. Defaults to $||F||^2 / k^2$.

SimpleLimitedMemoryBroyden(;
    threshold::Union{Val, Int} = Val(27), linesearch = Val(false), alpha = nothing
)

A limited memory implementation of Broyden. This method applies the L-BFGS scheme to Broyden's method.

If the threshold is larger than the problem size, then this method will use SimpleBroyden.

Keyword Arguments:

• linesearch: If linesearch is Val(true), then we use the LiFukushimaLineSearch line search else no line search is used. For advanced customization of the line search, use Broyden from NonlinearSolve.jl.
• alpha: Scale the initial jacobian initialization with alpha. If it is nothing, we will compute the scaling using 2 * norm(fu) / max(norm(u), true).