Internal Algorithm Helpers

SwitchedEvolutionRelaxation(mass_matrix = nothing)

Method for updating the damping parameter in the PseudoTransient method based on "switched evolution relaxation" [8] SER method.

The optional mass_matrix generalizes the damping term from the identity I to a user-supplied (constant) matrix M, so that the damped Newton step becomes $(J + (1/α) M) δu = -F$. When mass_matrix === nothing, the damping is the scalar 1/α applied to the diagonal, recovering the classical identity-damped method exactly.

SwitchedEvolutionRelaxationCache <: AbstractDampingFunctionCache

Cache for the SwitchedEvolutionRelaxation method.

IdentityInitialization(alpha, structure)

Initialize the Jacobian to be an Identity Matrix scaled by alpha and maintain the structure as specified by structure.

TrueJacobianInitialization(structure, autodiff)

Initialize the Jacobian to be the true Jacobian and maintain the structure as specified by structure. autodiff is used to compute the true Jacobian and if not specified we make a selection automatically.

BroydenLowRankInitialization(alpha, threshold::Val)

An initialization for LimitedMemoryBroyden that uses a low rank approximation of the Jacobian. The low rank updates to the Jacobian matrix corresponds to what SciPy calls "simple".

FullStructure()

Stores the full matrix.

DiagonalStructure()

Preserves only the Diagonal of the Matrix.

InitializedApproximateJacobianCache(
    J, structure, alg, cache, initialized::Bool, internalnorm
)

A cache for Approximate Jacobian.

Arguments

• J: The current Jacobian.
• structure: The structure of the Jacobian.
• alg: The initialization algorithm.
• cache: The Jacobian cache NonlinearSolveBase.construct_jacobian_cache (if needed).
• initialized: A boolean indicating whether the Jacobian has been initialized.
• internalnorm: The norm to be used.

Interface

(cache::InitializedApproximateJacobianCache)(::Nothing)

Returns the current Jacobian cache.J with the proper structure.

InternalAPI.solve!(cache::InitializedApproximateJacobianCache, fu, u, ::Val{reinit})

Solves for the Jacobian cache.J and returns it. If reinit is true, then the Jacobian is reinitialized.

NoChangeInStateReset(;
    nsteps::Int = 3, reset_tolerance = nothing,
    check_du::Bool = true, check_dfu::Bool = true
)

Recommends a reset if the state or the function value has not changed significantly in nsteps steps. This is used in Broyden.

Keyword Arguments

• nsteps: the number of steps to check for no change. Defaults to 3.
• reset_tolerance: the tolerance for the reset check. Defaults to eps(real(eltype(u)))^(3 // 4).
• check_du: whether to check the state. Defaults to true.
• check_dfu: whether to check the function value. Defaults to true.

IllConditionedJacobianReset()

Recommend resetting the Jacobian if the current jacobian is ill-conditioned. This is used in Klement.

GoodBroydenUpdateRule()

Broyden Update Rule corresponding to "good broyden's method" [3].

BadBroydenUpdateRule()

Broyden Update Rule corresponding to "bad broyden's method" [3].

KlementUpdateRule()

Update rule for Klement.

LevenbergMarquardtTrustRegion(b_uphill)

Trust Region method for LevenbergMarquardt. This method is tightly coupled with the Levenberg-Marquardt method and works by directly updating the damping parameter instead of specifying a trust region radius.

Arguments

• b_uphill: a factor that determines if a step is accepted or rejected. The standard choice in the Levenberg-Marquardt method is to accept all steps that decrease the cost and reject all steps that increase the cost. Although this is a natural and safe choice, it is often not the most efficient. Therefore downhill moves are always accepted, but uphill moves are only conditionally accepted. To decide whether an uphill move will be accepted at each iteration $i$, we compute $\beta_i = \cos(v_{\text{new}}, v_{\text{old}})$, which denotes the cosine angle between the proposed velocity $v_{\text{new}}$ and the velocity of the last accepted step $v_{\text{old}}$. The idea is to accept uphill moves if the angle is small. To specify, uphill moves are accepted if $(1-\beta_i)^{b_{\text{uphill}}} C_{i+1} \le C_i$, where $C_i$ is the cost at iteration $i$. Reasonable choices for b_uphill are 1.0 or 2.0, with b_uphill = 2.0 allowing higher uphill moves than b_uphill = 1.0. When b_uphill = 0.0, no uphill moves will be accepted. Defaults to 1.0. See Section 4 of Transtrum and Sethna [1].

GenericTrustRegionScheme(;
    method = RadiusUpdateSchemes.Simple,
    max_trust_radius = nothing, initial_trust_radius = nothing,
    step_threshold = nothing, shrink_threshold = nothing, expand_threshold = nothing,
    shrink_factor = nothing, expand_factor = nothing
)

Trust Region Method that updates and stores the current trust region radius in trust_region. For any of the keyword arguments, if the value is nothing, then we use the value used in the respective paper.

Keyword Arguments

• radius_update_scheme: the choice of radius update scheme to be used. Defaults to RadiusUpdateSchemes.Simple which follows the conventional approach. Other available schemes are documented in RadiusUpdateSchemes,. These schemes have the trust region radius converging to zero that is seen to improve convergence. For more details, see [1].
• max_trust_radius: the maximal trust region radius. Defaults to max(norm(fu), maximum(u) - minimum(u)), except for RadiusUpdateSchemes.NLsolve where it defaults to Inf.
• initial_trust_radius: the initial trust region radius. Defaults to max_trust_radius / 11, except for RadiusUpdateSchemes.NLsolve where it defaults to u0_norm > 0 ? u0_norm : 1.
• step_threshold: the threshold for taking a step. In every iteration, the threshold is compared with a value r, which is the actual reduction in the objective function divided by the predicted reduction. If step_threshold > r the model is not a good approximation, and the step is rejected. Defaults to nothing.
• shrink_threshold: the threshold for shrinking the trust region radius. In every iteration, the threshold is compared with a value r which is the actual reduction in the objective function divided by the predicted reduction. If shrink_threshold > r the trust region radius is shrunk by shrink_factor. Defaults to nothing.
• expand_threshold: the threshold for expanding the trust region radius. If a step is taken, i.e step_threshold < r (with r defined in shrink_threshold), a check is also made to see if expand_threshold < r. If that is true, the trust region radius is expanded by expand_factor. Defaults to nothing.
• shrink_factor: the factor to shrink the trust region radius with if shrink_threshold > r (with r defined in shrink_threshold). Defaults to 0.25.
• expand_factor: the factor to expand the trust region radius with if expand_threshold < r (with r defined in shrink_threshold). Defaults to 2.0.

callback_into_cache!(cache, internalcache, args...)

Define custom operations on internalcache tightly coupled with the calling cache. args... contain the sequence of caches calling into internalcache.

This unfortunately makes code very tightly coupled and not modular. It is recommended to not use this functionality unless it can't be avoided (like in LevenbergMarquardt).

concrete_jac(alg::AbstractNonlinearSolveAlgorithm)::Bool

Whether the algorithm uses a concrete Jacobian.

assert_extension_supported_termination_condition(
    termination_condition, alg; abs_norm_supported = true
)

Validate that a wrapped external solver can support termination_condition.

Extension packages call this before dispatching to an external nonlinear solver whose termination API is more limited than NonlinearSolveBase's native modes.

Arguments

• termination_condition: A nonlinear termination mode or nothing.
• alg: The wrapper algorithm, used in the error message.

Keyword Arguments

• abs_norm_supported: Whether AbsNormTerminationMode is supported.

Returns

nothing when the termination condition is supported; otherwise throws an AssertionError.

construct_extension_function_wrapper(
    prob; alias_u0 = false, can_handle_oop = Val(false),
    can_handle_scalar = Val(false), make_fixed_point = Val(false),
    force_oop = Val(false)
)

Adapt a SciML nonlinear problem into the flattened residual function expected by an external solver wrapper.

This developer API handles in-place vs out-of-place problems, scalar promotion, optional fixed-point residual conversion, and vector reshaping.

Arguments

• prob: A SciML nonlinear problem.

Keyword Arguments

• alias_u0: Allow the returned initial vector to alias prob.u0.
• can_handle_oop: Whether the external solver can call out-of-place residuals.
• can_handle_scalar: Whether the external solver accepts scalar states directly.
• make_fixed_point: Convert a fixed-point residual from f(u) to f(u) + u.
• force_oop: Force an out-of-place wrapper even for in-place problems when possible.

Returns

A tuple (f, u0, resid) containing the wrapped residual, flattened initial state, and flattened residual prototype.

Examples

using NonlinearSolveBase, SciMLBase

prob = NonlinearProblem((u, p) -> u^2 - 2, 1.0)
f, u0, resid = NonlinearSolveBase.construct_extension_function_wrapper(
    prob; can_handle_oop = Val(true), can_handle_scalar = Val(true)
)

construct_extension_jac(
    prob, alg, u0, fu;
    can_handle_oop = Val(false), can_handle_scalar = Val(false),
    autodiff = nothing, initial_jacobian = Val(false), kwargs...
)

Construct a Jacobian callback for an external solver wrapper.

The returned callback uses NonlinearSolveBase's AD selection, analytic Jacobian handling, and scalar adaptation rules so wrapper packages do not need to duplicate Jacobian setup.

Arguments

• prob: A SciML nonlinear problem.
• alg: The external solver wrapper algorithm.
• u0: Flattened or solver-compatible initial state.
• fu: Residual prototype at u0.

Keyword Arguments

• can_handle_oop: Whether the external solver can call out-of-place Jacobians.
• can_handle_scalar: Whether scalar states should remain scalar.
• autodiff: ADTypes backend or nothing for automatic selection.
• initial_jacobian: Return the initial Jacobian together with the callback.
• kwargs...: Additional keywords passed to construct_jacobian_cache.

Returns

J when initial_jacobian = Val(false), or (J, J0) when initial_jacobian = Val(true).

select_forward_mode_autodiff(prob, ad; warn_check_mode = true)

Choose a forward-mode-compatible automatic differentiation backend for prob.

If ad is an AbstractADType, the backend is returned when it is available and compatible with the problem. If ad === nothing, NonlinearSolveBase selects the first available compatible backend from its preferred forward-mode list.

Arguments

• prob: A SciML nonlinear problem.
• ad: An ADTypes backend or nothing.

Keyword Arguments

• warn_check_mode: Emit a warning when ad is not a forward-mode backend.

Returns

An ADTypes backend suitable for forward-mode differentiation.

Examples

using ADTypes, NonlinearSolveBase, SciMLBase

prob = NonlinearProblem((u, p) -> u^2 - 2, 1.0)
NonlinearSolveBase.select_forward_mode_autodiff(prob, AutoForwardDiff())

select_reverse_mode_autodiff(prob, ad; warn_check_mode = true)

Choose a reverse-mode-compatible automatic differentiation backend for prob.

If ad === nothing, NonlinearSolveBase selects the first available compatible backend from its preferred reverse-mode list. Finite differencing backends are accepted as explicit fallback choices.

Arguments

• prob: A SciML nonlinear problem.
• ad: An ADTypes backend or nothing.

Keyword Arguments

• warn_check_mode: Emit a warning when ad is not a reverse-mode backend.

Returns

An ADTypes backend suitable for reverse-mode differentiation.

select_jacobian_autodiff(prob, ad)

Choose an automatic differentiation backend for constructing Jacobians for prob.

If ad === nothing, NonlinearSolveBase prefers a compatible forward-mode backend that is not finite differencing, then falls back to compatible reverse-mode or finite-difference backends.

Arguments

• prob: A SciML nonlinear problem.
• ad: An ADTypes backend or nothing.

Returns

An ADTypes backend suitable for Jacobian construction.

L2_NORM(u)

Compute the Euclidean norm used by NonlinearSolve internals.

The implementation has fast paths for numbers, dense arrays, and static arrays, and falls back to norm(u, 2) for other array-like states.

Arguments

• u: A scalar or array-like state.

Examples

using NonlinearSolveBase

NonlinearSolveBase.L2_NORM([3.0, 4.0])

Linf_NORM(u)

Compute the infinity norm used by NonlinearSolve internals.

Arguments

• u: A scalar or array-like state.

Examples

using NonlinearSolveBase

NonlinearSolveBase.Linf_NORM([-3.0, 4.0])

UNITLESS_ABS2(x)

Return a unitless squared magnitude for x.

This developer API is used by nonlinear solver internals to compare residuals, steps, and state values without preserving physical units. Numbers use abs2; arrays and nested SciML array containers reduce over their stored values.

Arguments

• x: A number, array, AbstractVectorOfArray, or ArrayPartition.

Examples

using NonlinearSolveBase

NonlinearSolveBase.UNITLESS_ABS2([3.0, 4.0])

NAN_CHECK(x)::Bool

Return true when x or any value stored in x is NaN.

This developer API is used by solver implementations before accepting iterates and residuals.

Arguments

• x: A scalar or array-like value to inspect.

Examples

using NonlinearSolveBase

NonlinearSolveBase.NAN_CHECK([1.0, NaN])

get_tolerance([u], η, ::Type{T})

Convert or choose a nonlinear solver tolerance of real type T.

When η === nothing, NonlinearSolve chooses a default tolerance based on T and, for array-free scalar/static states, uses a GPU-compatible exponent path.

Arguments

• u: Optional state value used by some specialization paths.
• η: User-provided tolerance or nothing.
• T: Target numeric type.

Examples

using NonlinearSolveBase

NonlinearSolveBase.get_tolerance(nothing, Float64)
NonlinearSolveBase.get_tolerance([1.0], nothing, Float64)

nonlinearsolve_forwarddiff_solve(prob, alg, args...; kwargs...)

Solve prob through a ForwardDiff-aware wrapper and return the primal solution together with parameter partials.

This is developer API for solver packages that need to propagate dual-number sensitivities through specialized nonlinear solve implementations.

For box-constrained least-squares problems, sensitivities use the projected stationarity equation, including active bounds. They assume parameter-independent bounds, a locally unchanged active set, and a nonsingular stationarity Jacobian on the free variables. A classical derivative need not exist where the active set changes.

Arguments

• prob: A SciML nonlinear problem whose parameters may carry ForwardDiff dual values.
• alg: The nonlinear solver algorithm.
• args...: Additional positional arguments forwarded to solve.

Keyword Arguments

All keyword arguments are forwarded to the underlying nonlinear solve.

Returns

A pair (sol, partials) where sol is the primal nonlinear solution and partials contains the propagated parameter partials.

nonlinearsolve_dual_solution(u, partials, p)

Reconstruct a dual-valued nonlinear solution from a primal state and parameter partials.

This is developer API paired with nonlinearsolve_forwarddiff_solve.

Arguments

• u: The primal nonlinear solution state.
• partials: The partial derivatives returned by the ForwardDiff solve path.
• p: Original parameter value, used to recover the dual tag and partial layout.