Bounded Solvers

FastShortcutBoundedPolyalg(; concrete_jac = nothing, linsolve = nothing,
    autodiff = nothing, jvp_autodiff = nothing, vjp_autodiff = nothing, gtol = nothing, must_support_postcondition = false)

A polyalgorithm for real, box-constrained nonlinear systems and nonlinear least squares. It tries BoundedTrustRegion, followed by BoundedGaussNewton with projected backtracking if the first method fails. Each stage starts from the supplied initial guess and operates in the original bounded coordinates.

This is the default for NonlinearProblem and NonlinearLeastSquaresProblem with lb or ub. An out-of-bounds initial guess is projected onto the box, including when this polyalgorithm is selected explicitly. Least-squares problems may succeed at a constrained stationary point with nonzero residual; nonlinear systems must satisfy the residual tolerance. These are local methods and do not guarantee a global minimum.

concrete_jac, linsolve, and the differentiation backends are forwarded to both stages. Sparse Jacobian prototypes and matrix-free Krylov solvers use the usual Jacobian and linear-solver caches. gtol sets the projected-gradient tolerance; its default is sqrt(eps(T)) for least squares. For nonlinear systems, the trust-region stage disables this check by default and Gauss–Newton uses zero; detecting stationarity without a converged residual returns ReturnCode.Stalled.

Set must_support_postcondition = true to restrict the sequence to BoundedTrustRegion, which supports iterate correctors. Default algorithm selection sets this when a postcondition is supplied to the solve; pass the corrector itself to solve or init.

See the bounded solver recommendations for alternatives. The stage order may change as the benchmark coverage grows.

SobolMultistart(alg = FastShortcutBoundedPolyalg(); nstarts::Int = 16,
    search_scale = 10, early_exit::Bool = true, restoration::Bool = true,
    restoration_alg = nothing)

A deterministic multistart wrapper for NonlinearProblem and NonlinearLeastSquaresProblem, intended for box-constrained solves where a local method can converge to a constrained stationary point on an active bound that is not a root of the system. This wrapper is the default algorithm for problems with lb or ub; select alg directly for a purely local solve without restarts.

The wrapper runs alg from nstarts deterministic start points and returns the best result. The first start is always prob.u0, so the wrapper is never worse than a direct local solve; the remaining starts are points of a Sobol low-discrepancy sequence over the search region. Every start goes through the ordinary solve/init path, so linsolve, concrete_jac, sparse jac_prototype, JVP/VJP callbacks, AD backends, and keyword arguments such as abstol, reltol, and maxiters apply to each sub-solve exactly as they would to a direct solve(prob, alg; kwargs...) call.

Arguments

• alg: the local algorithm run from each start. Defaults to FastShortcutBoundedPolyalg.

Keyword Arguments

• nstarts: the number of start points, including prob.u0. Defaults to 16.
• search_scale: components with an infinite bound are sampled over a window of half-width search_scale * max(1, |u0_i|) centered on u0_i; components with a finite bound use the bound itself as the edge of the sampling region. Defaults to 10.
• early_exit: return immediately once a sub-solve reports a successful retcode. With early_exit = false every start runs and the feasible solution with the lowest residual norm is returned. Defaults to true.
• restoration: when a start stalls on an active bound, run one unbounded restoration probe from the stalled iterate with the nonfixed bounds relaxed. A probe that converges inside the original box returns that root as the start's result; a root outside the box confirms the boundary point is a genuine constrained solution and the stalled result is kept. Defaults to true.
• restoration_alg: the algorithm used for restoration probes. Defaults to alg.

The returned retcode is the sub-solve's own retcode on success and ReturnCode.Stalled when no start finds a solution; a failure is never reported as a success. The winning sub-solve is stored in the solution's original field. The wrapper also applies to problems without bounds, where all starts are drawn from the search_scale windows around u0.

BoundedTrustRegion(;
    concrete_jac = nothing, linsolve = nothing,
    max_trust_radius::Real = 0 // 1, initial_trust_radius::Real = 0 // 1,
    step_threshold::Real = 1 // 10000, shrink_threshold::Real = 1 // 4,
    expand_threshold::Real = 3 // 4, shrink_factor::Real = 1 // 4,
    expand_factor::Real = 2 // 1, max_shrink_times::Int = 32,
    gtol = nothing, vjp_autodiff = nothing, autodiff = nothing,
    jvp_autodiff = nothing,
)

A bound-constrained trust-region method for nonlinear systems and nonlinear least-squares problems. Bounds are handled directly in the original coordinates. The method compares a projected dogleg step with a feasible projected-Cauchy step and selects the step with the larger predicted reduction.

Unlike a coordinate transformation, this method permits iterates exactly on a bound and does not multiply the problem Jacobian by a transformation derivative that vanishes there. The initial guess must satisfy the problem bounds.

For a NonlinearLeastSquaresProblem, convergence is also detected when the infinity norm of the projected gradient is below gtol. This defaults to sqrt(eps(T)), where T is the working element type. Projected-gradient termination is disabled by default for a NonlinearProblem, because a constrained stationary point need not be a root. Setting gtol explicitly enables the check and reports such a point as ReturnCode.Stalled unless the residual has already converged.

Keyword Arguments

• max_trust_radius: the maximum trust-region radius. A value of zero selects Inf.
• initial_trust_radius: the initial radius. A value of zero selects max(norm(u0), one(T)).
• step_threshold: minimum actual-to-predicted reduction ratio for accepting a step.
• shrink_threshold, expand_threshold: reduction-ratio thresholds for changing the radius.
• shrink_factor, expand_factor: factors used to change the radius.
• gtol: projected-gradient tolerance for constrained stationarity.

TrustRegionReflective(; autodiff = nothing, jvp_autodiff = nothing, vjp_autodiff = nothing,
    linsolve = nothing, concrete_jac = nothing, gtol = nothing,
    initial_trust_radius = 1, max_trust_radius = Inf)

Coleman–Li interior reflective trust-region method for box-constrained nonlinear least squares and nonlinear equations. The quadratic model uses distance-to-bound scaling and the Coleman–Li diagonal correction. Each iteration compares a strictly feasible scaled trust-region step, a reflected direction, and a scaled gradient step.

Exact-bound initial guesses are moved into the strict interior; fixed variables remain fixed. For least squares, gtol bounds the infinity norm of u - clamp(u - J'F, lb, ub) and defaults to sqrt(eps(eltype(u0))). For nonlinear equations, stationarity with a nonzero residual returns ReturnCode.Stalled, never success; the default gtol is zero to avoid stopping near a root before residual convergence. abstol controls the default residual norm termination; other termination modes can be selected with termination_condition. Initial guesses must be feasible.

linsolve, concrete_jac, autodiff, jvp_autodiff, and vjp_autodiff use the same Jacobian and LinearSolve caches as GaussNewton. A sparse jac_prototype stays sparse; choosing a Krylov solver constructs a Jacobian operator unless a concrete Jacobian is requested. Analytic jvp and vjp callbacks are supported. Square-only linear solvers use normal equations; rectangular least-squares solvers operate on the scaled, diagonally augmented system. Rank-deficient and underdetermined problems require a suitable linear solver, such as SVDFactorization() or KrylovJL_LSMR().

The trust-region subproblem uses the subspace spanned by the scaled gradient and the Gauss–Newton step. Only that one- or two-dimensional model is diagonalized. AutoFiniteDiff uses bound-aware stencils for both concrete Jacobians and operator products; fixed coordinates are never perturbed. linsolve_kwargs passed to solve or init are forwarded to the linear-solver cache, including tolerances and preconditioners.

The positive initial_trust_radius is measured in scaled coordinates and is capped by max_trust_radius. Only real floating-point states and real residuals are supported.

Based on Coleman and Li, An Interior Trust Region Approach for Nonlinear Minimization Subject to Bounds, SIAM J. Optimization 6 (1996), 418–445, doi:10.1137/0806023.

BoundedLevenbergMarquardt(; autodiff = nothing, jvp_autodiff = nothing, vjp_autodiff = nothing,
    linsolve = nothing, concrete_jac = nothing, gtol = nothing,
    damping = 1e-3, max_backtracks = 40)

Native bound-constrained Levenberg–Marquardt method. An active-set solve minimizes the regularized Gauss–Newton model over the box, followed by a feasible Armijo line search. A projected-gradient line search supplies a fallback when the model step is unsuitable. The positive damping initializes residual-scaled regularization and is adapted using the agreement between actual and predicted reduction. max_backtracks limits each line search through the shared LineSearch.ProjectedBackTracking cache.

Uses the shared Jacobian and linear-solver caches, preserving sparse and operator representations. Bounds, linear solver and AD options, gtol, real state requirements, and root-versus-least-squares termination follow TrustRegionReflective, except that iterates may lie exactly on a bound. Fixed coordinates are eliminated from the subproblem. No coordinate transformation is used.

The constrained model and projected-gradient globalization follow the strategy of Kanzow, Yamashita and Fukushima, Levenberg–Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints, Journal of Computational and Applied Mathematics 172 (2004), 375–397. This implementation uses an active-set box subproblem rather than a general convex-programming solver.

Dogbox(; autodiff = nothing, jvp_autodiff = nothing, vjp_autodiff = nothing,
    linsolve = nothing, concrete_jac = nothing, gtol = nothing,
    initial_trust_radius = 1, max_trust_radius = Inf)

Rectangular trust-region dogleg method for box-constrained nonlinear least squares and nonlinear equations. The step follows the dogleg path from a constrained Cauchy point toward the Gauss–Newton step in the free variables, inside the intersection of the box and an infinity-norm trust region.

Uses the shared Jacobian and linear-solver caches, preserving sparse and operator representations. Rank deficiency requires a suitable linear solver and can cause slow convergence, so prefer BoundedLevenbergMarquardt for such problems. Bounds, autodiff, gtol, real state requirements, and root-versus-least-squares termination follow TrustRegionReflective, except that iterates can lie exactly on a bound and the trust radius uses the infinity norm in the original coordinates.

See Voglis and Lagaris, A Rectangular Trust Region Dogleg Approach for Unconstrained and Bound Constrained Nonlinear Optimization, WSEAS Applied Mathematics (2004).

BoundedGaussNewton(; globalization = :linesearch, autodiff = nothing, jvp_autodiff = nothing,
    vjp_autodiff = nothing, linsolve = nothing, concrete_jac = nothing, gtol = nothing,
    initial_trust_radius = 1, max_trust_radius = Inf, max_backtracks = 40)

Active-set Gauss–Newton method in the original bounded coordinates. Variables fixed by the box or satisfying the outward-gradient active-bound condition are excluded from the linear least-squares solve. For square nonlinear equations this gives a feasible reduced Newton path; a stationary point with nonzero residual is not reported as a root.

globalization selects :linesearch (projected Armijo search), :trustregion (spherical trust region with a projected Cauchy safeguard), or :trustregion_linesearch (trust region with a projected line-search fallback after rejection). max_backtracks bounds each line search. Projected searches use the shared LineSearch.ProjectedBackTracking cache. The trust radius is measured in the original coordinates.

Bounds, autodiff, gtol, real state requirements, and least-squares termination follow TrustRegionReflective, except that exact-bound iterates are permitted. The reduced systems use the shared Jacobian and linear-solver caches, preserving sparse and operator representations and honoring the same linear-solver and AD options. This is a Gauss–Newton model of the residual merit function, not an exact-Hessian constrained optimization method.