Defining OptimizationProblems

SciMLBase.OptimizationProblemType

Defines a optimization problem. Documentation Page: https://galacticoptim.sciml.ai/dev/API/optimization_problem/

Mathematical Specification of a Optimization Problem

To define an Optimization Problem, you simply need to give the function $f$ which defines the cost function to minimize:

\[min_u f(u,p)\]

$u₀$ is an initial guess of the minimum. f should be specified as f(u,p) and u₀ should be an AbstractArray (or number) whose geometry matches the desired geometry of u. Note that we are not limited to numbers or vectors for u₀; one is allowed to provide u₀ as arbitrary matrices / higher-dimension tensors as well.

Problem Type

Constructors

OptimizationProblem{iip}(f, x, p = SciMLBase.NullParameters(),;
                        lb = nothing,
                        ub = nothing,
                        lcons = nothing,
                        ucons = nothing,
                        sense = nothing,
                        kwargs...)

isinplace optionally sets whether the function is in-place or not. This is determined automatically, but not inferred. Note that for OptimizationProblem, in-place only refers to the Jacobian and Hessian functions, and thus by default if the OptimizationFunction is not defined directly then iip = true is done by default.

Parameters are optional, and if not given, then a NullParameters() singleton will be used, which will throw nice errors if you try to index non-existent parameters. Any extra keyword arguments are passed on to the solvers. For example, if you set a callback in the problem, then that callback will be added in every solve call.

lb and ub are the upper and lower bounds for box constraints on the optimization. They should be an AbstractArray matching the geometry of u, where (lb[I],ub[I]) is the box constraint (lower and upper bounds) for u[I].

lcons and ucons are the upper and lower bounds for equality constraints on the optimization. They should be an AbstractArray matching the geometry of u, where (lcons[I],ucons[I]) is the constraint (lower and upper bounds) for cons[I].

If f is a standard Julia function, it is automatically converted into an OptimizationFunction with NoAD(), i.e., no automatic generation of the derivative functions.

Any extra keyword arguments are captured to be sent to the optimizers.

Fields

  • f: the function in the problem.
  • u0: the initial guess for the optima.
  • p: the parameters for the problem. Defaults to NullParameters.
  • lb: the lower bounds for the optimization of u.
  • ub: the upper bounds for the optimization of u.
  • lcons: the vector of lower bounds for the constraints passed to OptimizationFunction. Defaults to nothing, implying no lower bounds for the constraints (i.e. the constraint bound is -Inf)
  • ucons: the vector of upper bounds for the constraints passed to OptimizationFunction. Defaults to nothing, implying no upper bounds for the constraints (i.e. the constraint bound is Inf)
  • sense: the objective sense, can take MaxSense or MinSense from Optimization.jl.
  • kwargs: the keyword arguments passed on to the solvers.

Inequality and Equality Constraints

Both inequality and equality constraints are defined by the f.cons function in the OptimizationFunction description of the problem structure. This f.cons is given as a function f.cons(u,p) which computes the value of the constraints at u. For example, take f.cons(u,p) = u[1] - u[2]. With these definitions, lcons and ucons define the bounds on the constraint that the solvers try to satisfy. If lcons and ucons are nothing, then there are no constraints bounds, meaning that the constraint is satisfied when -Inf < f.cons < Inf (which of course is always!). If lcons[i] = ucons[i] = 0, then the constraint is satisfied when f.cons(u,p)[i] = 0, and so this implies the equality constraint $u[1] = u[2]`. If `lcons[i] = ucons[i] = a`, then$u[1] - u[2] = a`` is the equality constraint.

Inequality constraints are then given by making lcons[i] != ucons[i]. For example, lcons[i] = -Inf and ucons[i] = 0 would imply the inequality constraint $u[1] <= u[2]$ since any f.cons[i] <= 0 satisfies the constraint. Similarly, lcons[i] = -1 and ucons[i] = 1 would imply that -1 <= f.cons[i] <= 1 is required or $-1 <= u[1] - u[2] <= 1$.

Note that these vectors must be sized to match the number of constraints, with one set of conditions for each constraint.