API Reference

build_loss_objective(prob::SciMLBase.AbstractSciMLProblem, alg, loss,
    adtype = SciMLBase.NoAD(), regularization = nothing, args...;
    priors = nothing, prob_generator = STANDARD_PROB_GENERATOR,
    kwargs...) -> SciMLBase.OptimizationFunction

Build an OptimizationFunction that solves prob for a candidate parameter vector and reduces the resulting solution to a scalar via loss.

The returned function has the signature cost(p, _ = nothing). For each p it regenerates a problem with prob_generator(prob, p), solves it with alg, and returns loss(sol). When loss is an L2Loss or LogLikeLoss, the solve saves only at the loss timepoints (saveat = loss.t, save_everystep = false, dense = false); otherwise the solve uses the passed keyword arguments as-is.

Arguments

• prob: the AbstractSciMLProblem (ODE, SDE, DAE, DDE, or another supported problem) to solve.
• alg: the solver algorithm matching prob.
• loss: a callable mapping a solution to a scalar cost, such as L2Loss, LogLikeLoss, or a user-defined loss(sol).
• adtype: the automatic differentiation choice passed to OptimizationFunction. Defaults to SciMLBase.NoAD().
• regularization: an optional callable of p added to the loss, such as Regularization. The default nothing adds no regularization.
• args...: extra positional arguments forwarded to solve.

Keywords

• priors: an array of univariate distributions, one per parameter, or a multivariate distribution. The negative log prior from prior_loss is added to the loss when supplied.
• prob_generator: a function (prob, p) -> newprob used to build the problem for each parameter vector. The default remakes prob with the element type of p and the new parameter vector.
• kwargs...: extra keyword arguments forwarded to the differential equation solve call.

Returns

An OptimizationFunction wrapping the cost function with adtype, ready to be used to construct an OptimizationProblem.

Examples

prob = ODEProblem((u, p, t) -> p[1] * u, [1.0], (0.0, 1.0), [0.5])
loss = L2Loss([0.0, 1.0], [1.0 1.5])
objective = build_loss_objective(prob, Tsit5(), loss)

multiple_shooting_objective(prob::SciMLBase.AbstractDEProblem, alg, loss,
    adtype = SciMLBase.NoAD(), regularization = nothing;
    priors = nothing, discontinuity_weight = 1.0,
    prob_generator = STANDARD_MS_PROB_GENERATOR,
    kwargs...) -> SciMLBase.OptimizationFunction

Build an OptimizationFunction that fits parameters by multiple shooting.

Multiple shooting splits prob.tspan into K shorter intervals and treats both the differential equation parameters and the initial state of each interval as optimization variables. The returned cost function solves each interval with alg, merges the segment solutions into a single trajectory, evaluates loss on it, and adds a discontinuity penalty for the mismatch between the end of each segment and the start of the next. This is often more robust than a single-shot objective, as in boundary value problems.

The parameter vector p is laid out as the concatenation of the per-interval initial states followed by the length(prob.p) differential equation parameters; the default prob_generator = STANDARD_MS_PROB_GENERATOR slices p into the correct per-segment problems.

Arguments

• prob: the AbstractDEProblem to fit.
• alg: the solver algorithm matching prob.
• loss: a callable mapping the merged solution to a scalar, such as L2Loss or LogLikeLoss.
• adtype: the automatic differentiation choice passed to OptimizationFunction. Defaults to SciMLBase.NoAD().
• regularization: an optional callable of p, such as Regularization. The default nothing adds no regularization.

Keywords

• priors: univariate distributions (one per parameter) or a multivariate distribution. The negative log prior from prior_loss is applied to the differential-equation parameters when supplied.
• discontinuity_weight: a scalar or array weight on the squared mismatch between consecutive segments. Defaults to 1.0.
• prob_generator: a function (prob, p, k) -> segment_prob producing the problem for interval k from p. The default splits the initial states and parameters according to the layout described above.
• kwargs...: extra keyword arguments forwarded to solve.

Returns

An OptimizationFunction wrapping the multiple-shooting cost with adtype.

Examples

objective = multiple_shooting_objective(prob, Tsit5(), L2Loss(tpoints, data))
value = objective(initial_state_and_parameter_vector)

two_stage_objective(prob::SciMLBase.AbstractDEProblem, tpoints, data,
    adtype = SciMLBase.NoAD(); kernel = EpanechnikovKernel()) ->
    SciMLBase.OptimizationFunction

Build an OptimizationFunction for the two-stage (non-parametric collocation) parameter estimation method.

The two-stage method avoids repeatedly solving the differential equation: it first builds a kernel-smoothed estimate of the state trajectory and its derivative from data, then measures, for a candidate p, the squared mismatch between the model right-hand side prob.f evaluated at the smoothed state and the smoothed derivative. It is less accurate than solve-based objectives but much faster, and is a good way to get into the neighborhood of good parameters before refining with another method.

Arguments

• prob: the AbstractDEProblem whose right-hand side is being fit. Both in-place and out-of-place right-hand sides are supported.
• tpoints: the timepoints at which data is sampled.
• data: the measured state values used for collocation smoothing.
• adtype: the automatic differentiation choice passed to OptimizationFunction. Defaults to SciMLBase.NoAD().

Keywords

• kernel: the collocation smoothing kernel, either a CollocationKernel instance or a Symbol selecting one of :Epanechnikov, :Uniform, :Triangular, :Quartic, :Triweight, :Tricube, :Gaussian, :Cosine, :Logistic, :Sigmoid, and :Silverman. Defaults to EpanechnikovKernel().

Returns

An OptimizationFunction wrapping a TwoStageCost with adtype. The returned cost also exposes the smoothed estimated_solution and estimated_derivative.

Examples

objective = two_stage_objective(prob, tpoints, data; kernel = :Triangular)
cost = objective([1.0])

L2Loss(t, data; differ_weight = nothing, data_weight = nothing,
    colloc_grad = nothing, dudt = nothing) -> L2Loss

An optimized L2-distance loss for fitting a differential equation solution to data. Calling the loss with a solution sol returns the weighted sum of squared residuals between sol and data at t. It returns Inf when the solution has an unsuccessful return code.

Arguments

• t: the timepoints at which the data are given.
• data: the measured values. Column i holds the state at t[i]; a vector is reshaped to a 1 x N matrix.

Keywords

• data_weight: a scalar or an array matching data that weights each squared residual. The default nothing gives unit weights.
• differ_weight: a scalar or an array weighting the first-difference residuals between consecutive solution and data values. The default nothing disables this term.
• colloc_grad: a matrix of collocation gradients for data, usually created by colloc_grad. When supplied, a derivative-residual term is added.
• dudt: a derivative buffer used with colloc_grad. The default nothing allocates a buffer automatically.

Fields

• t::T: the loss timepoints.
• data::D: the observed values, stored as a matrix.
• differ_weight::U: the first-difference weight or nothing.
• data_weight::W: the data weight or nothing.
• colloc_grad::G: the collocation derivative matrix or nothing.
• dudt::G: the derivative buffer used during evaluation.
• du_buf::B: an internal buffer used for derivative evaluation.

Examples

t = range(0, 1; length = 11)
data = reshape(sin.(t), 1, :)
loss = L2Loss(t, data; data_weight = 2.0)

LogLikeLoss(t, data_distributions)
LogLikeLoss(t, data_distributions, diff_distributions)
LogLikeLoss(t, data_distributions, diff_distributions, weight)

A negative log-likelihood loss for fitting a differential equation solution to a field of distributions. Calling a LogLikeLoss on a solution sol returns the negative total log-likelihood of sol under data_distributions at the timepoints t (so minimizing it performs maximum likelihood estimation), returning Inf if the solve was unsuccessful.

There are two forms for data_distributions:

• If data_distributions[i, j] is a UnivariateDistribution, it gives the likelihood at t[i] for component j.
• If data_distributions[i] is a MultivariateDistribution, it gives the likelihood at t[i] over the full state vector.

These distributions can be produced with fit_mle on a dataset against a chosen distribution type.

Arguments

• t: the timepoints at which the distributions apply.
• data_distributions: the likelihood distributions. A vector is reshaped to a 1 x N matrix.
• diff_distributions: an optional field of distributions for the first-difference terms sol[i] - sol[i - 1]. The three-argument constructor uses weight = 1 for this term.
• weight: the scalar multiplier for the first-difference likelihood term. Use nothing when diff_distributions is not supplied.

Fields

• t::T: the loss timepoints.
• data_distributions::D: the distributions for the observed state values.
• diff_distributions: the optional first-difference distributions.
• weight: the first-difference likelihood weight.

Examples

using Distributions

t = 0:2
data_distributions = [Normal(0, 1) for _ in t]
loss = LogLikeLoss(t, data_distributions)

Regularization(λ, penalty = L2Penalty()) -> Regularization

A callable regularization term for use with an objective builder such as build_loss_objective. Calling it with a parameter vector p returns λ * value(penalty, p). The default penalty is L2Penalty().

penalty can be any penalty function supported by PenaltyFunctions.jl.

Fields

• λ::L: the scalar weight applied to the penalty term.
• penalty::P: the penalty function evaluated on the parameter vector.

Examples

regularization = Regularization(0.1)
penalty = regularization([1.0, 2.0])

TwoStageCost(cost_function, estimated_solution, estimated_derivative) -> TwoStageCost

The callable cost object produced by two_stage_objective.

Calling a TwoStageCost on a parameter vector p forwards to cost_function(p, _p), which measures how well the differential equation's right-hand side evaluated at the collocation-smoothed solution matches the collocation-smoothed derivative. The smoothed trajectory and its derivative are stored so they can be inspected after construction.

Fields

• cost_function::F: the underlying cost closure with signature (p, _p = nothing).
• estimated_solution::D: the kernel-smoothed state trajectory at the collocation points.
• estimated_derivative::D: the kernel-smoothed state derivative at the collocation points.

Examples

cost = TwoStageCost((p, _) -> sum(abs2, p), [1.0], [0.0])
value = cost([1.0])

prior_loss(prior, p) -> Real

Return the negative log prior of the parameter vector p under prior.

If eltype(prior) <: UnivariateDistribution, prior is treated as a collection of univariate distributions and the result is -sum(logpdf(prior[i], p[i])). Otherwise prior is treated as a single multivariate distribution and the result is -logpdf(prior, p). Adding this term to a loss turns maximum likelihood estimation into a maximum a posteriori (MAP) objective. It is used by build_loss_objective and multiple_shooting_objective when priors is supplied.

Arguments

• prior: a collection of univariate distributions or one multivariate distribution.
• p: the parameter vector at which to evaluate the prior.

Examples

using Distributions

priors = [Normal(1.0, 0.2), Normal(2.0, 0.5)]
map_penalty = prior_loss(priors, [1.1, 1.8])

colloc_grad(t, data) -> AbstractMatrix

Estimate the time-derivative of data by spline collocation, for use as the colloc_grad argument of L2Loss.

For each state (row of data), a cubic (third-order) Dierckx.Spline1D is fit to data against t and differentiated at the timepoints in t. The derivatives are returned in a matrix with the same shape as data, where column i is the estimate at t[i].

Arguments

• t: a length-N vector of strictly ordered timepoints.
• data: an m x N matrix of measured state values.

Returns

An m x N matrix of collocation-estimated derivatives.

Examples

t = range(0, 1; length = 11)
data = reshape(sin.(t), 1, :)
grad = colloc_grad(t, data)

l2lossgradient!(grad, sol, data, sensitivities, num_p) -> nothing

Compute, in place, the gradient of an L2 loss with respect to num_p parameters and write it into grad.

Given a solution sol, the target data, and the parameter sensitivities (where sensitivities[i] is the derivative of the solution with respect to parameter i), this accumulates grad[i] -= sum(2 * (data - sol) .* sensitivities[i]) over all state components and timepoints. grad is zeroed before accumulation. Returns nothing.

Arguments

• grad: a length-num_p vector, overwritten with the loss gradient.
• sol: the solution values, shaped like data.
• data: the target data.
• sensitivities: a collection of length num_p of parameter sensitivities, each shaped like data.
• num_p: the number of parameters.

Examples

grad = zeros(1)
sensitivities = [ones(1, 2)]
l2lossgradient!(grad, zeros(1, 2), ones(1, 2), sensitivities, 1)