Multiple Shooting Functionality

Note

The form of multiple shooting found here is a specialized form for implicit layer deep learning (known as data shooting) which assumes full observability of the underlying dynamics and lack of noise. For a more general implementation of multiple shooting, see JuliaSimModelOptimizer. For an implementation more directly tied to parameter estimation against data, see DiffEqParamEstim.jl.

DiffEqFlux.multiple_shoot — Function
multiple_shoot(p, ode_data, tsteps, prob, loss_function,
    [continuity_loss = _default_continuity_loss], solver, group_size;
    continuity_term = 100, kwargs...)

Returns a total loss after trying a 'Direct multiple shooting' on ODE data and an array of predictions from each of the groups (smaller intervals). In Direct Multiple Shooting, the Neural Network divides the interval into smaller intervals and solves for them separately. The default continuity term is 100, implying any losses arising from the non-continuity of 2 different groups will be scaled by 100.

Arguments

  • p: The parameters of the Neural Network to be trained.
  • ode_data: Original Data to be modelled.
  • tsteps: Timesteps on which ode_data was calculated.
  • prob: ODE problem that the Neural Network attempts to solve.
  • loss_function: Any arbitrary function to calculate loss.
  • continuity_loss: Function that takes states $\hat{u}_{end}$ of group $k$ and $u_{0}$ of group $k+1$ as input and calculates prediction continuity loss between them. If no custom continuity_loss is specified, sum(abs, û_end - u_0) is used.
  • solver: ODE Solver algorithm.
  • group_size: The group size achieved after splitting the ode_data into equal sizes.
  • continuity_term: Weight term to ensure continuity of predictions throughout different groups.
  • kwargs: Additional arguments splatted to the ODE solver. Refer to the Local Sensitivity Analysis and Common Solver Arguments documentation for more details.

Returns

  • loss: Total data-fitting and continuity-penalty loss. Returns Inf if any group solve does not have a successful retcode.
  • group_predictions: Array of predictions, one entry per shooting group.

Examples

using DiffEqFlux, OrdinaryDiffEq, SciMLBase

f(u, p, t) = p .* u
prob = ODEProblem(f, [1.0], (0.0, 1.0), [1.0])
tsteps = range(0, 1; length = 6)
ode_data = reduce(hcat, ([exp(t)] for t in tsteps))
loss_function(u, û) = sum(abs2, u .- û)

loss, predictions = multiple_shoot(
    [1.0], ode_data, tsteps, prob, loss_function, Tsit5(), 3; abstol = 1e-8)
Note

The parameter 'continuity_term' should be a relatively big number to enforce a large penalty whenever the last point of any group doesn't coincide with the first point of next group.

source
multiple_shoot(p, ode_data, tsteps, ensembleprob, ensemblealg, loss_function,
    [continuity_loss = _default_continuity_loss], solver, group_size;
    continuity_term = 100, kwargs...)

Returns a total loss after trying a 'Direct multiple shooting' on ODE data and an array of predictions from each of the groups (smaller intervals). In Direct Multiple Shooting, the Neural Network divides the interval into smaller intervals and solves for them separately. The default continuity term is 100, implying any losses arising from the non-continuity of 2 different groups will be scaled by 100.

Arguments

  • p: The parameters of the Neural Network to be trained.
  • ode_data: Original Data to be modelled.
  • tsteps: Timesteps on which ode_data was calculated.
  • ensemble_prob: Ensemble problem that the Neural Network attempts to solve.
  • ensemble_alg: Ensemble algorithm, e.g. EnsembleThreads().
  • prob: ODE problem that the Neural Network attempts to solve.
  • loss_function: Any arbitrary function to calculate loss.
  • continuity_loss: Function that takes states $\hat{u}_{end}$ of group $k$ and $u_{0}$ of group $k+1$ as input and calculates prediction continuity loss between them. If no custom continuity_loss is specified, sum(abs, û_end - u_0) is used.
  • solver: ODE Solver algorithm.
  • group_size: The group size achieved after splitting the ode_data into equal sizes.
  • continuity_term: Weight term to ensure continuity of predictions throughout different groups.
  • kwargs: Additional arguments splatted to the ODE solver. Refer to the Local Sensitivity Analysis and Common Solver Arguments documentation for more details.

Returns

  • loss: Total data-fitting and continuity-penalty loss. Returns Inf if any ensemble group solve reports non-convergence.
  • group_predictions: Array of ensemble predictions, one entry per shooting group.

Examples

using DiffEqFlux, OrdinaryDiffEq, SciMLBase

f(u, p, t) = p .* u
prob = ODEProblem(f, [1.0], (0.0, 1.0), [1.0])
ensembleprob = EnsembleProblem(prob)
tsteps = range(0, 1; length = 6)
ode_data = reshape([exp(t) for t in tsteps], 1, :, 1)
loss_function(u, û) = sum(abs2, u .- û)

loss, predictions = multiple_shoot(
    [1.0], ode_data, tsteps, ensembleprob, EnsembleSerial(),
    loss_function, Tsit5(), 3; trajectories = 1, abstol = 1e-8)
Note

The parameter 'continuity_term' should be a relatively big number to enforce a large penalty whenever the last point of any group doesn't coincide with the first point of next group.

source
DiffEqFlux.group_ranges — Function
group_ranges(datasize, groupsize)

Get ranges that partition data of length datasize in groups of groupsize observations. If the data isn't perfectly dividable by groupsize, the last group contains the reminding observations.

Arguments

  • datasize: amount of data points to be partitioned.
  • groupsize: maximum amount of observations in each group.

Returns

A vector of overlapping UnitRange{Int} values. Adjacent ranges share one index so the last prediction from group k can be compared to the first point in group k + 1.

Examples

julia> group_ranges(10, 5)
3-element Vector{UnitRange{Int64}}:
 1:5
 5:9
 9:10
source