GitHub

Training Strategies

Training strategies are the choices for how the points are sampled for the definition of the physics-informed loss.

Recommendations

QuasiRandomTraining with its default LatinHyperCubeSample() is a well-rounded training strategy which can be used for most situations. It scales well for high dimensional spaces and is GPU-compatible. QuadratureTraining can lead to faster or more robust convergence with one of the H-Cubature or P-Cubature methods, but are not currently GPU compatible. For very high dimensional cases, QuadratureTraining with an adaptive Monte Carlo quadrature method, such as CubaVegas, can be beneficial for difficult or stiff problems.

GridTraining should only be used for testing purposes and should not be relied upon for real training cases. StochasticTraining achieves a lower convergence rate in the quasi-Monte Carlo methods and thus QuasiRandomTraining should be preferred in most cases. WeightedIntervalTraining can only be used with ODEs (NNODE).

API

NeuralPDE.GridTraining — Type
GridTraining(dx)

A training strategy that uses the grid points in a multidimensional grid with spacings dx. If the grid is multidimensional, then dx is expected to be an array of dx values matching the dimension of the domain, corresponding to the grid spacing in each dimension.

Positional Arguments

  • dx: the discretization of the grid.
source
NeuralPDE.StochasticTraining — Type
StochasticTraining(points; bcs_points = points)

Positional Arguments

  • points: number of points in random select training set

Keyword Arguments

  • bcs_points: number of points in random select training set for boundary conditions (by default, it equals points).
source
NeuralPDE.QuasiRandomTraining — Type
QuasiRandomTraining(points; bcs_points = points,
                            sampling_alg = LatinHypercubeSample(), resampling = true,
                            minibatch = 0)

A training strategy which uses quasi-Monte Carlo sampling for low discrepancy sequences that accelerate the convergence in high dimensional spaces over pure random sequences.

Positional Arguments

  • points: the number of quasi-random points in a sample

Keyword Arguments

  • bcs_points: the number of quasi-random points in a sample for boundary conditions (by default, it equals points),
  • sampling_alg: the quasi-Monte Carlo sampling algorithm,
  • resampling: if it's false - the full training set is generated in advance before training, and at each iteration, one subset is randomly selected out of the batch. If it's true - the training set isn't generated beforehand, and one set of quasi-random points is generated directly at each iteration in runtime. In this case, minibatch has no effect.
  • minibatch: the number of subsets, if !resampling.

For more information, see QuasiMonteCarlo.jl.

source
NeuralPDE.QuadratureTraining — Type
QuadratureTraining(; quadrature_alg = CubatureJLh(), reltol = 1e-6, abstol = 1e-3,
                    maxiters = 1_000, batch = 100)

A training strategy which treats the loss function as the integral of ||condition|| over the domain. Uses an Integrals.jl algorithm for computing the (adaptive) quadrature of this loss with respect to the chosen tolerances, with a batching batch corresponding to the maximum number of points to evaluate in a given integrand call.

Keyword Arguments

  • quadrature_alg: quadrature algorithm,
  • reltol: relative tolerance,
  • abstol: absolute tolerance,
  • maxiters: the maximum number of iterations in quadrature algorithm,
  • batch: the preferred number of points to batch.

For more information on the argument values and algorithm choices, see Integrals.jl.

source
NeuralPDE.WeightedIntervalTraining — Type
WeightedIntervalTraining(weights, samples)

A training strategy that generates points for training based on the given inputs. We split the timespan into equal segments based on the number of weights, then sample points in each segment based on that segments corresponding weight, such that the total number of sampled points is equivalent to the given samples

Positional Arguments

  • weights: A vector of weights that should sum to 1, representing the proportion of samples at each interval.
  • points: the total number of samples that we want, across the entire time span

Limitations

This training strategy can only be used with ODEs (NNODE).

source