GitHub

Surrogate

Surrogates.jl models are callable objects that fit observations and evaluate an approximation at new points. The abstract type hierarchy comes from SurrogatesBase.jl; the package provides deterministic and extension-backed stochastic implementations.

Generic interface

An implementation of AbstractSurrogate must provide the following operations:

  1. surrogate(point): evaluate the fitted approximation at one point. The point representation must be the same representation accepted by the constructor and should be documented by the implementation.
  2. update!(surrogate, x_new, y_new): incorporate one observation or a batch of observations and update the fitted state in place. A batch must contain one response for each new point. Implementations may additionally accept algorithm-specific keyword arguments, such as gradient observations.
  3. surrogate.x and surrogate.y: retain the training points and responses when the surrogate is used with surrogate_optimize! or potential_optimal_points. The optimization routines use these fields to identify the current best observation and to avoid duplicate candidate points.

Stochastic surrogates additionally implement std_error_at_point when they expose predictive uncertainty and logpdf_surrogate when they expose a log density or marginal likelihood. Implementations should document the response shape and units returned by these methods.

The interface is deliberately expressed in terms of the generic call syntax and update!; optimization code should not depend on a concrete surrogate type. A minimal implementation is:

using SurrogatesBase

mutable struct MySurrogate{T} <: SurrogatesBase.AbstractDeterministicSurrogate
    x::Vector{T}
    y::Vector{T}
end

(s::MySurrogate)(point) = point^2

function SurrogatesBase.update!(s::MySurrogate, x_new, y_new)
    push!(s.x, x_new)
    push!(s.y, y_new)
    return nothing
end

The generic contract is tested with a local implementation in the test suite; concrete surrogate tests then cover each model's fitting and numerical behavior.

Surrogates.AbstractSurrogate — Type
AbstractSurrogate

Union alias for deterministic and stochastic surrogate models accepted by the generic Surrogates.jl interfaces.

Concrete surrogates are expected to satisfy the SurrogatesBase interface:

  • surrogate(x) evaluates the fitted approximation at x.
  • update!(surrogate, x_new, y_new) incorporates one or more new observations.
  • sample storage is available through surrogate.x and surrogate.y for the optimization routines in this package.

This alias is a convenience type for method signatures. New surrogate implementations should subtype SurrogatesBase.AbstractDeterministicSurrogate or SurrogatesBase.AbstractStochasticSurrogate, not this union alias.

source
Surrogates.current_surrogates — Constant
current_surrogates

Names of surrogate model families currently listed by Surrogates.jl.

This vector is informational. Use the exported constructor names, such as RadialBasis, Kriging, or Wendland, to build surrogates programmatically.

Returns

A mutable vector of strings. The list is intended for display and discovery; its contents are not a dispatch contract.

Example

using Surrogates

filter(contains("Kriging"), current_surrogates)
source
Surrogates.std_error_at_point — Function
std_error_at_point(surrogate, point)

Return the predictive standard error of surrogate at point.

Surrogate implementations that expose uncertainty should add a method to this generic. The method must accept every point representation supported by the surrogate's call overload, must not mutate the surrogate, and must return a nonnegative scalar in the same response units as surrogate(point).

Arguments

  • surrogate: fitted surrogate with a predictive uncertainty model.
  • point: scalar or multidimensional query point accepted by surrogate.

Returns

A nonnegative scalar predictive standard error. Implementations should throw an ArgumentError for a point with incompatible dimensionality.

Example

using Surrogates

x = [0.0, 0.5, 1.0]
y = sin.(x)
surrogate = Kriging(x, y, 0.0, 1.0)
std_error_at_point(surrogate, 0.25)
source
  • Linear surrogate
Surrogates.LinearSurrogate — Type
LinearSurrogate(x, y, lb, ub)

Fit an affine least-squares surrogate to sampled inputs x and responses y. The fitted model is coeff[1] + coeff[2] * x[1] + … + coeff[end] * x[end], i.e. an intercept plus one slope per input dimension. The returned object is callable at new points and implements the SurrogatesBase deterministic-surrogate interface, including update!.

Fields

  • x: sampled scalar points or multidimensional points.
  • y: responses corresponding to x.
  • coeff: fitted coefficients, laid out as [intercept; slopes]. For vector-valued responses this is a matrix with one column per output.
  • lb: lower bound of the modeled domain.
  • ub: upper bound of the modeled domain.

Arguments

  • x: training inputs. Use a vector of numbers for one dimension or a vector of equal-length point containers for multiple dimensions.
  • y: training responses, with one response per element of x. Responses may be scalars or equal-length vectors; vector-valued responses are fitted one output per column and returned in the same container.
  • lb: scalar or vector lower domain bound.
  • ub: scalar or vector upper domain bound matching lb.

Returns

A callable LinearSurrogate. Calling surrogate(point) evaluates the fitted affine model, while update!(surrogate, x_new, y_new) appends observations and refits its coefficients.

Example

using Surrogates

x = [0.0, 1.0, 2.0]
y = 2 .* x .+ 5
surrogate = LinearSurrogate(x, y, 0.0, 2.0)
surrogate(1.5)
source
  • Radial basis function surrogate
Surrogates.RadialBasis — Type
RadialBasis(x, y, lb, ub; rad = linearRadial(), scale_factor = 0.5,
            sparse = false, regularization = 0.0)

Fit a radial-basis interpolant, optionally augmented by the polynomial term required by the selected RadialFunction. The result is callable at new points and implements the SurrogatesBase deterministic-surrogate interface.

Fields

  • phi: radial basis function applied to scaled distances.
  • dim_poly: degree of the accompanying polynomial basis.
  • x: sampled scalar points or multidimensional points.
  • y: scalar or vector responses corresponding to x.
  • lb: lower bound of the modeled domain.
  • ub: upper bound of the modeled domain.
  • coeff: fitted interpolation coefficients.
  • scale_factor: divisor applied to distances before evaluating phi.
  • sparse: whether coefficient construction uses a sparse matrix.
  • regularization: diagonal regularization added to the interpolation matrix.

Arguments

  • x: training inputs.
  • y: training responses, with one response per input.
  • lb: scalar or vector lower domain bound.
  • ub: scalar or vector upper domain bound matching lb.

Keywords

  • rad::RadialFunction = linearRadial(): radial basis descriptor. See also cubicRadial, multiquadricRadial, and thinplateRadial.
  • scale_factor::Real = 0.5: distance scale used by the radial function.
  • sparse::Bool = false: use sparse coefficient construction.
  • regularization::Real = 0.0: diagonal stabilization term.

Returns

A callable RadialBasis supporting update!(surrogate, x_new, y_new).

Example

using Surrogates

x = [0.0, 1.0, 2.0]
y = x .^ 2
surrogate = RadialBasis(x, y, 0.0, 2.0; rad = cubicRadial())
surrogate(1.5)
source
  • Kriging surrogate
Surrogates.Kriging — Type
Kriging(x, y, lb::Number, ub::Number; p = 2.0,
        theta = 0.5 / max(1.0e-6 * abs(ub - lb), std(x))^p)
Kriging(x, y, lb, ub;
        p = 2.0 .* collect(one.(x[1])),
        theta = [0.5 / max(1.0e-6 * norm(ub .- lb),
                           std(x_i[i] for x_i in x))^p[i]
                 for i in eachindex(x[1])])

Fit a Kriging interpolant with a power-exponential correlation model. The surrogate is callable for mean predictions, while std_error_at_point evaluates predictive uncertainty.

Based on: Jones, Schonlau and Welch (1998), "Efficient Global Optimization of Expensive Black-Box Functions", J Glob Optim 13:455-492; and Jones (2001), "A Taxonomy of Global Optimization Methods Based on Response Surfaces", J Glob Optim 21:345-383.

Fields

  • x: sampled scalar points or multidimensional points.
  • y: scalar responses corresponding to x.
  • lb: lower bound of the modeled domain.
  • ub: upper bound of the modeled domain.
  • p: correlation smoothness exponent: a scalar in one dimension, one entry per input coordinate otherwise.
  • theta: correlation scale, shaped like p.
  • mu: estimated constant process mean.
  • b: BLUP weights R⁻¹(y - 𝟙μ).
  • sigma: estimated process variance.
  • R_fact: Cholesky factorization of the nugget-regularized correlation matrix. The deprecated property inverse_of_R still materializes R⁻¹ from it.

Arguments

  • x: training inputs with no repeated points.
  • y: scalar training responses, with one response per input.
  • lb: scalar or vector lower domain bound.
  • ub: scalar or vector upper domain bound matching lb.

Keywords

  • p: correlation smoothness in the half-open interval (0, 2]. The default is 2.0 in one dimension and a vector filled with 2.0 otherwise. Zero is excluded: it makes every off-diagonal correlation exp(-θ), so the correlation matrix is singular for more than two samples. Only p = 2 gives a mean-square differentiable process; smaller values give rougher sample paths.
  • theta: positive correlation scale. When left unset it is fitted by maximum likelihood, starting from a value derived from the sample spread and the domain width as shown above; update! then refits it as samples are added. An explicitly supplied theta is a modelling choice: it is used as given and preserved across update!.
  • optimize_theta: whether to fit theta by maximizing the concentrated log-likelihood -n/2 log σ̂²(θ) - 1/2 log|R(θ)|, as in Jones (2001) §2 and DACE. Defaults to true exactly when theta is not supplied. Fitting costs a Nelder-Mead search whose every step factorizes an n × n matrix, so set it to false for a cheap fit on a large design.
  • n_start: Latin-hypercube starts for that search, in addition to the data-derived one. Ignored when optimize_theta is false.
  • maxiters: Nelder-Mead iteration cap per start.

Returns

A callable Kriging supporting update!(surrogate, x_new, y_new) and std_error_at_point. Duplicate points make the correlation matrix singular: the constructors reject them with an ArgumentError, while update! warns and leaves the surrogate unchanged, since the observation is already present and optimizers routinely re-propose points near convergence.

Example

using Surrogates

x = [0.0, 0.5, 1.0]
y = sin.(x)
surrogate = Kriging(x, y, 0.0, 1.0)
surrogate(0.25)
std_error_at_point(surrogate, 0.25)
source
  • Lobachevsky surrogate
Surrogates.LobachevskySurrogate — Type
LobachevskySurrogate(x, y, lb, ub; alpha = 1.0, n = 4, sparse = false)

Construct a univariate or multivariate Lobachevsky-spline interpolant. The kernel order n must be even; multidimensional models use one alpha scale per input dimension.

Fields

  • x: sampled scalar points or multidimensional points.
  • y: responses corresponding to x.
  • alpha: scalar or per-dimension kernel scale.
  • n: even Lobachevsky kernel order.
  • lb: lower bound of the modeled domain.
  • ub: upper bound of the modeled domain.
  • coeff: fitted interpolation coefficients, one per sample for scalar responses and an n x m matrix with one column per output for vector-valued ones.
  • sparse: whether coefficient construction uses a sparse matrix.

Arguments

  • x: training inputs.
  • y: training responses, with one response per input. Responses may be scalars or equal-length vectors; vector-valued responses are fitted one output at a time and the surrogate then returns a vector.
  • lb: scalar or vector lower domain bound.
  • ub: scalar or vector upper domain bound matching lb.

Keywords

  • alpha = 1.0: kernel scale, which must lie in (0, 4]. A scale of zero makes every kernel value identical and the interpolation system singular. For multidimensional inputs, supply one scale per input dimension; the default is a vector of ones matching one training point.
  • n::Int = 4: even, positive kernel order, at most 20 (factorial(Int64(n)) overflows Int64 beyond that).
  • sparse::Bool = false: use sparse coefficient construction.

Returns

A callable LobachevskySurrogate supporting update!(surrogate, x_new, y_new) and the closed-form integration helpers lobachevsky_integral and lobachevsky_integrate_dimension.

Example

using Surrogates

x = [0.0, 1.0, 2.0]
y = sin.(x)
surrogate = LobachevskySurrogate(x, y, 0.0, 2.0; alpha = 1.0, n = 4)
surrogate(0.5)
source
  • Support vector machine surrogate, requires using LIBSVM.
SVMSurrogate(x,y,lb::Number,ub::Number)
  • Gradient boosted trees surrogate, requires using XGBoost.
XGBoostSurrogate(x,y,lb,ub;num_round::Int = 1)
  • Neural network surrogate, requires using Flux.
NeuralSurrogate(x,y,lb,ub; model = Chain(Dense(length(x[1]),1), first), loss = (x,y) -> Flux.mse(model(x), y),opt = Descent(0.01),n_echos::Int = 1)
Surrogates.SVMSurrogate — Type
SVMSurrogate(x, y, lb, ub)

Support-vector-machine surrogate backed by LIBSVM.jl.

This type is available when the LIBSVM extension is loaded. The fitted model is callable through the generic surrogate interface and can be updated by refitting after adding samples.

Fields

  • x: training inputs.
  • y: training responses.
  • model: fitted LIBSVM model.
  • lb: lower bound of the input domain.
  • ub: upper bound of the input domain.

Arguments

  • x: sample locations.
  • y: observed values at x.
  • lb: lower bound of the input domain.
  • ub: upper bound of the input domain.

Returns

An SVMSurrogate satisfying the generic surrogate interface.

source

Structure Descriptors

Surrogates.RadialBasisStructure — Function
RadialBasisStructure(; radial_function, scale_factor, sparse)

Create a named-tuple configuration for a RadialBasis surrogate.

Keywords

  • radial_function: radial basis function object, for example linearRadial() or cubicRadial().
  • scale_factor: scale factor passed to the RadialBasis constructor.
  • sparse: whether to use the sparse interpolation matrix path.

Returns

A named tuple with fields name, radial_function, scale_factor, and sparse. Composite constructors such as VariableFidelitySurrogate consume this value to build the requested surrogate internally.

source
Surrogates.KrigingStructure — Function
KrigingStructure(; p, theta)

Create a named-tuple configuration for a Kriging surrogate.

Keywords

  • p: Kriging correlation exponent.
  • theta: Kriging correlation scale parameter.

Returns

A named tuple with fields name, p, and theta.

source
Surrogates.GEKStructure — Function
GEKStructure(; p, theta)

Create a named-tuple configuration for a GEK surrogate.

Keywords

  • p: correlation exponent. GEK requires 2.
  • theta: Kriging correlation scale parameter.

Returns

A named tuple with fields name, p, and theta.

source
Surrogates.LobachevskyStructure — Function
LobachevskyStructure(; alpha, n, sparse)

Create a named-tuple configuration for a LobachevskySurrogate.

Keywords

  • alpha: Lobachevsky basis scale parameter.
  • n::Int: Lobachevsky basis order.
  • sparse: whether to use the sparse coefficient path.

Returns

A named tuple with fields name, alpha, n, and sparse.

source
Surrogates.NeuralStructure — Function
NeuralStructure(; model, loss, opt, n_epochs)

Create a named-tuple configuration for a NeuralSurrogate.

Keywords

  • model: Flux model used by the neural surrogate.
  • loss: training loss.
  • opt: optimizer state or optimizer object accepted by the extension.
  • n_epochs: number of training epochs.

Returns

A named tuple with fields name, model, loss, opt, and n_epochs.

source
Surrogates.GENNStructure — Function
GENNStructure(; model, opt, n_epochs, gamma)

Create a named-tuple configuration for a GENNSurrogate.

Keywords

  • model: Flux model used by the gradient-enhanced neural surrogate.
  • opt: optimizer state or optimizer object accepted by the extension.
  • n_epochs: number of training epochs.
  • gamma: weight applied to derivative information during training.

Returns

A named tuple with fields name, model, opt, n_epochs, and gamma.

source
Surrogates.WendlandStructure — Function
WendlandStructure(; eps, maxiters, tol)

Create a named-tuple configuration for a Wendland surrogate.

Keywords

  • eps: reciprocal of the kernel support radius.
  • maxiters::Integer: maximum number of conjugate-gradient iterations.
  • tol: relative tolerance for the coefficient solve.

Returns

A named tuple with fields name, eps, maxiters, and tol.

source

Creating another surrogate

It's great that you want to add another surrogate to the library! You will need to:

  1. Define a new mutable struct and a constructor function
  2. Define update!(your_surrogate, x_new, y_new)
  3. Define your_surrogate(value) for the approximation

Example

mutable struct NewSurrogate{X, Y, L, U, C, A, B} <: AbstractDeterministicSurrogate
    x::X
    y::Y
    lb::L
    ub::U
    coeff::C
    alpha::A
    beta::B
end

function NewSurrogate(x, y, lb, ub, parameters)
    ...
    return NewSurrogate(x, y, lb, ub, calculated \ _coeff, alpha, beta)
end

function update!(NewSurrogate, x_new, y_new)
    ...
end

function (s::NewSurrogate)(value)
    return s.coeff * value + s.alpha
end