Surrogate

Surrogates.jl models are callable objects that fit observations and evaluate an approximation at new points. The abstract type hierarchy comes from SurrogatesBase.jl; the package provides deterministic and extension-backed stochastic implementations.

Generic interface

An implementation of AbstractSurrogate must provide the following operations:

  1. surrogate(point): evaluate the fitted approximation at one point. The point representation must be the same representation accepted by the constructor and should be documented by the implementation.
  2. update!(surrogate, x_new, y_new): incorporate one observation or a batch of observations and update the fitted state in place. A batch must contain one response for each new point. Implementations may additionally accept algorithm-specific keyword arguments, such as gradient observations.
  3. surrogate.x and surrogate.y: retain the training points and responses when the surrogate is used with surrogate_optimize! or potential_optimal_points. The optimization routines use these fields to identify the current best observation and to avoid duplicate candidate points.

Stochastic surrogates additionally implement std_error_at_point when they expose predictive uncertainty and logpdf_surrogate when they expose a log density or marginal likelihood. Implementations should document the response shape and units returned by these methods.

Kriging, GEK, KPLS, KPLSK, GEKPLS and AbstractGPSurrogate answer std_error_at_point. Only those can be used with the acquisition functions that score candidates by predictive uncertainty, EI() and LCBS(); given a surrogate without one, both raise an ArgumentError naming the alternatives rather than failing deeper in the search.

The interface is deliberately expressed in terms of the generic call syntax and update!; optimization code should not depend on a concrete surrogate type. A minimal implementation is:

using SurrogatesBase

mutable struct MySurrogate{T} <: SurrogatesBase.AbstractDeterministicSurrogate
    x::Vector{T}
    y::Vector{T}
end

(s::MySurrogate)(point) = point^2

function SurrogatesBase.update!(s::MySurrogate, x_new, y_new)
    push!(s.x, x_new)
    push!(s.y, y_new)
    return nothing
end

The generic contract is tested with a local implementation in the test suite; concrete surrogate tests then cover each model's fitting and numerical behavior.

Surrogates.AbstractSurrogate — Type
AbstractSurrogate

Union alias for deterministic and stochastic surrogate models accepted by the generic Surrogates.jl interfaces.

Concrete surrogates are expected to satisfy the SurrogatesBase interface:

  • surrogate(x) evaluates the fitted approximation at x.
  • update!(surrogate, x_new, y_new) incorporates one or more new observations.
  • sample storage is available through surrogate.x and surrogate.y for the optimization routines in this package.

This alias is a convenience type for method signatures. New surrogate implementations should subtype SurrogatesBase.AbstractDeterministicSurrogate or SurrogatesBase.AbstractStochasticSurrogate, not this union alias.

source
Surrogates.current_surrogates — Constant
current_surrogates

Names of surrogate model families currently listed by Surrogates.jl.

This vector is informational. Use the exported constructor names, such as RadialBasis, Kriging, or Wendland, to build surrogates programmatically.

Returns

A mutable vector of strings. The list is intended for display and discovery; its contents are not a dispatch contract.

Example

using Surrogates

filter(contains("Kriging"), current_surrogates)
source
Surrogates.std_error_at_point — Function
std_error_at_point(surrogate, point)

Return the predictive standard error of surrogate at point.

Surrogate implementations that expose uncertainty should add a method to this generic. The method must accept every point representation supported by the surrogate's call overload, must not mutate the surrogate, and must return a nonnegative scalar in the same response units as surrogate(point).

Arguments

  • surrogate: fitted surrogate with a predictive uncertainty model.
  • point: scalar or multidimensional query point accepted by surrogate.

Returns

A nonnegative scalar predictive standard error. Implementations should throw an ArgumentError for a point with incompatible dimensionality.

Example

using Surrogates

x = [0.0, 0.5, 1.0]
y = sin.(x)
surrogate = Kriging(x, y, 0.0, 1.0)
std_error_at_point(surrogate, 0.25)
source
SurrogatesBase.parameters — Function
parameters(surrogate)

Return learned parameters or fitted state exposed by surrogate.

Arguments

  • surrogate: a concrete surrogate implementation that documents this optional method.

Returns

An implementation-defined representation of learned parameters or fitted state. The returned object may be a scalar, tuple, named tuple, array, or model object.

Interface Contract

parameters is optional. Extend it only for concrete surrogate subtypes that expose learned state; calling it for a subtype without a method raises a MethodError.

Examples

julia> struct ParameterExampleSurrogate <: AbstractDeterministicSurrogate
           weight::Float64
       end

julia> (surrogate::ParameterExampleSurrogate)(x) = surrogate.weight * x;

julia> SurrogatesBase.parameters(surrogate::ParameterExampleSurrogate) = (; weight = surrogate.weight);

julia> parameters(ParameterExampleSurrogate(2.0))
(weight = 2.0,)
source
SurrogatesBase.hyperparameters — Function
hyperparameters(surrogate)

Return the tunable fitting configuration exposed by surrogate.

Arguments

  • surrogate: a concrete surrogate implementation that documents this optional method.

Returns

An implementation-defined representation of tunable configuration, commonly a named tuple or a small immutable configuration object.

Interface Contract

hyperparameters is optional. When both this method and update_hyperparameters! are implemented, the returned configuration must describe the setting used for subsequent evaluations.

Examples

julia> struct HyperparameterReadExample <: AbstractDeterministicSurrogate
           scale::Float64
       end

julia> (surrogate::HyperparameterReadExample)(x) = surrogate.scale * x;

julia> SurrogatesBase.hyperparameters(surrogate::HyperparameterReadExample) =
           (; scale = surrogate.scale);

julia> hyperparameters(HyperparameterReadExample(2.0))
(scale = 2.0,)
source
SurrogatesBase.update_hyperparameters! — Function
update_hyperparameters!(surrogate, prior)

Update the tunable fitting configuration of surrogate in place.

Arguments

  • surrogate: a concrete surrogate implementation with tunable hyperparameters.
  • prior: implementation-defined prior, bounds, or optimization configuration.

Interface Contract

This optional method must leave surrogate consistent with its updated hyperparameters. If those hyperparameters affect the fitted representation, the implementation must refit or invalidate that representation before subsequent evaluation. The return value is not part of the interface.

Examples

julia> mutable struct HyperparameterExample <: AbstractDeterministicSurrogate
           scale::Float64
       end

julia> (surrogate::HyperparameterExample)(x) = surrogate.scale * x;

julia> function SurrogatesBase.update_hyperparameters!(surrogate::HyperparameterExample, prior)
           surrogate.scale = prior.scale
           return nothing
       end;

julia> surrogate = HyperparameterExample(2.0);

julia> update_hyperparameters!(surrogate, (; scale = 4.0)); surrogate(0.5)
2.0
source
  • Linear surrogate
Surrogates.LinearSurrogate — Type
LinearSurrogate(x, y, lb, ub)

Fit an affine least-squares surrogate to sampled inputs x and responses y. The fitted model is coeff[1] + coeff[2] * x[1] + … + coeff[end] * x[end], i.e. an intercept plus one slope per input dimension. The returned object is callable at new points and implements the SurrogatesBase deterministic-surrogate interface, including update!.

Fields

  • x: sampled scalar points or multidimensional points.
  • y: responses corresponding to x.
  • coeff: fitted coefficients, laid out as [intercept; slopes]. For vector-valued responses this is a matrix with one column per output.
  • lb: lower bound of the modeled domain.
  • ub: upper bound of the modeled domain.

Arguments

  • x: training inputs. Use a vector of numbers for one dimension or a vector of equal-length point containers for multiple dimensions.
  • y: training responses, with one response per element of x. Responses may be scalars or equal-length vectors; vector-valued responses are fitted one output per column and returned in the same container.
  • lb: scalar or vector lower domain bound.
  • ub: scalar or vector upper domain bound matching lb.

Returns

A callable LinearSurrogate. Calling surrogate(point) evaluates the fitted affine model, while update!(surrogate, x_new, y_new) appends observations and refits its coefficients.

Example

using Surrogates

x = [0.0, 1.0, 2.0]
y = 2 .* x .+ 5
surrogate = LinearSurrogate(x, y, 0.0, 2.0)
surrogate(1.5)
source
  • Radial basis function surrogate
Surrogates.RadialBasis — Type
RadialBasis(x, y, lb, ub; rad = linearRadial(), scale_factor = 0.5,
            sparse = false, regularization = 0.0)

Fit a radial-basis interpolant, optionally augmented by the polynomial term required by the selected RadialFunction. The result is callable at new points and implements the SurrogatesBase deterministic-surrogate interface.

Fields

  • phi: radial basis function applied to scaled distances.

  • dim_poly: degree the kernel's conditional positive definiteness would ask of a polynomial tail.

    Warning

    No polynomial tail is fitted. _calc_coeffs solves the kernel system alone, so this field records the kernel's requirement and does not describe the model. hyperparameters reports it for the same reason.

  • x: sampled scalar points or multidimensional points.

  • y: scalar or vector responses corresponding to x.

  • lb: lower bound of the modeled domain.

  • ub: upper bound of the modeled domain.

  • coeff: fitted interpolation coefficients.

  • scale_factor: divisor applied to distances before evaluating phi.

  • sparse: whether coefficient construction uses a sparse matrix.

  • regularization: diagonal regularization added to the interpolation matrix.

Arguments

  • x: training inputs.
  • y: training responses, with one response per input.
  • lb: scalar or vector lower domain bound.
  • ub: scalar or vector upper domain bound matching lb.

Keywords

  • rad::RadialFunction = linearRadial(): radial basis descriptor. See also cubicRadial, multiquadricRadial, and thinplateRadial.
  • scale_factor::Real = 0.5: distance scale used by the radial function.
  • sparse::Bool = false: use sparse coefficient construction.
  • regularization::Real = 0.0: diagonal stabilization term.

Returns

A callable RadialBasis supporting update!(surrogate, x_new, y_new).

Example

using Surrogates

x = [0.0, 1.0, 2.0]
y = x .^ 2
surrogate = RadialBasis(x, y, 0.0, 2.0; rad = cubicRadial())
surrogate(1.5)
source
  • Kriging surrogate
Surrogates.Kriging — Type
Kriging(x, y, lb, ub; p = 2, theta = nothing, optimize_theta = theta === nothing,
        n_start = 4, maxiters = 250)

Fit a Kriging interpolant with a power-exponential correlation model. The surrogate is callable for mean predictions, while std_error_at_point evaluates predictive uncertainty.

Based on: Jones, Schonlau and Welch (1998), "Efficient Global Optimization of Expensive Black-Box Functions", J Glob Optim 13:455-492; and Jones (2001), "A Taxonomy of Global Optimization Methods Based on Response Surfaces", J Glob Optim 21:345-383.

Fields

  • x: sampled scalar points or multidimensional points.
  • y: scalar responses corresponding to x.
  • lb: lower bound of the modeled domain.
  • ub: upper bound of the modeled domain.
  • p: correlation smoothness exponent: a scalar in one dimension, one entry per input coordinate otherwise.
  • theta: correlation scale, shaped like p.
  • mu: estimated constant process mean.
  • b: BLUP weights R⁻¹(y - 𝟙μ).
  • sigma: estimated process variance.
  • R_fact: Cholesky factorization of the nugget-regularized correlation matrix. The deprecated property inverse_of_R still materializes R⁻¹ from it.

Arguments

  • x: training inputs with no repeated points.
  • y: scalar training responses, with one response per input.
  • lb: scalar or vector lower domain bound.
  • ub: scalar or vector upper domain bound matching lb.

Keywords

  • p: correlation smoothness in the half-open interval (0, 2]. The default is 2.0 in one dimension and a vector filled with 2.0 otherwise. Zero is excluded: it makes every off-diagonal correlation exp(-θ), so the correlation matrix is singular for more than two samples. Only p = 2 gives a mean-square differentiable process; smaller values give rougher sample paths.
  • theta: positive correlation scale. When left unset it is fitted by maximum likelihood, starting from a value derived from the sample spread and the domain width as shown above; update! then refits it as samples are added. An explicitly supplied theta is a modelling choice: it is used as given and preserved across update!.
  • optimize_theta: whether to fit theta by maximizing the concentrated log-likelihood -n/2 log σ̂²(θ) - 1/2 log|R(θ)|, as in Jones (2001) §2 and DACE. Defaults to true exactly when theta is not supplied. Fitting costs a Nelder-Mead search whose every step factorizes an n × n matrix, so set it to false for a cheap fit on a large design.
  • n_start: Latin-hypercube starts for that search, in addition to the data-derived one. Ignored when optimize_theta is false.
  • maxiters: Nelder-Mead iteration cap per start.

Returns

A callable Kriging supporting update!(surrogate, x_new, y_new) and std_error_at_point. Duplicate points make the correlation matrix singular: the constructors reject them with an ArgumentError, while update! warns and leaves the surrogate unchanged, since the observation is already present and optimizers routinely re-propose points near convergence.

Example

using Surrogates

x = [0.0, 0.5, 1.0]
y = sin.(x)
surrogate = Kriging(x, y, 0.0, 1.0)
surrogate(0.25)
std_error_at_point(surrogate, 0.25)
source
  • Lobachevsky surrogate
Surrogates.LobachevskySurrogate — Type
LobachevskySurrogate(x, y, lb, ub; alpha = 1.0, n = 4, sparse = false)

Construct a univariate or multivariate Lobachevsky-spline interpolant. The kernel order n must be even; multidimensional models use one alpha scale per input dimension.

Fields

  • x: sampled scalar points or multidimensional points.
  • y: responses corresponding to x.
  • alpha: scalar or per-dimension kernel scale.
  • n: even Lobachevsky kernel order.
  • lb: lower bound of the modeled domain.
  • ub: upper bound of the modeled domain.
  • coeff: fitted interpolation coefficients, one per sample for scalar responses and an n x m matrix with one column per output for vector-valued ones.
  • sparse: whether coefficient construction uses a sparse matrix.

Arguments

  • x: training inputs.
  • y: training responses, with one response per input. Responses may be scalars or equal-length vectors; vector-valued responses are fitted one output at a time and the surrogate then returns a vector.
  • lb: scalar or vector lower domain bound.
  • ub: scalar or vector upper domain bound matching lb.

Keywords

  • alpha = 1.0: kernel scale, which must lie in (0, 4]. A scale of zero makes every kernel value identical and the interpolation system singular. For multidimensional inputs, supply one scale per input dimension; the default is a vector of ones matching one training point.
  • n::Int = 4: even, positive kernel order, at most 20 (factorial(Int64(n)) overflows Int64 beyond that).
  • sparse::Bool = false: use sparse coefficient construction.

Returns

A callable LobachevskySurrogate supporting update!(surrogate, x_new, y_new) and the closed-form integration helpers lobachevsky_integral and lobachevsky_integrate_dimension.

Example

using Surrogates

x = [0.0, 1.0, 2.0]
y = sin.(x)
surrogate = LobachevskySurrogate(x, y, 0.0, 2.0; alpha = 1.0, n = 4)
surrogate(0.5)
source
Surrogates.SVMSurrogate — Type
SVMSurrogate(x, y, lb, ub)

Support-vector-machine surrogate backed by LIBSVM.jl.

This type is available when the LIBSVM extension is loaded. The fitted model is callable through the generic surrogate interface and can be updated by refitting after adding samples.

Fields

  • x: training inputs.
  • y: training responses.
  • model: fitted LIBSVM model.
  • lb: lower bound of the input domain.
  • ub: upper bound of the input domain.

Arguments

  • x: sample locations.
  • y: observed values at x.
  • lb: lower bound of the input domain.
  • ub: upper bound of the input domain.

Returns

An SVMSurrogate satisfying the generic surrogate interface.

source

Structure Descriptors

Surrogates.RadialBasisStructure — Function
RadialBasisStructure(; radial_function, scale_factor, sparse)

Create a named-tuple configuration for a RadialBasis surrogate.

Keywords

  • radial_function: radial basis function object, for example linearRadial() or cubicRadial().
  • scale_factor: scale factor passed to the RadialBasis constructor.
  • sparse: whether to use the sparse interpolation matrix path.

Returns

A named tuple with fields name, radial_function, scale_factor, and sparse. Composite constructors such as VariableFidelitySurrogate consume this value to build the requested surrogate internally.

source
Surrogates.KrigingStructure — Function
KrigingStructure(; p, theta)

Create a named-tuple configuration for a Kriging surrogate.

Keywords

  • p: Kriging correlation exponent.
  • theta: Kriging correlation scale parameter.

Returns

A named tuple with fields name, p, and theta.

source
Surrogates.GEKStructure — Function
GEKStructure(; p, theta)

Create a named-tuple configuration for a GEK surrogate.

Keywords

  • p: correlation exponent. GEK requires 2.
  • theta: Kriging correlation scale parameter.

Returns

A named tuple with fields name, p, and theta.

source
Surrogates.LobachevskyStructure — Function
LobachevskyStructure(; alpha, n, sparse)

Create a named-tuple configuration for a LobachevskySurrogate.

Keywords

  • alpha: Lobachevsky basis scale parameter.
  • n::Int: Lobachevsky basis order.
  • sparse: whether to use the sparse coefficient path.

Returns

A named tuple with fields name, alpha, n, and sparse.

source
Surrogates.NeuralStructure — Function
NeuralStructure(; model, loss, opt, n_epochs)

Create a named-tuple configuration for a NeuralSurrogate.

Keywords

  • model: Flux model used by the neural surrogate.
  • loss: training loss.
  • opt: optimizer state or optimizer object accepted by the extension.
  • n_epochs: number of training epochs.

Returns

A named tuple with fields name, model, loss, opt, and n_epochs.

source
Surrogates.GENNStructure — Function
GENNStructure(; model, opt, n_epochs, gamma)

Create a named-tuple configuration for a GENNSurrogate.

Keywords

  • model: Flux model used by the gradient-enhanced neural surrogate.
  • opt: optimizer state or optimizer object accepted by the extension.
  • n_epochs: number of training epochs.
  • gamma: weight applied to derivative information during training.

Returns

A named tuple with fields name, model, opt, n_epochs, and gamma.

Note

Unlike the other *Structure helpers, this one cannot be used as a component of MOE or VariableFidelitySurrogate. GENNSurrogate is gradient-enhanced: it requires a per-sample gradient array alongside x and y, and a composite surrogate carries only function values, so there is nothing to supply. Both composites reject the descriptor with a message saying so. The helper is retained for symmetry and for direct use in code that does have gradients to hand.

source
Surrogates.WendlandStructure — Function
WendlandStructure(; eps, maxiters, tol)

Create a named-tuple configuration for a Wendland surrogate.

Keywords

  • eps: reciprocal of the kernel support radius.
  • maxiters::Integer: maximum number of conjugate-gradient iterations.
  • tol: relative tolerance for the coefficient solve.

Returns

A named tuple with fields name, eps, maxiters, and tol.

source

Creating another surrogate

It's great that you want to add another surrogate to the library! You will need to:

  1. Define a new mutable struct and a constructor function
  2. Define update!(your_surrogate, x_new, y_new)
  3. Define your_surrogate(value) for the approximation

Example

mutable struct NewSurrogate{X, Y, L, U, C, A, B} <: AbstractDeterministicSurrogate
    x::X
    y::Y
    lb::L
    ub::U
    coeff::C
    alpha::A
    beta::B
end

function NewSurrogate(x, y, lb, ub, parameters)
    ...
    return NewSurrogate(x, y, lb, ub, calculated \ _coeff, alpha, beta)
end

function update!(NewSurrogate, x_new, y_new)
    ...
end

function (s::NewSurrogate)(value)
    return s.coeff * value + s.alpha
end