Surrogate
Surrogates.jl models are callable objects that fit observations and evaluate an approximation at new points. The abstract type hierarchy comes from SurrogatesBase.jl; the package provides deterministic and extension-backed stochastic implementations.
Generic interface
An implementation of AbstractSurrogate must provide the following operations:
surrogate(point): evaluate the fitted approximation at one point. The point representation must be the same representation accepted by the constructor and should be documented by the implementation.update!(surrogate, x_new, y_new): incorporate one observation or a batch of observations and update the fitted state in place. A batch must contain one response for each new point. Implementations may additionally accept algorithm-specific keyword arguments, such as gradient observations.surrogate.xandsurrogate.y: retain the training points and responses when the surrogate is used withsurrogate_optimize!orpotential_optimal_points. The optimization routines use these fields to identify the current best observation and to avoid duplicate candidate points.
Stochastic surrogates additionally implement std_error_at_point when they expose predictive uncertainty and logpdf_surrogate when they expose a log density or marginal likelihood. Implementations should document the response shape and units returned by these methods.
Kriging, GEK, KPLS, KPLSK, GEKPLS and AbstractGPSurrogate answer std_error_at_point. Only those can be used with the acquisition functions that score candidates by predictive uncertainty, EI() and LCBS(); given a surrogate without one, both raise an ArgumentError naming the alternatives rather than failing deeper in the search.
The interface is deliberately expressed in terms of the generic call syntax and update!; optimization code should not depend on a concrete surrogate type. A minimal implementation is:
using SurrogatesBase
mutable struct MySurrogate{T} <: SurrogatesBase.AbstractDeterministicSurrogate
x::Vector{T}
y::Vector{T}
end
(s::MySurrogate)(point) = point^2
function SurrogatesBase.update!(s::MySurrogate, x_new, y_new)
push!(s.x, x_new)
push!(s.y, y_new)
return nothing
endThe generic contract is tested with a local implementation in the test suite; concrete surrogate tests then cover each model's fitting and numerical behavior.
Surrogates.AbstractSurrogate — Type
AbstractSurrogateUnion alias for deterministic and stochastic surrogate models accepted by the generic Surrogates.jl interfaces.
Concrete surrogates are expected to satisfy the SurrogatesBase interface:
surrogate(x)evaluates the fitted approximation atx.update!(surrogate, x_new, y_new)incorporates one or more new observations.- sample storage is available through
surrogate.xandsurrogate.yfor the optimization routines in this package.
This alias is a convenience type for method signatures. New surrogate implementations should subtype SurrogatesBase.AbstractDeterministicSurrogate or SurrogatesBase.AbstractStochasticSurrogate, not this union alias.
Surrogates.current_surrogates — Constant
current_surrogatesNames of surrogate model families currently listed by Surrogates.jl.
This vector is informational. Use the exported constructor names, such as RadialBasis, Kriging, or Wendland, to build surrogates programmatically.
Returns
A mutable vector of strings. The list is intended for display and discovery; its contents are not a dispatch contract.
Example
using Surrogates
filter(contains("Kriging"), current_surrogates)Surrogates.std_error_at_point — Function
std_error_at_point(surrogate, point)Return the predictive standard error of surrogate at point.
Surrogate implementations that expose uncertainty should add a method to this generic. The method must accept every point representation supported by the surrogate's call overload, must not mutate the surrogate, and must return a nonnegative scalar in the same response units as surrogate(point).
Arguments
surrogate: fitted surrogate with a predictive uncertainty model.point: scalar or multidimensional query point accepted bysurrogate.
Returns
A nonnegative scalar predictive standard error. Implementations should throw an ArgumentError for a point with incompatible dimensionality.
Example
using Surrogates
x = [0.0, 0.5, 1.0]
y = sin.(x)
surrogate = Kriging(x, y, 0.0, 1.0)
std_error_at_point(surrogate, 0.25)SurrogatesBase.parameters — Function
parameters(surrogate)Return learned parameters or fitted state exposed by surrogate.
Arguments
surrogate: a concrete surrogate implementation that documents this optional method.
Returns
An implementation-defined representation of learned parameters or fitted state. The returned object may be a scalar, tuple, named tuple, array, or model object.
Interface Contract
parameters is optional. Extend it only for concrete surrogate subtypes that expose learned state; calling it for a subtype without a method raises a MethodError.
Examples
julia> struct ParameterExampleSurrogate <: AbstractDeterministicSurrogate
weight::Float64
end
julia> (surrogate::ParameterExampleSurrogate)(x) = surrogate.weight * x;
julia> SurrogatesBase.parameters(surrogate::ParameterExampleSurrogate) = (; weight = surrogate.weight);
julia> parameters(ParameterExampleSurrogate(2.0))
(weight = 2.0,)SurrogatesBase.hyperparameters — Function
hyperparameters(surrogate)Return the tunable fitting configuration exposed by surrogate.
Arguments
surrogate: a concrete surrogate implementation that documents this optional method.
Returns
An implementation-defined representation of tunable configuration, commonly a named tuple or a small immutable configuration object.
Interface Contract
hyperparameters is optional. When both this method and update_hyperparameters! are implemented, the returned configuration must describe the setting used for subsequent evaluations.
Examples
julia> struct HyperparameterReadExample <: AbstractDeterministicSurrogate
scale::Float64
end
julia> (surrogate::HyperparameterReadExample)(x) = surrogate.scale * x;
julia> SurrogatesBase.hyperparameters(surrogate::HyperparameterReadExample) =
(; scale = surrogate.scale);
julia> hyperparameters(HyperparameterReadExample(2.0))
(scale = 2.0,)SurrogatesBase.update_hyperparameters! — Function
update_hyperparameters!(surrogate, prior)Update the tunable fitting configuration of surrogate in place.
Arguments
surrogate: a concrete surrogate implementation with tunable hyperparameters.prior: implementation-defined prior, bounds, or optimization configuration.
Interface Contract
This optional method must leave surrogate consistent with its updated hyperparameters. If those hyperparameters affect the fitted representation, the implementation must refit or invalidate that representation before subsequent evaluation. The return value is not part of the interface.
Examples
julia> mutable struct HyperparameterExample <: AbstractDeterministicSurrogate
scale::Float64
end
julia> (surrogate::HyperparameterExample)(x) = surrogate.scale * x;
julia> function SurrogatesBase.update_hyperparameters!(surrogate::HyperparameterExample, prior)
surrogate.scale = prior.scale
return nothing
end;
julia> surrogate = HyperparameterExample(2.0);
julia> update_hyperparameters!(surrogate, (; scale = 4.0)); surrogate(0.5)
2.0- Linear surrogate
Surrogates.LinearSurrogate — Type
LinearSurrogate(x, y, lb, ub)Fit an affine least-squares surrogate to sampled inputs x and responses y. The fitted model is coeff[1] + coeff[2] * x[1] + … + coeff[end] * x[end], i.e. an intercept plus one slope per input dimension. The returned object is callable at new points and implements the SurrogatesBase deterministic-surrogate interface, including update!.
Fields
x: sampled scalar points or multidimensional points.y: responses corresponding tox.coeff: fitted coefficients, laid out as[intercept; slopes]. For vector-valued responses this is a matrix with one column per output.lb: lower bound of the modeled domain.ub: upper bound of the modeled domain.
Arguments
x: training inputs. Use a vector of numbers for one dimension or a vector of equal-length point containers for multiple dimensions.y: training responses, with one response per element ofx. Responses may be scalars or equal-length vectors; vector-valued responses are fitted one output per column and returned in the same container.lb: scalar or vector lower domain bound.ub: scalar or vector upper domain bound matchinglb.
Returns
A callable LinearSurrogate. Calling surrogate(point) evaluates the fitted affine model, while update!(surrogate, x_new, y_new) appends observations and refits its coefficients.
Example
using Surrogates
x = [0.0, 1.0, 2.0]
y = 2 .* x .+ 5
surrogate = LinearSurrogate(x, y, 0.0, 2.0)
surrogate(1.5)- Radial basis function surrogate
Surrogates.RadialBasis — Type
RadialBasis(x, y, lb, ub; rad = linearRadial(), scale_factor = 0.5,
sparse = false, regularization = 0.0)Fit a radial-basis interpolant, optionally augmented by the polynomial term required by the selected RadialFunction. The result is callable at new points and implements the SurrogatesBase deterministic-surrogate interface.
Fields
phi: radial basis function applied to scaled distances.dim_poly: degree the kernel's conditional positive definiteness would ask of a polynomial tail.x: sampled scalar points or multidimensional points.y: scalar or vector responses corresponding tox.lb: lower bound of the modeled domain.ub: upper bound of the modeled domain.coeff: fitted interpolation coefficients.scale_factor: divisor applied to distances before evaluatingphi.sparse: whether coefficient construction uses a sparse matrix.regularization: diagonal regularization added to the interpolation matrix.
Arguments
x: training inputs.y: training responses, with one response per input.lb: scalar or vector lower domain bound.ub: scalar or vector upper domain bound matchinglb.
Keywords
rad::RadialFunction = linearRadial(): radial basis descriptor. See alsocubicRadial,multiquadricRadial, andthinplateRadial.scale_factor::Real = 0.5: distance scale used by the radial function.sparse::Bool = false: use sparse coefficient construction.regularization::Real = 0.0: diagonal stabilization term.
Returns
A callable RadialBasis supporting update!(surrogate, x_new, y_new).
Example
using Surrogates
x = [0.0, 1.0, 2.0]
y = x .^ 2
surrogate = RadialBasis(x, y, 0.0, 2.0; rad = cubicRadial())
surrogate(1.5)- Kriging surrogate
Surrogates.Kriging — Type
Kriging(x, y, lb, ub; p = 2, theta = nothing, optimize_theta = theta === nothing,
n_start = 4, maxiters = 250)Fit a Kriging interpolant with a power-exponential correlation model. The surrogate is callable for mean predictions, while std_error_at_point evaluates predictive uncertainty.
Based on: Jones, Schonlau and Welch (1998), "Efficient Global Optimization of Expensive Black-Box Functions", J Glob Optim 13:455-492; and Jones (2001), "A Taxonomy of Global Optimization Methods Based on Response Surfaces", J Glob Optim 21:345-383.
Fields
x: sampled scalar points or multidimensional points.y: scalar responses corresponding tox.lb: lower bound of the modeled domain.ub: upper bound of the modeled domain.p: correlation smoothness exponent: a scalar in one dimension, one entry per input coordinate otherwise.theta: correlation scale, shaped likep.mu: estimated constant process mean.b: BLUP weightsR⁻¹(y - 𝟙μ).sigma: estimated process variance.R_fact: Cholesky factorization of the nugget-regularized correlation matrix. The deprecated propertyinverse_of_Rstill materializesR⁻¹from it.
Arguments
x: training inputs with no repeated points.y: scalar training responses, with one response per input.lb: scalar or vector lower domain bound.ub: scalar or vector upper domain bound matchinglb.
Keywords
p: correlation smoothness in the half-open interval(0, 2]. The default is2.0in one dimension and a vector filled with2.0otherwise. Zero is excluded: it makes every off-diagonal correlationexp(-θ), so the correlation matrix is singular for more than two samples. Onlyp = 2gives a mean-square differentiable process; smaller values give rougher sample paths.theta: positive correlation scale. When left unset it is fitted by maximum likelihood, starting from a value derived from the sample spread and the domain width as shown above;update!then refits it as samples are added. An explicitly suppliedthetais a modelling choice: it is used as given and preserved acrossupdate!.optimize_theta: whether to fitthetaby maximizing the concentrated log-likelihood-n/2 log σ̂²(θ) - 1/2 log|R(θ)|, as in Jones (2001) §2 and DACE. Defaults totrueexactly whenthetais not supplied. Fitting costs a Nelder-Mead search whose every step factorizes ann × nmatrix, so set it tofalsefor a cheap fit on a large design.n_start: Latin-hypercube starts for that search, in addition to the data-derived one. Ignored whenoptimize_thetaisfalse.maxiters: Nelder-Mead iteration cap per start.
Returns
A callable Kriging supporting update!(surrogate, x_new, y_new) and std_error_at_point. Duplicate points make the correlation matrix singular: the constructors reject them with an ArgumentError, while update! warns and leaves the surrogate unchanged, since the observation is already present and optimizers routinely re-propose points near convergence.
Example
using Surrogates
x = [0.0, 0.5, 1.0]
y = sin.(x)
surrogate = Kriging(x, y, 0.0, 1.0)
surrogate(0.25)
std_error_at_point(surrogate, 0.25)- Lobachevsky surrogate
Surrogates.LobachevskySurrogate — Type
LobachevskySurrogate(x, y, lb, ub; alpha = 1.0, n = 4, sparse = false)Construct a univariate or multivariate Lobachevsky-spline interpolant. The kernel order n must be even; multidimensional models use one alpha scale per input dimension.
Fields
x: sampled scalar points or multidimensional points.y: responses corresponding tox.alpha: scalar or per-dimension kernel scale.n: even Lobachevsky kernel order.lb: lower bound of the modeled domain.ub: upper bound of the modeled domain.coeff: fitted interpolation coefficients, one per sample for scalar responses and ann x mmatrix with one column per output for vector-valued ones.sparse: whether coefficient construction uses a sparse matrix.
Arguments
x: training inputs.y: training responses, with one response per input. Responses may be scalars or equal-length vectors; vector-valued responses are fitted one output at a time and the surrogate then returns a vector.lb: scalar or vector lower domain bound.ub: scalar or vector upper domain bound matchinglb.
Keywords
alpha = 1.0: kernel scale, which must lie in(0, 4]. A scale of zero makes every kernel value identical and the interpolation system singular. For multidimensional inputs, supply one scale per input dimension; the default is a vector of ones matching one training point.n::Int = 4: even, positive kernel order, at most 20 (factorial(Int64(n))overflowsInt64beyond that).sparse::Bool = false: use sparse coefficient construction.
Returns
A callable LobachevskySurrogate supporting update!(surrogate, x_new, y_new) and the closed-form integration helpers lobachevsky_integral and lobachevsky_integrate_dimension.
Example
using Surrogates
x = [0.0, 1.0, 2.0]
y = sin.(x)
surrogate = LobachevskySurrogate(x, y, 0.0, 2.0; alpha = 1.0, n = 4)
surrogate(0.5)Surrogates.lobachevsky_integral — Method
lobachevsky_integral(loba::LobachevskySurrogate,lb,ub)
Calculates the integral of the Lobachevsky surrogate, which has a closed form.
- Support vector machine surrogate, requires
using LIBSVM. - Gradient boosted trees surrogate, requires
using XGBoost; documented in the XGBoost Surrogate Tutorial. - Neural network surrogate, requires
using Flux; documented in the Neural Network Surrogate Tutorial.
Surrogates.SVMSurrogate — Type
SVMSurrogate(x, y, lb, ub)Support-vector-machine surrogate backed by LIBSVM.jl.
This type is available when the LIBSVM extension is loaded. The fitted model is callable through the generic surrogate interface and can be updated by refitting after adding samples.
Fields
x: training inputs.y: training responses.model: fitted LIBSVM model.lb: lower bound of the input domain.ub: upper bound of the input domain.
Arguments
x: sample locations.y: observed values atx.lb: lower bound of the input domain.ub: upper bound of the input domain.
Returns
An SVMSurrogate satisfying the generic surrogate interface.
Structure Descriptors
Surrogates.RadialBasisStructure — Function
RadialBasisStructure(; radial_function, scale_factor, sparse)Create a named-tuple configuration for a RadialBasis surrogate.
Keywords
radial_function: radial basis function object, for examplelinearRadial()orcubicRadial().scale_factor: scale factor passed to theRadialBasisconstructor.sparse: whether to use the sparse interpolation matrix path.
Returns
A named tuple with fields name, radial_function, scale_factor, and sparse. Composite constructors such as VariableFidelitySurrogate consume this value to build the requested surrogate internally.
Surrogates.KrigingStructure — Function
KrigingStructure(; p, theta)Create a named-tuple configuration for a Kriging surrogate.
Keywords
p: Kriging correlation exponent.theta: Kriging correlation scale parameter.
Returns
A named tuple with fields name, p, and theta.
Surrogates.GEKStructure — Function
GEKStructure(; p, theta)Create a named-tuple configuration for a GEK surrogate.
Keywords
p: correlation exponent.GEKrequires2.theta: Kriging correlation scale parameter.
Returns
A named tuple with fields name, p, and theta.
Surrogates.LinearStructure — Function
LinearStructure()Create a named-tuple configuration for a LinearSurrogate.
Returns
A named tuple with the field name = "LinearSurrogate".
Surrogates.InverseDistanceStructure — Function
InverseDistanceStructure(; p)Create a named-tuple configuration for an InverseDistanceSurrogate.
Keywords
p: inverse-distance power parameter.
Returns
A named tuple with fields name and p.
Surrogates.LobachevskyStructure — Function
LobachevskyStructure(; alpha, n, sparse)Create a named-tuple configuration for a LobachevskySurrogate.
Keywords
alpha: Lobachevsky basis scale parameter.n::Int: Lobachevsky basis order.sparse: whether to use the sparse coefficient path.
Returns
A named tuple with fields name, alpha, n, and sparse.
Surrogates.NeuralStructure — Function
NeuralStructure(; model, loss, opt, n_epochs)Create a named-tuple configuration for a NeuralSurrogate.
Keywords
model: Flux model used by the neural surrogate.loss: training loss.opt: optimizer state or optimizer object accepted by the extension.n_epochs: number of training epochs.
Returns
A named tuple with fields name, model, loss, opt, and n_epochs.
Surrogates.GENNStructure — Function
GENNStructure(; model, opt, n_epochs, gamma)Create a named-tuple configuration for a GENNSurrogate.
Keywords
model: Flux model used by the gradient-enhanced neural surrogate.opt: optimizer state or optimizer object accepted by the extension.n_epochs: number of training epochs.gamma: weight applied to derivative information during training.
Returns
A named tuple with fields name, model, opt, n_epochs, and gamma.
Unlike the other *Structure helpers, this one cannot be used as a component of MOE or VariableFidelitySurrogate. GENNSurrogate is gradient-enhanced: it requires a per-sample gradient array alongside x and y, and a composite surrogate carries only function values, so there is nothing to supply. Both composites reject the descriptor with a message saying so. The helper is retained for symmetry and for direct use in code that does have gradients to hand.
Surrogates.XGBoostStructure — Function
XGBoostStructure(; num_round)Create a named-tuple configuration for an XGBoostSurrogate.
Keywords
num_round::Integer: number of boosting rounds.
Returns
A named tuple with fields name and num_round.
Surrogates.SecondOrderPolynomialStructure — Function
SecondOrderPolynomialStructure()Create a named-tuple configuration for a SecondOrderPolynomialSurrogate.
Returns
A named tuple with the field name = "SecondOrderPolynomialSurrogate".
Surrogates.WendlandStructure — Function
WendlandStructure(; eps, maxiters, tol)Create a named-tuple configuration for a Wendland surrogate.
Keywords
eps: reciprocal of the kernel support radius.maxiters::Integer: maximum number of conjugate-gradient iterations.tol: relative tolerance for the coefficient solve.
Returns
A named tuple with fields name, eps, maxiters, and tol.
Surrogates.PolyChaosStructure — Function
PolyChaosStructure(; op)Create a named-tuple configuration for a PolynomialChaosSurrogate.
Keywords
op: orthogonal-polynomial basis object from PolyChaos.jl.
Returns
A named tuple with fields name and op.
Creating another surrogate
It's great that you want to add another surrogate to the library! You will need to:
- Define a new mutable struct and a constructor function
- Define update!(your_surrogate, x_new, y_new)
- Define your_surrogate(value) for the approximation
Example
mutable struct NewSurrogate{X, Y, L, U, C, A, B} <: AbstractDeterministicSurrogate
x::X
y::Y
lb::L
ub::U
coeff::C
alpha::A
beta::B
end
function NewSurrogate(x, y, lb, ub, parameters)
...
return NewSurrogate(x, y, lb, ub, calculated \ _coeff, alpha, beta)
end
function update!(NewSurrogate, x_new, y_new)
...
end
function (s::NewSurrogate)(value)
return s.coeff * value + s.alpha
end