Hessians

Functions for computing Hessian matrices of scalar-valued functions.

Function Requirements

Hessian functions are designed for scalar-valued functions f(x) where:

x is a vector of parameters
f(x) returns a scalar value
The Hessian H[i,j] = ∂²f/(∂x[i]∂x[j]) is automatically symmetrized

Mathematical Background

For a scalar function f: ℝⁿ → ℝ, the Hessian central difference approximation is:

H[i,j] ≈ (f(x + eᵢhᵢ + eⱼhⱼ) - f(x + eᵢhᵢ - eⱼhⱼ) - f(x - eᵢhᵢ + eⱼhⱼ) + f(x - eᵢhᵢ - eⱼhⱼ)) / (4hᵢhⱼ)

where eᵢ is the i-th unit vector and hᵢ is the step size in dimension i.

Performance Considerations

Complexity: Requires O(n²) function evaluations for an n-dimensional input
Accuracy: Central differences provide O(h²) accuracy for second derivatives
Memory: The result is returned as a Symmetric matrix view
Alternative: For large problems, consider computing the gradient twice instead

StaticArrays Support

The cache constructor automatically detects StaticArray types and adjusts the inplace parameter accordingly for optimal performance.

Functions

FiniteDiff.finite_difference_hessian — Function

finite_difference_hessian(
    f,
    x       :: AbstractArray{<:Number},
    fdtype  :: Type{T1}      = Val{:hcentral},
    inplace :: Type{Val{T2}} = x isa StaticArray ? Val{true} : Val{false};
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep)

Compute the Hessian matrix of scalar function f at point x using finite differences.

This is the cache-less version that allocates temporary arrays internally. The Hessian H[i,j] = ∂²f/(∂x[i]∂x[j]) is computed using the specified finite difference method, typically central differences for optimal accuracy.

Arguments

f: Scalar-valued function to differentiate (vector→scalar map)
x::AbstractArray{<:Number}: Point at which to evaluate the Hessian
fdtype::Type{T1}=Val{:hcentral}: Finite difference method (:hcentral for Hessian central differences)
inplace::Type{Val{T2}}: Whether to use in-place operations (auto-detected for StaticArrays)

Keyword Arguments

relstep: Relative step size (default: method-dependent optimal value)
absstep=relstep: Absolute step size fallback

Returns

Hessian matrix H where H[i,j] = ∂²f/(∂x[i]∂x[j])
Returns Symmetric(H) view to enforce mathematical symmetry

Examples

f(x) = x[1]^2 + x[1]*x[2] + x[2]^2  # Quadratic function
x = [1.0, 2.0]
H = finite_difference_hessian(f, x)  # 2×2 Symmetric matrix

Mathematical Background

For a scalar function f: ℝⁿ → ℝ, the Hessian central difference approximation is:

H[i,j] ≈ (f(x + eᵢhᵢ + eⱼhⱼ) - f(x + eᵢhᵢ - eⱼhⱼ) - f(x - eᵢhᵢ + eⱼhⱼ) + f(x - eᵢhᵢ - eⱼhⱼ)) / (4hᵢhⱼ)

where eᵢ is the i-th unit vector and hᵢ is the step size in dimension i.

Notes

Requires O(n²) function evaluations for an n-dimensional input
Central differences provide O(h²) accuracy for second derivatives
The result is automatically symmetrized using Symmetric() wrapper
For large problems, consider using finite_difference_gradient twice instead

source

finite_difference_hessian(
    f,
    x,
    cache::HessianCache{T,fdtype,inplace};
    relstep=default_relstep(fdtype, eltype(x)),
    absstep=relstep)

Cached.

source

FiniteDiff.finite_difference_hessian! — Function

finite_difference_hessian!(
    H          :: AbstractMatrix,
    f,
    x          :: AbstractArray{<:Number},
    fdtype     :: Type{T1} = Val{:hcentral},
    inplace    :: Type{Val{T2}} = x isa StaticArray ? Val{true} : Val{false};
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep)

Cache-less.

source

finite_difference_hessian!(
    H,
    f,
    x,
    cache::HessianCache{T,fdtype,inplace};
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep)

Cached.

source

Cache

FiniteDiff.HessianCache — Type

HessianCache(
    xpp, xpm, xmp, xmm,
    fdtype  :: Type{T1}      = Val{:hcentral},
    inplace :: Type{Val{T2}} = x isa StaticArray ? Val{true} : Val{false})

Non-allocating cache constructor.

source

HessianCache(
    x,
    fdtype  :: Type{T1}      = Val{:hcentral},
    inplace :: Type{Val{T2}} = x isa StaticArray ? Val{true} : Val{false})

Allocating cache constructor.

source