Jacobian-Vector Products (JVP)

Functions for computing Jacobian-vector products efficiently without forming the full Jacobian matrix.

Mathematical Background

The JVP computes J(x) * v where J(x) is the Jacobian of function f at point x and v is a direction vector. This is computed using finite difference approximations:

Forward: J(x) * v ≈ (f(x + h*v) - f(x)) / h
Central: J(x) * v ≈ (f(x + h*v) - f(x - h*v)) / (2h)

where h is the step size and v is the direction vector.

Performance Benefits

JVP functions are particularly efficient when you only need directional derivatives:

Function evaluations: Only 2 function evaluations (vs O(n) for full Jacobian)
Forward differences: 2 function evaluations, O(h) accuracy
Central differences: 2 function evaluations, O(h²) accuracy
Memory efficient: No need to store the full Jacobian matrix

Use Cases

JVP is particularly useful for:

Optimization: Computing directional derivatives along search directions
Sparse directions: When v has few non-zero entries
Memory constraints: Avoiding storage of large Jacobian matrices
Newton methods: Computing Newton steps J⁻¹ * v iteratively

Limitations

Complex step: JVP does not currently support complex step differentiation (Val(:complex))
In-place functions: For in-place function evaluation, ensure proper cache sizing

Functions

FiniteDiff.finite_difference_jvp — Function

FiniteDiff.finite_difference_jvp(
    f,
    x      :: AbstractArray{<:Number},
    v      :: AbstractArray{<:Number},
    fdtype :: Type{T1}                = Val{:forward},
    f_in                              = nothing;
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep)

Compute the Jacobian-vector product J(x) * v using finite differences.

This function computes the directional derivative of f at x in direction v without explicitly forming the Jacobian matrix. This is more efficient than computing the full Jacobian when only J*v is needed.

Arguments

f: Function to differentiate (vector→vector map)
x::AbstractArray{<:Number}: Point at which to evaluate the Jacobian
v::AbstractArray{<:Number}: Direction vector for the product
fdtype::Type{T1}=Val{:forward}: Finite difference method (:forward, :central)
f_in=nothing: Pre-computed f(x) value (if available)

Keyword Arguments

relstep: Relative step size (default: method-dependent optimal value)
absstep=relstep: Absolute step size fallback

Returns

Vector J(x) * v representing the Jacobian-vector product

Examples

f(x) = [x[1]^2 + x[2], x[1] * x[2], x[2]^3]
x = [1.0, 2.0]
v = [1.0, 0.0]  # Direction vector
jvp = finite_difference_jvp(f, x, v)  # Directional derivative

Mathematical Background

The JVP is computed using the finite difference approximation:

Forward: J(x) * v ≈ (f(x + h*v) - f(x)) / h
Central: J(x) * v ≈ (f(x + h*v) - f(x - h*v)) / (2h)

where h is the step size and v is the direction vector.

Notes

Requires only 2 function evaluations (vs O(n) for full Jacobian)
Forward differences: 2 function evaluations, O(h) accuracy
Central differences: 2 function evaluations, O(h²) accuracy
Particularly efficient when v is sparse or when only one directional derivative is needed

source

FiniteDiff.finite_difference_jvp(
    f,
    x,
    v,
    cache::JVPCache;
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep)

Cached.

source

FiniteDiff.finite_difference_jvp! — Function

finite_difference_jvp!(
    jvp        :: AbstractArray{<:Number},
    f,
    x          :: AbstractArray{<:Number},
    v          :: AbstractArray{<:Number},
    fdtype     :: Type{T1}          = Val{:forward},
    returntype :: Type{T2}          = eltype(x),
    f_in       :: Union{T2,Nothing} = nothing;
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep)

Cache-less.

source

FiniteDiff.finite_difference_jvp!(
    jvp   :: AbstractArray{<:Number},
    f,
    x     :: AbstractArray{<:Number},
    v     :: AbstractArray{<:Number},
    cache :: JVPCache;
    relstep = default_relstep(fdtype, eltype(x)),
    absstep = relstep,
    dir     = true)

Cached.

source

Cache

FiniteDiff.JVPCache — Type

JVPCache{X1, FX1, FDType}

Cache structure for Jacobian-vector product (JVP) computations.

Stores temporary arrays needed for efficient JVP computation without repeated allocations. The JVP computes J(x) * v where J(x) is the Jacobian of function f at point x and v is a vector.

Fields

x1::X1: Temporary array for perturbed input values
fx1::FX1: Temporary array for function evaluations

source