GitHub

ADTypes.jl

Documentation for ADTypes.jl.

ADTypes.ADTypes — Module
ADTypes.jl

ADTypes.jl is a multi-valued logic system to choose an automatic differentiation (AD) package and specify its parameters.

source

Dense AD

Forward mode

Algorithmic differentiation:

ADTypes.AutoForwardDiff — Type
AutoForwardDiff{chunksize,T}

Struct used to select the ForwardDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoForwardDiff(; chunksize=nothing, tag=nothing)

Type parameters

  • chunksize: the preferred chunk size to evaluate several derivatives at once

Fields

  • tag::T: a custom tag to handle nested differentiation calls (usually not necessary)
source
ADTypes.AutoMooncakeForward — Type
AutoMooncakeForward

Struct used to select the Mooncake.jl backend for automatic differentiation in forward mode.

Defined by ADTypes.jl.

Info

This struct was introduced when forward mode became available in Mooncake.jl v0.4.147. It was kept separate from AutoMooncake to avoid a breaking release of ADTypes.jl. AutoMooncake remains for reverse mode only.

Constructors

AutoMooncakeForward(; config=nothing)

Fields

  • config: either nothing or an instance of Mooncake.Config – see the docstring of Mooncake.Config for more information. AutoForwardMooncake(; config=nothing) is equivalent to AutoForwardMooncake(; config=Mooncake.Config()), i.e. the default configuration.
source

Finite differences:

ADTypes.AutoFiniteDiff — Type
AutoFiniteDiff{T1,T2,T3}

Struct used to select the FiniteDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoFiniteDiff(;
    fdtype=Val(:forward), fdjtype=fdtype, fdhtype=Val(:hcentral),
    relstep=nothing, absstep=nothing, dir=true
)

Fields

  • fdtype::T1: finite difference type
  • fdjtype::T2: finite difference type for the Jacobian
  • fdhtype::T3: finite difference type for the Hessian
  • relstep: relative finite difference step size
  • absstep: absolute finite difference step size
  • dir: direction of the finite difference step
source

Taylor mode:

ADTypes.AutoGTPSA — Type
AutoGTPSA{D}

Struct used to select the GTPSA.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoGTPSA(; descriptor=nothing)

Fields

  • descriptor::D: can be either

    • a GTPSA Descriptor specifying the number of variables/parameters, parameter order, individual variable/parameter truncation orders, and maximum order. See the GTPSA.jl documentation for more details.
    • nothing to automatically use a Descriptor given the context.
source
ADTypes.AutoTaylorDiff — Type
AutoTaylorDiff{order}

Struct used to select the TaylorDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoTaylorDiff(; order = 1)

Type parameters

  • order: the order of the Taylor-mode automatic differentiation
source

Reverse mode

ADTypes.AutoMooncake — Type
AutoMooncake

Struct used to select the Mooncake.jl backend for automatic differentiation in reverse mode.

Defined by ADTypes.jl.

Info

When forward mode became available in Mooncake.jl v0.4.147, another struct called AutoMooncakeForward was introduced. It was kept separate to avoid a breaking release of ADTypes.jl. AutoMooncake remains for reverse mode only.

Constructors

AutoMooncake(; config=nothing)

Fields

  • config: either nothing or an instance of Mooncake.Config – see the docstring of Mooncake.Config for more information. AutoMooncake(; config=nothing) is equivalent to AutoMooncake(; config=Mooncake.Config()), i.e. the default configuration.
source
ADTypes.AutoReverseDiff — Type
AutoReverseDiff{compile}

Struct used to select the ReverseDiff.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoReverseDiff(; compile::Union{Val, Bool} = Val(false))

Fields

  • compile::Union{Val, Bool}: whether to allow pre-recording and reusing a tape (which speeds up the differentiation process).

    • If compile=false or compile=Val(false), a new tape must be recorded at every call to the differentiation operator.
    • If compile=true or compile=Val(true), a tape can be pre-recorded on an example input and then reused at every differentiation call.

    The boolean version of this keyword argument is taken as the type parameter.

Warning

Pre-recording a tape only captures the path taken by the differentiated function when executed on the example input. If said function has value-dependent branching behavior, reusing pre-recorded tapes can lead to incorrect results. In such situations, you should keep the default setting compile=Val(false). For more details, please refer to ReverseDiff's AbstractTape API documentation.

Info

Despite what its name may suggest, the compile setting does not prescribe whether or not the tape is compiled with ReverseDiff.compile after being recorded. This is left as a private implementation detail.

source

Forward or reverse mode

ADTypes.AutoEnzyme — Type
AutoEnzyme{M,A}

Struct used to select the Enzyme.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoEnzyme(; mode::M=nothing, function_annotation::Type{A}=Nothing)

Type parameters

  • A determines how the function f to differentiate is passed to Enzyme. It can be:

    • a subtype of EnzymeCore.Annotation (like EnzymeCore.Const or EnzymeCore.Duplicated) to enforce a given annotation
    • Nothing to simply pass f and let Enzyme choose the most appropriate annotation

Fields

  • mode::M determines the autodiff mode (forward or reverse). It can be:

    • an object subtyping EnzymeCore.Mode (like EnzymeCore.Forward or EnzymeCore.Reverse) if a specific mode is required
    • nothing to choose the best mode automatically
source

Forward, reverse, or sparse mode

ADTypes.AutoReactant — Type
AutoReactant{M<:AutoEnzyme}

Struct used to select the Reactant.jl compilation atop Enzyme for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoReactant(; mode::Union{AutoEnzyme,Nothing}=nothing)

Fields

  • mode::M specifies the parameterization of differentiation. It can be:

    • an AutoEnzyme object if a specific mode is required
    • nothing to choose the best mode automatically
source

Symbolic mode

Sparse AD

ADTypes.AutoSparse — Type
AutoSparse{D,S,C}

Wraps an ADTypes.jl object to deal with sparse Jacobians and Hessians.

Fields

Constructors

AutoSparse(
    dense_ad;
    sparsity_detector=ADTypes.NoSparsityDetector(),
    coloring_algorithm=ADTypes.NoColoringAlgorithm()
)
source
ADTypes.dense_ad — Function
dense_ad(ad::AutoSparse)::AbstractADType
dense_ad(ad::AbstractADType)::AbstractADType

Return the underlying AD package for a sparse AD choice, act as the identity on a dense AD choice.

See also

source

Sparsity detector

ADTypes.jacobian_sparsity — Function
jacobian_sparsity(f, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}
jacobian_sparsity(f!, y, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}

Use detector sd to construct a (typically sparse) matrix S describing the pattern of nonzeroes in the Jacobian of f (resp. f!) applied at x (resp. (y, x)).

source
ADTypes.hessian_sparsity — Function
hessian_sparsity(f, x, sd::AbstractSparsityDetector)::AbstractMatrix{Bool}

Use detector sd to construct a (typically sparse) matrix S describing the pattern of nonzeroes in the Hessian of f applied at x.

source

Coloring algorithm

ADTypes.column_coloring — Function
column_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}

Use algorithm ca to construct a structurally orthogonal partition of the columns of M.

The result is a coloring vector c of length size(M, 2) such that for every non-zero coefficient M[i, j], column j is the only column of its color c[j] with a non-zero coefficient in row i.

source
ADTypes.row_coloring — Function
row_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}

Use algorithm ca to construct a structurally orthogonal partition of the rows of M.

The result is a coloring vector c of length size(M, 1) such that for every non-zero coefficient M[i, j], row i is the only row of its color c[i] with a non-zero coefficient in column j.

source
ADTypes.symmetric_coloring — Function
symmetric_coloring(M::AbstractMatrix, ca::ColoringAlgorithm)::AbstractVector{<:Integer}

Use algorithm ca to construct a symmetrically structurally orthogonal partition of the columns (or rows) of the symmetric matrix M.

The result is a coloring vector c of length size(M, 1) == size(M, 2) such that for every non-zero coefficient M[i, j], at least one of the following conditions holds:

  • column j is the only column of its color c[j] with a non-zero coefficient in row i;
  • column i is the only column of its color c[i] with a non-zero coefficient in row j.
source

No automatic differentiation

Modes

Miscellaneous

ADTypes.Auto — Function
ADTypes.Auto(package::Symbol)
ADTypes.Auto(nothing)::NoAutoDiff

A shortcut that converts an AD package name into an instance of AbstractADType, with all parameters set to their default values.

Warning

This function is type-unstable by design and might lead to suboptimal performance. In most cases, you should never need it: use the individual backend types directly.

Example

import ADTypes
backend = ADTypes.Auto(:Zygote)

# output

ADTypes.AutoZygote()
source

Deprecated

ADTypes.AutoTapir — Type
AutoTapir
Danger

AutoTapir is deprecated following a package renaming, please use AutoMooncake instead.

Struct used to select the Tapir.jl backend for automatic differentiation.

Defined by ADTypes.jl.

Constructors

AutoTapir(; safe_mode=true)

Fields

  • safe_mode::Bool: whether to run additional checks to catch errors early.
source