weighted_init

weighted_init([rng], [T], dims...;
    scaling=0.1, return_sparse=false)

Create and return a weighted input layer matrix. In this matrix, each of the input signals in_size connects to the reservoir nodes res_size/in_size. The nonzero entries are distributed uniformly within a range defined by scaling (Lu et al., 2017).

Please note that this initializer computes its own reservoir size! If the computed reservoir size is different than the provided one it will raise a warning.

Arguments

• rng: Random number generator. Default is Utils.default_rng() from WeightInitializers.
• T: Type of the elements in the reservoir matrix. Default is Float32.
• dims: Dimensions of the matrix. Should follow res_size x in_size.

Keyword arguments

• scaling: A scaling factor to define the range of the uniform distribution. The factor can be passed in three different ways:

  • A single number. In this case, the matrix elements will be randomly chosen from the range [-scaling, scaling]. Default option, with a the scaling value set to 0.1.

  • A tuple (lower, upper). The values define the range of the distribution. the matrix elements will be randomly created and scaled the range [lower, upper].

  • A vector of length = in_size. In this case, the columns will be scaled individually by the entries of the vector. The entries can be numbers or tuples, which will mirror the behavior described above.

  • return_sparse: flag for returning a sparse matrix. Default is false.

Examples

Standard call with scaling provided by a scalar:

julia> using Random, Test

julia> using ReservoirComputing: weighted_init

julia> res_input = weighted_init(MersenneTwister(5), Float32, 9, 3; scaling = 0.1);

julia> size(res_input) == (9, 3) && all(count(!iszero, column) == 3 for column in eachcol(res_input)) && all(abs.(res_input) .<= 0.1f0)
true

Scaling with a tuple, providing lower and upper bound of the uniform distribution from which the weights will be sampled:

julia> res_input = weighted_init(MersenneTwister(6), Float32, 9, 3; scaling = (0.1, 0.5));

julia> nonzero_weights = res_input[res_input .!= 0]; all(0.1f0 .<= nonzero_weights .<= 0.5f0)
true

Scaling with a vector of scalars, where each provides the upper bound and its negative provides the lower bound. Each column is scaled in order: first element provides bounds for the first column, and so on:

julia> res_input = weighted_init(MersenneTwister(7), Float32, 9, 3; scaling = [0.1, 0.5, 0.9]);

julia> all(all(abs.(column) .<= limit) for (column, limit) in zip(eachcol(res_input), (0.1f0, 0.5f0, 0.9f0)))
true

Scaling with a vector of tuples, each providing both upper and lower bound. Each column is scaled in order: first element provides bounds for the first column, and so on:

julia> res_input = weighted_init(MersenneTwister(8), Float32, 9, 3;
       scaling = [(0.1, 0.2), (-0.2, -0.1), (0.3, 0.5)]);

julia> all(all(iszero(weight) || lower <= weight <= upper for weight in column) for (column, (lower, upper)) in zip(eachcol(res_input), ((0.1f0, 0.2f0), (-0.2f0, -0.1f0), (0.3f0, 0.5f0))))
true

Example of matrix size change:

julia> res_input = @test_logs (:warn, r"Reservoir size has changed") weighted_init(MersenneTwister(9), Float32, 8, 3);

julia> size(res_input) == (6, 3) && all(count(!iszero, column) == 2 for column in eachcol(res_input))
true

Return sparse:

julia> using SparseArrays

julia> res_input = weighted_init(MersenneTwister(10), Float32, 9, 3; return_sparse = true);

julia> issparse(res_input) && size(res_input) == (9, 3) && nnz(res_input) == 9
true