Reservoir Computing using Cellular Automata

We showcase how to use reservoir computing models with cellular automata (ReCA) with ReservoirComputing.jl. While introduced in (Yilmaz, 2014) (Margem and Yilmaz, 2017), the implementation in this package follows (Nichele and Molund, 2017). To showcase ReCA models we show how to solve the 5 bit memory task.

5 bit memory task

We read the data can be read as follows:

using DelimitedFiles

input = readdlm("./5bitinput.txt", ',', Float64)
output = readdlm("./5bitoutput.txt", ',', Float64)

4×6720 Matrix{Float64}:
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  …  0.0  0.0  1.0  1.0  1.0  1.0  1.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0     0.0  0.0  0.0  0.0  0.0  0.0  0.0
 1.0  1.0  1.0  1.0  1.0  1.0  1.0  1.0     1.0  1.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0     0.0  0.0  0.0  0.0  0.0  0.0  0.0

To use a ReCA model, it is necessary to define the rule one intends to use. To do so, ReservoirComputing.jl leverages CellularAutomata.jl that needs to be called as well to define the RECA struct:

using ReservoirComputing, CellularAutomata, Random
Random.seed!(42)
rng = MersenneTwister(17)

ca = DCA(90)

CellularAutomata.DCA{Int64, Vector{Int64}, Int64}(90, [0, 1, 0, 1, 1, 0, 1, 0], 2, 1)

To define the ReCA model, it suffices to call:

reca = RECA(4, 4, DCA(90);
           generations=16,
           input_encoding=RandomMapping(16, 40))
ps, st = setup(rng, reca)

((reservoir = NamedTuple(), states_modifiers = (), readout = (weight = Float32[0.8549528 0.87197685 … 0.4896065 0.5931492; 0.7217107 0.43086243 … 0.7985357 0.8520769; 0.05982375 0.9083108 … 0.30078495 0.66918766; 0.99557173 0.7772248 … 0.31089818 0.5717627],)), (reservoir = (cell = (ca = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0  …  0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],), carry = nothing), states_modifiers = (), readout = NamedTuple()))

After this, the training can be performed with the chosen method.

ps, st = train!(reca, input, output, ps, st, StandardRidge(0.00001))

((reservoir = NamedTuple(), states_modifiers = (), readout = (weight = [-0.00321409649132883 0.0034360327191860908 … 0.0036390144413238823 0.003630186767441112; 0.0021210265405745303 -0.000774710645856225 … -0.004063046436346977 -0.003998166019450989; 0.0008684704711368244 0.0001724196629308859 … 0.0007402028733666944 0.0005895332854325549; -0.0 -0.0 … 0.0 -0.0],)), (reservoir = (cell = (ca = Float32[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0  …  0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],), carry = ([1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0  …  0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 1.0, 0.0],)), states_modifiers = (), readout = NamedTuple()))

We are going to test the recall ability of the model, feeding the input data and investigating whether the predicted output equals the output data.

st0 = resetcarry!(rng, reca, st) #reset the first ca state
pred_out, st = predict(reca, input, ps, st0)
final_pred = convert(AbstractArray{Float32}, pred_out .> 0.5)
final_pred == output

true