Training Reservoir Computing Models
Training an RC model means fitting the readout. The default objective is ridge regression; other linear and SVM objectives are available through extensions.
Ridge regression
using ReservoirComputing
using LuxCore: setup
using Random
Random.seed!(42)
rng = MersenneTwister(42)
input_data = rand(Float32, 3, 100)
target_data = rand(Float32, 5, 100)
model = ESN(3, 100, 5)
ps, st = setup(rng, model)
ps, st = train(model, input_data, target_data, ps, st;
objective = RidgeRegression(),
solver = QRFactorization())((reservoir = (input_matrix = Float32[0.086408615 0.0998888 -0.013271046; -0.06170206 -0.06359978 0.0921186; … ; -0.027093245 -0.04336114 -0.04771917; 0.07098396 0.015740562 0.019439578], reservoir_matrix = Float32[0.0 0.0 … 0.0045833415 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0]), state_modifiers = (), readout = (weight = [-653.597492490338 -247.25807418310598 … 126.37102722607627 663.6413997356498; -486.3743086285802 38.02180496645455 … -77.22950170183633 -99.6384323676058; … ; 396.2246665195202 -86.8648959077094 … 26.078160022515345 -405.2525292694728; -535.9488971657303 87.67290197939872 … -94.1879512210102 -157.435352595814],)), (reservoir = (cell = (rng = Random.MersenneTwister(42, (0, 23562, 10020, 692, 22308, 882)),), carry = (Float32[0.095440336; -0.010096364; … ; -0.17061083; -0.22067434;;],)), state_modifiers = (), readout = NamedTuple()))objective chooses what to fit (here ridge). solver chooses how to solve it; omitting solver uses QRFactorization(), the QR algorithm of LinearSolve.jl, which ReservoirComputing reexports.
ps, st = train(model, input_data, target_data, ps, st;
objective = RidgeRegression())((reservoir = (input_matrix = Float32[0.086408615 0.0998888 -0.013271046; -0.06170206 -0.06359978 0.0921186; … ; -0.027093245 -0.04336114 -0.04771917; 0.07098396 0.015740562 0.019439578], reservoir_matrix = Float32[0.0 0.0 … 0.0045833415 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0]), state_modifiers = (), readout = (weight = [4480.000296004649 928.8349000207866 … -2579.313279730249 -3101.8880251303344; -23639.40643600179 -3838.8241781194524 … 11076.115526963154 13031.279997790645; … ; -8218.735898864483 -1783.603703256959 … 4191.210905979773 4693.319981326809; -799.4328445187732 -210.44631000273253 … 490.23567335584136 172.56756042242426],)), (reservoir = (cell = (rng = Random.MersenneTwister(42, (0, 23562, 10020, 692, 22308, 882)),), carry = (Float32[0.09544154; -0.010095287; … ; -0.17061086; -0.22067615;;],)), state_modifiers = (), readout = NamedTuple()))Changing the ridge solver
Other LinearSolve.jl algorithms:
using LinearSolve
ps, st = train(model, input_data, target_data, ps, st;
objective = RidgeRegression(),
solver = SVDFactorization())((reservoir = (input_matrix = Float32[0.086408615 0.0998888 -0.013271046; -0.06170206 -0.06359978 0.0921186; … ; -0.027093245 -0.04336114 -0.04771917; 0.07098396 0.015740562 0.019439578], reservoir_matrix = Float32[0.0 0.0 … 0.0045833415 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0]), state_modifiers = (), readout = (weight = [4473.8306528394405 927.9582944074056 … -2576.664820289712 -3098.785253117207; -23621.202117858928 -3836.6164810493683 … 11068.556148978572 13022.93805098243; … ; -8209.137821165012 -1782.2516372812802 … 4187.023127804454 4688.559009231197; -799.8902543512833 -210.566141127396 … 490.4831398143892 172.92740710286702],)), (reservoir = (cell = (rng = Random.MersenneTwister(42, (0, 23562, 10020, 692, 22308, 882)),), carry = (Float32[0.09544154; -0.010095247; … ; -0.17061083; -0.22067612;;],)), state_modifiers = (), readout = NamedTuple()))Legacy built-in path:
ps, st = train(model, input_data, target_data, ps, st;
objective = RidgeRegression(),
solver = QRSolver())((reservoir = (input_matrix = Float32[0.086408615 0.0998888 -0.013271046; -0.06170206 -0.06359978 0.0921186; … ; -0.027093245 -0.04336114 -0.04771917; 0.07098396 0.015740562 0.019439578], reservoir_matrix = Float32[0.0 0.0 … 0.0045833415 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0]), state_modifiers = (), readout = (weight = [4488.217692143507 930.4221019684785 … -2583.583808727147 -3107.1537824951106; -23658.395698572986 -3842.788567913841 … 11086.596373096376 13044.80041476877; … ; -8226.620955566133 -1785.202358355448 … 4195.454055414475 4698.830399248021; -798.6199502554297 -210.31949165361473 … 489.8735718016149 172.20555359551327],)), (reservoir = (cell = (rng = Random.MersenneTwister(42, (0, 23562, 10020, 692, 22308, 882)),), carry = (Float32[0.09544155; -0.010095281; … ; -0.17061086; -0.22067615;;],)), state_modifiers = (), readout = NamedTuple()))See LinearSolve's solver list.
Other linear objectives
MLJLinearModels.jl provides additional regressors (lasso, elastic net, …).
using MLJLinearModels
ps, st = train(model, input_data, target_data, ps, st;
objective = LassoRegression(fit_intercept = false),
solver = ProxGrad())((reservoir = (input_matrix = Float32[0.086408615 0.0998888 -0.013271046; -0.06170206 -0.06359978 0.0921186; … ; -0.027093245 -0.04336114 -0.04771917; 0.07098396 0.015740562 0.019439578], reservoir_matrix = Float32[0.0 0.0 … 0.0045833415 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0]), state_modifiers = (), readout = (weight = Float32[0.0 0.0 … -0.0 -0.0; 0.0 0.0 … -0.0 -0.0; … ; 0.0 0.0 … -0.0 -0.0; 0.0 0.0 … -0.0 -0.0],)), (reservoir = (cell = (rng = Random.MersenneTwister(42, (0, 23562, 10020, 692, 22308, 882)),), carry = (Float32[0.09544155; -0.010095294; … ; -0.17061086; -0.22067615;;],)), state_modifiers = (), readout = NamedTuple()))See MLJLinearModels models and solvers. Not every solver works with every model. MLJ also exports a type named RidgeRegression; write MLJLinearModels.RidgeRegression when both packages are loaded.
Extending readout fitting
Package extensions can support another training objective by defining:
ReservoirComputing.__fit_readout(
objective::MyObjective,
states::AbstractMatrix,
target_data::AbstractMatrix;
solver = nothing,
kwargs...,
)The columns of states and target_data are aligned training samples. A matrix-valued result must have size (n_outputs, n_features) and is installed in a standard linear readout. Backends that return another fitted object must also implement ReservoirComputing.addreadout! for compatible model and readout types.
The model-level train function omits the solver keyword when the user supplies solver=nothing; otherwise it forwards the solver together with any additional keywords. Extension methods should reject unsupported solvers and keywords with an ArgumentError.
__fit_readout is an internal extension interface and is not a user-facing training entry point. Its compatibility is not guaranteed before version 1.0. Users should always call the model-level train API.
Support Vector Regression
ReservoirComputing.jl also allows users to train RC models with support vector regression through LIBSVM.jl. However, the majority of builtin models in the library uses a LinearReadout by default, which can only be trained with linear regression. In order to use support vector regression, one needs to build a model with SVMReadout. Load LIBSVM.jl with import LIBSVM rather than using LIBSVM: LIBSVM exports its own predict, which makes ReservoirComputing's predict ambiguous when both packages are brought in with using.
import LIBSVM
model = ReservoirComputer(
StatefulLayer(ESNCell(3=>100)),
SVMReadout(100=>5)
)
ps, st = setup(rng, model)((reservoir = (input_matrix = Float32[-0.08695352 0.065400176 0.034358095; 0.062833786 0.07213054 -0.043287825; … ; -0.08326278 -0.018158173 -0.027971745; 0.0061006784 0.09752395 -0.03747182], reservoir_matrix = Float32[0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0]), state_modifiers = (), readout = NamedTuple()), (reservoir = (cell = (rng = Random.MersenneTwister(42, (0, 46122, 32580, 920, 44868, 762)),), carry = nothing), state_modifiers = (), readout = NamedTuple()))We can now train our new model similarly to before:
ps, st = train(model, input_data, target_data, ps, st;
objective = LIBSVM.EpsilonSVR())((reservoir = (input_matrix = Float32[-0.08695352 0.065400176 0.034358095; 0.062833786 0.07213054 -0.043287825; … ; -0.08326278 -0.018158173 -0.027971745; 0.0061006784 0.09752395 -0.03747182], reservoir_matrix = Float32[0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0]), state_modifiers = (), readout = (models = Any[LIBSVM.EpsilonSVR(LIBSVM.Kernel.RadialBasis, 0.01, 0.1, 1.0, 3, 0.0, 0.001, true, false, LIBSVM.SVM{Float64, LIBSVM.Kernel.KERNEL}(LIBSVM.EpsilonSVR, LIBSVM.Kernel.RadialBasis, nothing, 100, 100, 2, Float64[], Int32[], Float64[], Int32[], LIBSVM.SupportVectors{Vector{Float64}, Matrix{Float32}}(77, Int32[], [0.1586969494819641, 0.40045297145843506, 0.18177849054336548, 0.5965102314949036, 0.931290328502655, 0.6301283836364746, 0.8153464198112488, 0.05460089445114136, 0.1727055311203003, 0.5511121153831482 … 0.912855327129364, 0.7364910244941711, 0.6849258542060852, 0.8720908164978027, 0.9727347493171692, 0.9505835175514221, 0.8818221688270569, 0.3563631772994995, 0.07670968770980835, 0.20116615295410156], Float32[-0.5009399 -0.23965345 … -0.07096513 -0.011674796; 0.50529057 0.193262 … 0.27819282 0.32498732; … ; 0.56494606 0.22042651 … -0.061495535 -0.13789222; 0.97362113 0.16069841 … 0.036586475 -0.009694248], Int32[1, 3, 5, 6, 7, 8, 9, 10, 12, 15 … 91, 92, 93, 94, 95, 96, 97, 98, 99, 100], LIBSVM.SVMNode[LIBSVM.SVMNode(1, -0.5009399056434631), LIBSVM.SVMNode(1, -0.23965345323085785), LIBSVM.SVMNode(1, -0.002833075588569045), LIBSVM.SVMNode(1, 0.05375999957323074), LIBSVM.SVMNode(1, 0.12402049452066422), LIBSVM.SVMNode(1, -0.002047029323875904), LIBSVM.SVMNode(1, -0.17693717777729034), LIBSVM.SVMNode(1, -0.07985340058803558), LIBSVM.SVMNode(1, 0.03588742017745972), LIBSVM.SVMNode(1, -0.05035760998725891) … LIBSVM.SVMNode(1, -0.11737559735774994), LIBSVM.SVMNode(1, 0.01387971080839634), LIBSVM.SVMNode(1, 0.058396246284246445), LIBSVM.SVMNode(1, -0.01189055759459734), LIBSVM.SVMNode(1, 0.019903670996427536), LIBSVM.SVMNode(1, 0.0048777032643556595), LIBSVM.SVMNode(1, 0.004893505945801735), LIBSVM.SVMNode(1, -0.0604410320520401), LIBSVM.SVMNode(1, -0.07096513360738754), LIBSVM.SVMNode(1, -0.011674796231091022)]), 0.0, [-0.30637944450114624; -0.14420871423490464; … ; -1.0; -1.0;;], Float64[], Float64[], [-0.37185807173840424], 3, 0.01, 200.0, 0.001, 1.0, 0.5, 0.1, true, false)), LIBSVM.EpsilonSVR(LIBSVM.Kernel.RadialBasis, 0.01, 0.1, 1.0, 3, 0.0, 0.001, true, false, LIBSVM.SVM{Float64, LIBSVM.Kernel.KERNEL}(LIBSVM.EpsilonSVR, LIBSVM.Kernel.RadialBasis, nothing, 100, 100, 2, Float64[], Int32[], Float64[], Int32[], LIBSVM.SupportVectors{Vector{Float64}, Matrix{Float32}}(82, Int32[], [0.7555027008056641, 0.004748523235321045, 0.0630447268486023, 0.09847414493560791, 0.7945293188095093, 0.7741878628730774, 0.27521514892578125, 0.3039790391921997, 0.08023208379745483, 0.12636154890060425 … 0.4503591060638428, 0.2856241464614868, 0.8881773948669434, 0.8232199549674988, 0.11052101850509644, 0.288784921169281, 0.30006706714630127, 0.9062978029251099, 0.0848768949508667, 0.7946451902389526], Float32[-0.5009399 0.3044097 … 0.004893506 -0.011674796; 0.50529057 0.470386 … 0.27349594 0.32498732; … ; 0.56494606 0.21516198 … -0.10404492 -0.13789222; 0.97362113 -0.4151342 … -0.12574933 -0.009694248], Int32[1, 2, 3, 4, 5, 7, 8, 9, 10, 12 … 88, 89, 90, 91, 92, 93, 94, 95, 97, 100], LIBSVM.SVMNode[LIBSVM.SVMNode(1, -0.5009399056434631), LIBSVM.SVMNode(1, 0.3044097125530243), LIBSVM.SVMNode(1, -0.23965345323085785), LIBSVM.SVMNode(1, 0.5313342809677124), LIBSVM.SVMNode(1, -0.002833075588569045), LIBSVM.SVMNode(1, 0.12402049452066422), LIBSVM.SVMNode(1, -0.002047029323875904), LIBSVM.SVMNode(1, -0.17693717777729034), LIBSVM.SVMNode(1, -0.07985340058803558), LIBSVM.SVMNode(1, 0.03588742017745972) … LIBSVM.SVMNode(1, 0.05576758086681366), LIBSVM.SVMNode(1, -0.03471464663743973), LIBSVM.SVMNode(1, 0.06710212677717209), LIBSVM.SVMNode(1, -0.11737559735774994), LIBSVM.SVMNode(1, 0.01387971080839634), LIBSVM.SVMNode(1, 0.058396246284246445), LIBSVM.SVMNode(1, -0.01189055759459734), LIBSVM.SVMNode(1, 0.019903670996427536), LIBSVM.SVMNode(1, 0.004893505945801735), LIBSVM.SVMNode(1, -0.011674796231091022)]), 0.0, [0.09478175280342419; -1.0; … ; -1.0; 1.0;;], Float64[], Float64[], [-0.285778323379013], 3, 0.01, 200.0, 0.001, 1.0, 0.5, 0.1, true, false)), LIBSVM.EpsilonSVR(LIBSVM.Kernel.RadialBasis, 0.01, 0.1, 1.0, 3, 0.0, 0.001, true, false, LIBSVM.SVM{Float64, LIBSVM.Kernel.KERNEL}(LIBSVM.EpsilonSVR, LIBSVM.Kernel.RadialBasis, nothing, 100, 100, 2, Float64[], Int32[], Float64[], Int32[], LIBSVM.SupportVectors{Vector{Float64}, Matrix{Float32}}(83, Int32[], [0.8459857702255249, 0.26018762588500977, 0.36676234006881714, 0.7839549779891968, 0.19145554304122925, 0.6931900382041931, 0.23401367664337158, 0.7798506617546082, 0.41250449419021606, 0.12423431873321533 … 0.6778095960617065, 0.7155494093894958, 0.9813255667686462, 0.01926577091217041, 0.76598060131073, 0.2356317639350891, 0.16908544301986694, 0.00823664665222168, 0.8741745352745056, 0.3594043254852295], Float32[-0.5009399 0.3044097 … -0.07096513 -0.011674796; 0.50529057 0.470386 … 0.27819282 0.32498732; … ; 0.56494606 0.21516198 … -0.061495535 -0.13789222; 0.97362113 -0.4151342 … 0.036586475 -0.009694248], Int32[1, 2, 3, 4, 5, 6, 7, 8, 9, 10 … 89, 90, 91, 94, 95, 96, 97, 98, 99, 100], LIBSVM.SVMNode[LIBSVM.SVMNode(1, -0.5009399056434631), LIBSVM.SVMNode(1, 0.3044097125530243), LIBSVM.SVMNode(1, -0.23965345323085785), LIBSVM.SVMNode(1, 0.5313342809677124), LIBSVM.SVMNode(1, -0.002833075588569045), LIBSVM.SVMNode(1, 0.05375999957323074), LIBSVM.SVMNode(1, 0.12402049452066422), LIBSVM.SVMNode(1, -0.002047029323875904), LIBSVM.SVMNode(1, -0.17693717777729034), LIBSVM.SVMNode(1, -0.07985340058803558) … LIBSVM.SVMNode(1, -0.03471464663743973), LIBSVM.SVMNode(1, 0.06710212677717209), LIBSVM.SVMNode(1, -0.11737559735774994), LIBSVM.SVMNode(1, -0.01189055759459734), LIBSVM.SVMNode(1, 0.019903670996427536), LIBSVM.SVMNode(1, 0.0048777032643556595), LIBSVM.SVMNode(1, 0.004893505945801735), LIBSVM.SVMNode(1, -0.0604410320520401), LIBSVM.SVMNode(1, -0.07096513360738754), LIBSVM.SVMNode(1, -0.011674796231091022)]), 0.0, [0.4023034927725955; -0.47866948385899377; … ; 1.0; -1.0;;], Float64[], Float64[], [-0.5159824795034809], 3, 0.01, 200.0, 0.001, 1.0, 0.5, 0.1, true, false)), LIBSVM.EpsilonSVR(LIBSVM.Kernel.RadialBasis, 0.01, 0.1, 1.0, 3, 0.0, 0.001, true, false, LIBSVM.SVM{Float64, LIBSVM.Kernel.KERNEL}(LIBSVM.EpsilonSVR, LIBSVM.Kernel.RadialBasis, nothing, 100, 100, 2, Float64[], Int32[], Float64[], Int32[], LIBSVM.SupportVectors{Vector{Float64}, Matrix{Float32}}(77, Int32[], [0.34674155712127686, 0.9884548187255859, 0.3550981879234314, 0.9256850481033325, 0.7402107119560242, 0.1578449010848999, 0.8683705925941467, 0.19215458631515503, 0.9921402335166931, 0.8766130805015564 … 0.04867589473724365, 0.7412928938865662, 0.7370309233665466, 0.2706311345100403, 0.8965302109718323, 0.07030045986175537, 0.9454501867294312, 0.06412506103515625, 0.15291476249694824, 0.1212969422340393], Float32[-0.5009399 0.3044097 … -0.07096513 -0.011674796; 0.50529057 0.470386 … 0.27819282 0.32498732; … ; 0.56494606 0.21516198 … -0.061495535 -0.13789222; 0.97362113 -0.4151342 … 0.036586475 -0.009694248], Int32[1, 2, 4, 5, 6, 7, 9, 10, 12, 13 … 87, 88, 89, 91, 92, 93, 94, 97, 99, 100], LIBSVM.SVMNode[LIBSVM.SVMNode(1, -0.5009399056434631), LIBSVM.SVMNode(1, 0.3044097125530243), LIBSVM.SVMNode(1, 0.5313342809677124), LIBSVM.SVMNode(1, -0.002833075588569045), LIBSVM.SVMNode(1, 0.05375999957323074), LIBSVM.SVMNode(1, 0.12402049452066422), LIBSVM.SVMNode(1, -0.17693717777729034), LIBSVM.SVMNode(1, -0.07985340058803558), LIBSVM.SVMNode(1, 0.03588742017745972), LIBSVM.SVMNode(1, -0.04715218022465706) … LIBSVM.SVMNode(1, -0.0515398345887661), LIBSVM.SVMNode(1, 0.05576758086681366), LIBSVM.SVMNode(1, -0.03471464663743973), LIBSVM.SVMNode(1, -0.11737559735774994), LIBSVM.SVMNode(1, 0.01387971080839634), LIBSVM.SVMNode(1, 0.058396246284246445), LIBSVM.SVMNode(1, -0.01189055759459734), LIBSVM.SVMNode(1, 0.004893505945801735), LIBSVM.SVMNode(1, -0.07096513360738754), LIBSVM.SVMNode(1, -0.011674796231091022)]), 0.0, [-0.22819160384878637; 1.0; … ; -1.0; -1.0;;], Float64[], Float64[], [-0.5738610309433977], 3, 0.01, 200.0, 0.001, 1.0, 0.5, 0.1, true, false)), LIBSVM.EpsilonSVR(LIBSVM.Kernel.RadialBasis, 0.01, 0.1, 1.0, 3, 0.0, 0.001, true, false, LIBSVM.SVM{Float64, LIBSVM.Kernel.KERNEL}(LIBSVM.EpsilonSVR, LIBSVM.Kernel.RadialBasis, nothing, 100, 100, 2, Float64[], Int32[], Float64[], Int32[], LIBSVM.SupportVectors{Vector{Float64}, Matrix{Float32}}(76, Int32[], [0.4359814524650574, 0.9522580504417419, 0.09448134899139404, 0.9848877787590027, 0.43619465827941895, 0.4318832755088806, 0.8892332315444946, 0.9453982710838318, 0.8963696360588074, 0.0807352066040039 … 0.28807246685028076, 0.05103331804275513, 0.7284014821052551, 0.36523377895355225, 0.017303049564361572, 0.4506831169128418, 0.9959995150566101, 0.1694324016571045, 0.279338002204895, 0.7792866826057434], Float32[-0.5009399 0.3044097 … -0.07096513 -0.011674796; 0.50529057 0.470386 … 0.27819282 0.32498732; … ; 0.56494606 0.21516198 … -0.061495535 -0.13789222; 0.97362113 -0.4151342 … 0.036586475 -0.009694248], Int32[1, 2, 3, 4, 5, 7, 8, 9, 10, 11 … 88, 89, 90, 91, 92, 93, 94, 98, 99, 100], LIBSVM.SVMNode[LIBSVM.SVMNode(1, -0.5009399056434631), LIBSVM.SVMNode(1, 0.3044097125530243), LIBSVM.SVMNode(1, -0.23965345323085785), LIBSVM.SVMNode(1, 0.5313342809677124), LIBSVM.SVMNode(1, -0.002833075588569045), LIBSVM.SVMNode(1, 0.12402049452066422), LIBSVM.SVMNode(1, -0.002047029323875904), LIBSVM.SVMNode(1, -0.17693717777729034), LIBSVM.SVMNode(1, -0.07985340058803558), LIBSVM.SVMNode(1, -0.10801807045936584) … LIBSVM.SVMNode(1, 0.05576758086681366), LIBSVM.SVMNode(1, -0.03471464663743973), LIBSVM.SVMNode(1, 0.06710212677717209), LIBSVM.SVMNode(1, -0.11737559735774994), LIBSVM.SVMNode(1, 0.01387971080839634), LIBSVM.SVMNode(1, 0.058396246284246445), LIBSVM.SVMNode(1, -0.01189055759459734), LIBSVM.SVMNode(1, -0.0604410320520401), LIBSVM.SVMNode(1, -0.07096513360738754), LIBSVM.SVMNode(1, -0.011674796231091022)]), 0.0, [-0.3128827329986668; 1.0; … ; -1.0; 1.0;;], Float64[], Float64[], [-0.5804899421286054], 3, 0.01, 200.0, 0.001, 1.0, 0.5, 0.1, true, false))],)), (reservoir = (cell = (rng = Random.MersenneTwister(42, (0, 47124, 46122, 18, 44868, 762)),), carry = (Float32[-0.011674796; 0.32498732; … ; -0.13789222; -0.009694248;;],)), state_modifiers = (), readout = NamedTuple()))The SVR hyperparameters belong to the LIBSVM objective passed to train, for example LIBSVM.EpsilonSVR(cost = 10.0, epsilon = 0.01); SVMReadout itself takes only the dimensions and include_collect.