I have the following parameters set up :
parameter_space = {
'hidden_layer_sizes': [(sp_randint(100,600),sp_randint(100,600),), (sp_randint(100,600),)],
'activation': ['tanh', 'relu', 'logistic'],
'solver': ['sgd', 'adam', 'lbfgs'],
'alpha': stats.uniform(0.0001, 0.9),
'learning_rate': ['constant','adaptive']}
All the parameters except the hidden_layer_sizes is working as expected. However, fitting this RandomizedSearchCV model and displaying it's verbose text shows that it treats hidden_layer_sizes as :
hidden_layer_sizes=(<scipy.stats._distn_infrastructure.rv_frozen object
and goes on to throw : TypeError: '<=' not supported between instances of 'rv_frozen' and 'int'
This result is obtained instead of the expected 1 or 2 layer MLP with hidden layer neurons between 100 and 600. Any ideas / other related tips?
1 Answer
sp_randint returns an instance of the rv_discrete class / a randint object, in order to generate random numbers, the correct syntax should be sp_randint.rvs(low, high, size).
In order to make it work you need to define the parameter_space as below:
parameter_space = {
'hidden_layer_sizes': [(sp_randint.rvs(100,600,1),sp_randint.rvs(100,600,1),),
(sp_randint.rvs(100,600,1),)],
'activation': ['tanh', 'relu', 'logistic'],
'solver': ['sgd', 'adam', 'lbfgs'],
'alpha': uniform(0.0001, 0.9),
'learning_rate': ['constant','adaptive']}
