Keras Wrapper für Scikit Learn - AUC Scorer funktioniert nicht

Question

Ich versuche Keras Scikit Learn Wrapper zu verwenden, um die zufällige Suche nach Parametern einfacher zu machen. Ich schrieb hier ein Beispiel-Code, wo:

1. I erzeugen eine künstliche Datenmenge:

I moons von scikit learn

from sklearn.datasets import make_moons 
dataset = make_moons(1000) 

bin mit

2. Modellbauerdefinition:

I definieren build_fn Funktion benötigt:

def build_fn(nr_of_layers = 2, 
      first_layer_size = 10, 
      layers_slope_coeff = 0.8, 
      dropout = 0.5, 
      activation = "relu", 
      weight_l2 = 0.01, 
      act_l2 = 0.01, 
      input_dim = 2): 

    result_model = Sequential() 
    result_model.add(Dense(first_layer_size, 
          input_dim = input_dim, 
          activation=activation, 
          W_regularizer= l2(weight_l2), 
          activity_regularizer=activity_l2(act_l2) 
          )) 

    current_layer_size = int(first_layer_size * layers_slope_coeff) + 1 

    for index_of_layer in range(nr_of_layers - 1): 

     result_model.add(BatchNormalization()) 
     result_model.add(Dropout(dropout)) 
     result_model.add(Dense(current_layer_size, 
           W_regularizer= l2(weight_l2), 
           activation=activation, 
           activity_regularizer=activity_l2(act_l2) 
           )) 

     current_layer_size = int(current_layer_size * layers_slope_coeff) + 1 

    result_model.add(Dense(1, 
          activation = "sigmoid", 
          W_regularizer = l2(weight_l2))) 

    result_model.compile(optimizer="rmsprop", metrics = ["accuracy"], loss = "binary_crossentropy") 

    return result_model 

NeuralNet = KerasClassifier(build_fn) 

3. Parameter grid Definition:

Dann I definiert einen Parameter grid:

param_grid = { 
    "nr_of_layers" : [2, 3, 4, 5], 
    "first_layer_size" : [5, 10, 15], 
    "layers_slope_coeff" : [0.4, 0.6, 0.8], 
    "dropout" : [0.3, 0.5, 0.8], 
    "weight_l2" : [0.01, 0.001, 0.0001], 
    "verbose" : [0], 
    "batch_size" : [1], 
    "nb_epoch" : [30] 
} 

4. Lief domizedSearchCV Phase:

I definiert RandomizedSearchCV Objekt und mit Werten aus künstlichen Daten-Set ausgestattet:

random_search = RandomizedSearchCV(NeuralNet, 
    param_distributions=param_grid, verbose=2, n_iter=1, scoring="roc_auc") 
random_search.fit(dataset[0], dataset[1]) 

Was habe ich (nach diesem Code in der Konsole ausgeführt wird) ist:

Traceback (most recent call last): 
    File "C:\Anaconda2\lib\site-packages\IPython\core\interactiveshell.py", line 2885, in run_code 
    exec(code_obj, self.user_global_ns, self.user_ns) 
    File "<ipython-input-3-c5bdbc2770b7>", line 2, in <module> 
    random_search.fit(dataset[0], dataset[1]) 
    File "C:\Anaconda2\lib\site-packages\sklearn\grid_search.py", line 996, in fit 
    return self._fit(X, y, sampled_params) 
    File "C:\Anaconda2\lib\site-packages\sklearn\grid_search.py", line 553, in _fit 
    for parameters in parameter_iterable 
    File "C:\Anaconda2\lib\site-packages\sklearn\externals\joblib\parallel.py", line 800, in __call__ 
    while self.dispatch_one_batch(iterator): 
    File "C:\Anaconda2\lib\site-packages\sklearn\externals\joblib\parallel.py", line 658, in dispatch_one_batch 
    self._dispatch(tasks) 
    File "C:\Anaconda2\lib\site-packages\sklearn\externals\joblib\parallel.py", line 566, in _dispatch 
    job = ImmediateComputeBatch(batch) 
    File "C:\Anaconda2\lib\site-packages\sklearn\externals\joblib\parallel.py", line 180, in __init__ 
    self.results = batch() 
    File "C:\Anaconda2\lib\site-packages\sklearn\externals\joblib\parallel.py", line 72, in __call__ 
    return [func(*args, **kwargs) for func, args, kwargs in self.items] 
    File "C:\Anaconda2\lib\site-packages\sklearn\cross_validation.py", line 1550, in _fit_and_score 
    test_score = _score(estimator, X_test, y_test, scorer) 
    File "C:\Anaconda2\lib\site-packages\sklearn\cross_validation.py", line 1606, in _score 
    score = scorer(estimator, X_test, y_test) 
    File "C:\Anaconda2\lib\site-packages\sklearn\metrics\scorer.py", line 175, in __call__ 
    y_pred = y_pred[:, 1] 
IndexError: index 1 is out of bounds for axis 1 with size 1 

Diese Code funktioniert gut, wenn statt scoring = "roc_auc" ich accuracy metric verwendet. Kann mir jemand erklären, was los ist? Hatte jemand ähnliches Problem?

Answer 1

Es gibt einen Fehler im KerasClassifier, der dieses Problem verursacht. Ich habe ein Problem dafür im Repo geöffnet. https://github.com/fchollet/keras/issues/2864

Die Lösung ist auch da drin. Sie können zwischenzeitlich Ihren eigenen KerasClassifier als temporären Workaround definieren.

class FixedKerasClassifier(KerasClassifier): 
    def predict_proba(self, X, **kwargs): 
     kwargs = self.filter_sk_params(Sequential.predict_proba, kwargs) 
     probs = self.model.predict_proba(X, **kwargs) 
     if(probs.shape[1] == 1): 
      probs = np.hstack([1-probs,probs]) 
     return probs