Wie trainiere ich ein Modell mit optimierten Gewichten?

Question

Ich arbeite an der Optimierung der Parameter des Modells. Grob gesagt trainierte ich die Gewichte eines Modells mit einer gewissen Struktur. Ich benutze jetzt diese Gewichte, um ein anderes Modell mit exakt derselben Struktur zu trainieren. Aber ich bin nicht in der Lage, die gleiche Genauigkeit zu erreichen, in der ersten Epoche nicht annähernd.

Der folgende Codeabschnitt erklärt das ganze Problem.

from __future__ import print_function 

from keras.preprocessing import sequence 
from keras.models import Sequential 
from keras.layers import Dense, Dropout, Activation 
from keras.layers import Embedding 
from keras.layers import Conv1D, GlobalMaxPooling1D 
from keras.datasets import imdb 

# set parameters: 
max_features = 5000 
maxlen = 400 
batch_size = 32 
embedding_dims = 50 
filters = 250 
kernel_size = 3 
hidden_dims = 250 

print('Loading data...') 
(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=max_features) 
print(len(x_train), 'train sequences') 
print(len(x_test), 'test sequences') 

print('Pad sequences (samples x time)') 
x_train = sequence.pad_sequences(x_train, maxlen=maxlen) 
x_test = sequence.pad_sequences(x_test, maxlen=maxlen) 
print('x_train shape:', x_train.shape) 
print('x_test shape:', x_test.shape) 

print('Build model...') 
model = Sequential() 

# we start off with an efficient embedding layer which maps 
# our vocab indices into embedding_dims dimensions 
model.add(Embedding(max_features, 
        embedding_dims, 
        input_length=maxlen)) 
model.add(Dropout(0.2)) 

# we add a Convolution1D, which will learn filters 
# word group filters of size filter_length: 
model.add(Conv1D(filters, kernel_size, padding='valid', activation='relu', strides=1)) 
# we use max pooling: 
model.add(GlobalMaxPooling1D()) 

# We add a vanilla hidden layer: 
model.add(Dense(hidden_dims)) 
model.add(Dropout(0.2)) 
model.add(Activation('relu')) 

# We project onto a single unit output layer, and squash it with a sigmoid: 
model.add(Dense(1)) 
model.add(Activation('sigmoid')) 

model.compile(loss='mse',Optimizer='sgd', metrics=['accuracy']) 
model.fit(x_train, y_train,batch_size=batch_size,epochs=50, validation_data=(x_test, y_test)) 

# Extraction of weights 
con_weight=model.layers[2].get_weights()[0] 
con_bias=model.layers[2].get_weights()[1] 

mid_weight=model.layers[4].get_weights()[0] 
mid_bias=model.layers[4].get_weights()[1] 

pl_weight=model.layers[7].get_weights()[0] 
pl_bias=model.layers[7].get_weights()[1] 

Gewichte unter Verwendung eines anderen Modells zu trainieren

model1 = Sequential() 

# we start off with an efficient embedding layer which maps 
# our vocab indices into embedding_dims dimensions 
model1.add(Embedding(max_features, 
        embedding_dims, 
        input_length=maxlen)) 
model1.add(Dropout(0.2)) 

# we add a Convolution1D, which will learn filters 
# word group filters of size filter_length: 
model1.add(Conv1D(filters, kernel_size, padding='valid',activation='relu', strides=1,weights=[con_weight,con_bias])) 
# we use max pooling: 
model1.add(GlobalMaxPooling1D()) 

# We add a vanilla hidden layer: 
model1.add(Dense(hidden_dims,weights=[mid_weight,mid_bias])) 
model1.add(Dropout(0.2)) 
model1.add(Activation('relu')) 

# We project onto a single unit output layer, and squash it with a sigmoid: 
model1.add(Dense(1,weights=[pl_weight,pl_bias])) 
model1.add(Activation('sigmoid')) 
model1.trainable=False 
model1.compile(loss='mse', optimizer='sgd', metrics=['accuracy']) 
model1.fit(x_train, y_train, batch_size=batch_size, epochs=1, 
validation_data=(x_test, y_test)) 

Answer 1

Einbettungen haben Gewichte tun, und du vergisst sie.

Aber Ihre Modelle sind identisch, nur:

model1.set_weights(model.get_weights())