我正在尝试在我的ResNet-50模型上实现分层K折交叉验证。不幸的是,当我对标签进行独热编码并尝试使用分层K折来分割我的数据时,我遇到了这个错误:TypeError: A sparse matrix was passed, but dense data is required. Use X.toarray() to convert to a dense numpy array.
独热编码器的实现方式如下:
from sklearn.preprocessing import OneHotEncoderenc = OneHotEncoder()enc.fit(Y)Y = enc.transform(Y)Y.toarray()
如果我不对标签进行独热编码,我在拟合模型时会得到这个错误:ValueError: Shapes (None, 1) and (None, 4) are incompatible
这是实现分层K折的代码:
for train, test in skf.split(X, Y): x = base_model.output x = GlobalAveragePooling2D()(x) x = Dropout(0.7)(x) predictions = Dense(num_classes, activation= 'softmax')(x) model = Model(inputs = base_model.input, outputs = predictions) adam = Adam(lr=0.0001) model.compile(optimizer= adam, loss='categorical_crossentropy', metrics=['accuracy']) # Training history = model.fit(X[train], Y[train], epochs = 100, batch_size = 16)
其中num_classes = 4。
所以我的问题是:
Q: 如何使独热编码的标签与skf.split()一起工作?
回答:
如果问题是模型不接受使用OneHotEncoding后生成的sparse matrix
(这是通常的预期行为),你可以尝试通过设置OneHotEncoding模型的参数sparse=False
来改变它。
from sklearn.preprocessing import OneHotEncoderenc = OneHotEncoder(sparse=False)enc.fit(Y)Y = enc.transform(Y)