我有大约10000张图像，无法一次性装入内存。因此，目前我只能读取1000张图像并进行训练…

我的代码如下：

img_dir = "TrainingSet" # 输入所有图像的目录
image_path = os.path.join(img_dir+"/images",'*.bmp')
files = glob.glob(image_path)
images = []
masks = []
contours = []
indexes = []
files_names = []
for f1 in np.sort(files):
    img = cv2.imread(f1)
    result = re.search('original_cropped_(.*).bmp', str(f1))
    idx = result.group(1)
    mask_path = img_dir+"/masks/mask_cropped_"+str(idx)+".bmp"
    mask = cv2.imread(mask_path,0)
    contour_path = img_dir+"/contours/contour_cropped_"+str(idx)+".bmp"
    contour = cv2.imread(contour_path,0)
    indexes.append(idx)
    images.append(img)
    masks.append(mask)
    contours.append(contour)
train_df = pd.DataFrame({"id":indexes,"masks": masks, "images": images,"contours": contours })
train_df.sort_values(by="id",ascending=True,inplace=True)
print(train_df.shape)
img_size_target = (256,256)
ids_train, ids_valid, x_train, x_valid, y_train, y_valid, c_train, c_valid = train_test_split(
    train_df.index.values,
    np.array(train_df.images.apply(lambda x: cv2.resize(x,img_size_target).reshape(img_size_target[0],img_size_target[1],3))),
    np.array(train_df.masks.apply(lambda x: cv2.resize(x,img_size_target).reshape(img_size_target[0],img_size_target[1],1))),
    np.array(train_df.contours.apply(lambda x: cv2.resize(x,img_size_target).reshape(img_size_target[0],img_size_target[1],1))),
    test_size=0.2, random_state=1337)
#在这里我们定义模型架构...
#......
#模型定义结束
# 训练
optimizer = Adam(lr=1e-3,decay=1e-10)
model.compile(loss="binary_crossentropy", optimizer=optimizer, metrics=["accuracy"])
early_stopping = EarlyStopping(patience=10, verbose=1)
model_checkpoint = ModelCheckpoint("./keras.model", save_best_only=True, verbose=1)
reduce_lr = ReduceLROnPlateau(factor=0.5, patience=5, min_lr=0.00001, verbose=1)
epochs = 200
batch_size = 32
history = model.fit(x_train, y_train,
                validation_data=[x_valid, y_valid],
                epochs=epochs,
                batch_size=batch_size,
                callbacks=[early_stopping, model_checkpoint, reduce_lr])

我想知道如何修改我的代码，以便每次处理一小批图像，而不需要将所有10000张图像都加载到内存中？这意味着算法将在每次epoch从目录中读取X张图像并进行训练，然后继续处理下一批X张图像，直到处理完所有图像。

这里的X是指可以装入内存的合理数量的图像。

回答：

使用fit_generator替代fit

def generate_batch_data(num):
    # 在这里加载X张图像
    return images
model.fit_generator(generate_batch_data(X),
        samples_per_epoch=10000, nb_epoch=10)

或者，您可以使用train_on_batch替代fit

关于这个话题的GitHub讨论：https://github.com/keras-team/keras/issues/2708