我是机器学习和机器视觉的新手。我之前用903张灰度图像(尺寸为640×480)训练了一个模型,一切都很好。但是现在当我尝试用4203张灰度图像(尺寸为80×120)来训练模型时,它会报错:
Epoch 1/10 14/132 [==>...........................] - ETA: 2:42:35 - loss: 0.0103 - accuracy: 0.9955---------------------------------------------------------------------------UnknownError Traceback (most recent call last)<ipython-input-15-b5674ef6abfe> in <module>()----> 1 model.fit(ds_train, epochs=10, verbose=1)6 frames/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name) 58 ctx.ensure_initialized() 59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,---> 60 inputs, attrs, num_outputs) 61 except core._NotOkStatusException as e: 62 if name is not None:UnknownError: 2 root error(s) found. (0) Unknown: /content/anime2/My Drive/Colab Notebooks/ML/All_train_cut/forward_1170.png; Input/output error [[{{node ReadFile}}]] [[IteratorGetNext]] [[IteratorGetNext/_4]] (1) Unknown: /content/anime2/My Drive/Colab Notebooks/ML/All_train_cut/forward_1170.png; Input/output error [[{{node ReadFile}}]] [[IteratorGetNext]]0 successful operations.0 derived errors ignored. [Op:__inference_train_function_445]Function call stack:train_function -> train_function
我使用Google Colab,并从Google Drive获取图像。
以下是我的模型结构以及我获取图像的方式:
import matplotlib.pyplot as pltimport numpy as npimport pandas as pdimport tensorflow as tfimport tensorflow_datasets as tfdsfrom tensorflow import kerasimport osos.environ['TF_CPP_MIN_LOG_LEVEL'] = "2"from google.colab import drivedrive.mount('/content/anime2')directory = '/content/anime2/My Drive/Colab Notebooks/ML/All_train_cut/'df = pd.read_csv(directory + 'train.csv')file_paths = df['file_name'].valueslabels = df['label'].valuesds_train = tf.data.Dataset.from_tensor_slices((file_paths, labels))def read_image(img_path, label): image = tf.io.read_file(directory + img_path) image = tf.image.decode_image(image, channels=1, dtype=tf.float32) return image, labelds_train = ds_train.map(read_image).batch(32)image_w = 80image_h = 120batch_size = 32model = keras.Sequential([ keras.layers.Input((image_h, image_w, 1)), keras.layers.Flatten(), keras.layers.Dense(5), keras.layers.Dense(3, activation='sigmoid'),])model.compile( optimizer=keras.optimizers.Adam(), loss=[ keras.losses.SparseCategoricalCrossentropy(from_logits=True), ], metrics='accuracy')model.fit(ds_train, epochs=10, verbose=1)
Google Drive文件夹中图像的链接:https://drive.google.com/drive/folders/1Uj7YhqyDJiq5d2ufQbI_MXe7T2-fOlk9?usp=sharing
但是所有图像都是正确的。而且每次报错的图像都不一样。所以我真的不明白哪里出了问题…谢谢!
回答:
现在我没有遇到错误了。我只是在尝试之前等了一天。所以这里是一些来自Google Colab GitHub问题#510的帖子:
- 我是一个偶尔的Colab Pro/Drive(付费订阅)用户,可能一个月使用2-3次。在过去几个月中,我大约有一半的时间遇到这个问题。我稍后会回来运行相同的代码,然后它会正常工作…
- 从其他人发布的内容来看,似乎Google对Drive设置了一些读/写限制,如果你超过了这些限制,除非你等待24小时不进行读写操作,否则你会得到这个错误。如果错误消息能更具体一些就好了。
- 是的,即使是使用google colab pro并且在drive中有100gb的存储空间,如果发生这种错误(在我这里,这种错误只会在重置运行时并再次运行colab时发生),那么你必须等待一天来重置配额。这可能是一个云端问题,他们需要“清空”用于计算大文件的虚拟存储和机器。