如何使用Caffe从自定义数据集正确生成LMDB文件?

我正在尝试使用Caffe生成LMDB文件来训练我的CNN。

我使用的数据集是斯坦福狗数据集。下载数据集后,我准备了训练数据集和测试数据集。我将配置保存到了train.txt和val.txt文件中。

然后,我尝试通过在控制台上运行以下两个命令来生成LMDB文件:

GLOG_logtostderr=1 /home/bme/caffe/build/tools/convert_imageset \
    --resize_height=200 --resize_width=200 --shuffle  \
    /home/bme/caffe/data/dogs/data/ \
    /home/bme/caffe/data/dogs/dogs_data/train.txt \
    /home/bme/caffe/examples/dogs/dogs_train_lmdbGLOG_logtostderr=1 /home/bme/caffe/build/tools/convert_imageset \
    --resize_height=200 --resize_width=200 --shuffle  \
    /home/bme/caffe/data/dogs/data/ \
    /home/bme/caffe/data/dogs/dogs_data/val.txt \
    /home/bme/caffe/examples/dogs/dogs_val_lmdb

但是,运行第一个命令时出现了以下错误:

I0331 18:13:39.625823  8810 convert_imageset.cpp:86] Shuffling dataI0331 18:13:39.627516  8810 convert_imageset.cpp:89] A total of 17493 images.I0331 18:13:39.627708  8810 db_lmdb.cpp:35] Opened lmdb /home/bme/caffe/examples/dogs/dogs_train_lmdbE0331 18:13:39.627734  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/108_n02111889-Samoyed17.jpg 108   E0331 18:13:39.627743  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/12_n02104365-schipperke145.jpg 12   E0331 18:13:39.627753  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/17_n02113712-miniature_poodle107.jpg 17   E0331 18:13:39.627760  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/12_n02104365-schipperke63.jpg 12   E0331 18:13:39.627768  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/97_n02105056-groenendael102.jpg 97   E0331 18:13:39.627774  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/74_n02096585-Boston_bull38.jpg 74   E0331 18:13:39.627781  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/4_n02113799-standard_poodle145.jpg 4   E0331 18:13:39.627789  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/22_n02097047-miniature_schnauzer27.jpg 22   E0331 18:13:39.627797  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/80_n02107312-miniature_pinscher167.jpg 80   E0331 18:13:39.627805  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/95_n02091134-whippet42.jpg 95 

第二个命令的错误如下:

I0331 18:13:39.625823  8810 convert_imageset.cpp:86] Shuffling dataI0331 18:13:39.627516  8810 convert_imageset.cpp:89] A total of 17493 images.I0331 18:13:39.627708  8810 db_lmdb.cpp:35] Opened lmdb /home/bme/caffe/examples/dogs/dogs_train_lmdbE0331 18:13:39.627734  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/108_n02111889-Samoyed17.jpg 108   E0331 18:13:39.627743  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/12_n02104365-schipperke145.jpg 12   E0331 18:13:39.627753  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/17_n02113712-miniature_poodle107.jpg 17   E0331 18:13:39.627760  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/12_n02104365-schipperke63.jpg 12   E0331 18:13:39.627768  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/97_n02105056-groenendael102.jpg 97   E0331 18:13:39.627774  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/74_n02096585-Boston_bull38.jpg 74   E0331 18:13:39.627781  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/4_n02113799-standard_poodle145.jpg 4   E0331 18:13:39.627789  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/22_n02097047-miniature_schnauzer27.jpg 22   E0331 18:13:39.627797  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/80_n02107312-miniature_pinscher167.jpg 80   E0331 18:13:39.627805  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/95_n02091134-whippet42.jpg 95   E0331 18:13:39.627812  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/56_n02101556-clumber86.jpg 56   E0331 18:13:39.627820  8810 io.cpp:80] Could not open or find file /home/bme/caffe/data/dogs/data/26_n02085782-Japanese_spaniel36.jpg 26  

我尝试通过更改train.txt和val.txt文件的权限来解决这个问题,但错误仍然存在。那么,有什么建议可以解决这个问题吗?

谢谢


回答:

您发布的输出都是关于训练集生成的(它们是相同的)。看起来在17,493张图片中,有12个损坏的文件,Caffe无法找到/打开这些文件。
请检查这些文件是否存在,并且它们没有损坏。

Related Posts

使用LSTM在Python中预测未来值

这段代码可以预测指定股票的当前日期之前的值,但不能预测…

如何在gensim的word2vec模型中查找双词组的相似性

我有一个word2vec模型,假设我使用的是googl…

dask_xgboost.predict 可以工作但无法显示 – 数据必须是一维的

我试图使用 XGBoost 创建模型。 看起来我成功地…

ML Tuning – Cross Validation in Spark

我在https://spark.apache.org/…

如何在React JS中使用fetch从REST API获取预测

我正在开发一个应用程序,其中Flask REST AP…

如何分析ML.NET中多类分类预测得分数组?

我在ML.NET中创建了一个多类分类项目。该项目可以对…

发表回复

您的邮箱地址不会被公开。 必填项已用 * 标注