training_set_name, training_set_class, training_set_image = sess.run([fR.training_set_name, fR.training_set_class, fR.training_set_image])
文件读取器类的相关代码。
def return_training_set(): image_tra_name_batch, image_tra_class_batch = labelFileBatchProcessor(50, 1, "training")files_training = filenameLister2(image_tra_name_batch)images = build_images(files_training)return image_tra_name_batch, image_tra_class_batch, imagestraining_set_name, training_set_class, training_set_image = return_training_set()
labelFileBatchProcessor函数读取csv文件,进行洗牌操作,并返回一个包含名称和标签的批次。Filenamelister()
函数将这些名称转换为路径,并返回一个队列。该队列被传递给imageloader()
函数,返回图像张量。
问题在于加载的图像张量与image_tra_name_batch
不匹配。我认为问题出在执行build_images(files_training)
时加载了另一个批次。有没有解决方法?
我希望避免将所有内容存储在一个文件中,宁愿从csv文件中读取图像名称,进行洗牌,然后读取图像。
编辑1:一些相关函数
labelFileBatchProcessor函数:
def labelFileBatchProcessor(batch_size, num_epochs=None, what_set="training"): if what_set == "training": inputCsv = ["./data/BDRW_train/BDRW_train_1/labels.csv"] elif what_set == "validation": inputCsv = ["./data/BDRW_train/BDRW_train_2/labels.csv"] labelFile_queue = tf.train.string_input_producer(inputCsv, num_epochs=1, shuffle=False) image_name, image_class = labelFileInit(labelFile_queue) min_after_dequeue = 50 capacity = min_after_dequeue + 3 * batch_size image_name_batch, image_class_batch = tf.train.shuffle_batch( [image_name, image_class], batch_size=batch_size, capacity=capacity, min_after_dequeue=min_after_dequeue) return image_name_batch, image_class_batch
filenameLister2函数:
def filenameLister2(imageNameBatch): FILES_TRAINING = TRAINING_DIR + imageNameBatch + ".jpg" FILES_TRAINING = tf.train.string_input_producer(FILES_TRAINING, name="CSVFilenames") return FILES_TRAINING
build_images函数:
def build_images(files_training):image_reader = tf.WholeFileReader()_, image_file = image_reader.read(files_training)image_orig = tf.image.decode_jpeg(image_file)image = tf.image.resize_images(image_orig, [48, 48])image.set_shape((48, 48, 3))num_preprocess_threads = 1min_queue_examples = 256images = tf.train.batch([image], batch_size=BATCH_SIZE, num_threads=NUM_PREPROCESS_THREADS, capacity=BATCH_SIZE)return images
编辑2
我更新了labelFileBatchProcessor()
函数,并从build_images()
中移除了tf.train_batch()
。这可能是由其他问题引起的。我将张量放在同一个批次中,图像仍然不匹配。
def labelFileBatchProcessor(batch_size, num_epochs=None, what_set="training"):if what_set == "training": inputCsv = ["./data/BDRW_train/BDRW_train_1/labels.csv"]elif what_set == "validation": inputCsv = ["./data/BDRW_train/BDRW_train_2/labels.csv"]labelFile_queue = tf.train.string_input_producer(inputCsv, num_epochs=1, shuffle=False)image_name, image_class = labelFileInit(labelFile_queue)min_after_dequeue = 50capacity = min_after_dequeue + 3 * batch_sizefilenames = [TRAINING_DIR + image_name + ".jpg"]files_training = tf.train.string_input_producer(filenames, name="CSVFilenames", shuffle=False)images = build_images(files_training)image_name_batch, image_class_batch, images, filenames2 = tf.train.shuffle_batch( [image_name, image_class, images, filenames], batch_size=batch_size, capacity=capacity, min_after_dequeue=min_after_dequeue, allow_smaller_final_batch=True)return image_name_batch, image_class_batch, images, filenames2
回答:
据我所知,问题出现在输入管道中有两个独立的批处理阶段:
-
image_tra_name_batch
和image_tra_class_batch
在labelFileBatchProcessor()
中通过tf.train.shuffle_batch()
进行批处理。 -
images
在build_images()
中通过tf.train.batch()
进行批处理。
因此,每个文件名和类别与每个图像之间的关联丢失了。解决此类问题的一般方法是确保训练示例的所有属性(在本例中为image_name
、image_class
和image
)都传递给同一个tf.train.batch()
(或tf.train.shuffle_batch()
)调用。这样可以确保它们作为不可分割的张量元组进行预处理,并且保持关联性。