我正在进行一个从音频中分离出人声部分的项目。我使用的是 DSD100 数据集,但在进行测试时,我使用了 DSD100subset 数据集,我只使用了混合音和人声。我的工作基于这篇文章。
首先,我处理音频以提取频谱图,并将其放入列表中,所有音频形成了四个列表(trainMixed, trainVocals, testMixed, testVocals)。如下所示:
def to_spec(wav, n_fft=1024, hop_length=256): return librosa.stft(wav, n_fft=n_fft, hop_length=hop_length)def prepareData(filename, sr=22050, hop_length=256, n_fft=1024): audio_wav = librosa.load(filename, sr=sr, mono=True, duration=30)[0] audio_spec=to_spec(audio_wav, n_fft=n_fft, hop_length=hop_length) audio_spec_mag = np.abs(audio_spec) maxVal = np.max(audio_spec_mag) return audio_spec_mag, maxVal# FOR EVERY LIST (trainMixed, trainVocals, testMixed, testVocals)trainMixed = []trainMixedNum = 0for (root, dirs, files) in walk('./Dev-subset-mix/Dev/'): for d in dirs: filenameMix = './Dev-subset-mix/Dev/'+d+'/mixture.wav' spec_mag, maxVal = prepareData(filenameMix, n_fft=1024, hop_length=256) trainMixed.append(spec_mag/maxVal)
接下来我构建模型:
然后运行模型:
model.fit(trainMixed, trainVocals,epochs=10, validation_data=(testMixed, testVocals))
但我得到了这样的结果:
ValueError: in user code: /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:806 train_function * return step_function(self, iterator) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:796 step_function ** outputs = model.distribute_strategy.run(run_step, args=(data,)) /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:1211 run return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs) /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2585 call_for_each_replica return self._call_for_each_replica(fn, args, kwargs) /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2945 _call_for_each_replica return fn(*args, **kwargs) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:789 run_step ** outputs = model.train_step(data) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:747 train_step y_pred = self(x, training=True) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py:976 __call__ self.name) /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/input_spec.py:158 assert_input_compatibility ' input tensors. Inputs received: ' + str(inputs)) ValueError: Layer sequential_1 expects 1 inputs, but it received 2 input tensors. Inputs received: [<tf.Tensor 'IteratorGetNext:0' shape=(None, 2584) dtype=float32>, <tf.Tensor 'IteratorGetNext:1' shape=(None, 2584) dtype=float32>]
我对这个话题还比较新,提前感谢提供的帮助。
回答:
这可能是向 Keras 的 fit()
函数指定输入数据的问题。我建议使用 tf.data.Dataset
作为 fit()
的输入,如下所示:
然后你还可以在 TF 数据集上使用像 shuffle()
和 batch()
这样的函数。
EDIT: 看起来你的输入形状也有问题。你为第一个卷积层指定的 input_shape
是 (513, 25, 1)
,所以输入应该是一个形状为 (batch_size, 513, 25, 1)
的批次张量,而你输入的形状是 (batch_size, 2584)
。因此,你需要重塑并可能剪切你的输入到指定的形状,或者指定一个新的形状。