我正在尝试重现使用神经网络降低数据维度一文中，使用自动编码器对olivetti人脸数据集进行处理的结果。我使用的是MNIST数字集的matlab代码的修改版本，但我遇到了一些困难。无论我如何调整epoch的数量、学习率或动量，堆叠的RBMs在进入微调阶段时都存在大量误差，因此在微调阶段很难有显著改善。我在另一个实值数据集上也遇到了类似的问题。

对于第一层，我使用了一个具有较小学习率的RBM（如论文中所述），并且有：

negdata = poshidstates*vishid' + repmat(visbiases,numcases,1);

我相当确信我正在按照补充材料中的说明进行操作，但我无法获得正确的误差值。

我是否遗漏了什么？ 请看下面我用于实值可见单元RBMs的代码，以及整个深度训练的代码。 其余代码可以在这里找到。

rbmvislinear.m:

epsilonw      = 0.001; % Learning rate for weights epsilonvb     = 0.001; % Learning rate for biases of visible unitsepsilonhb     = 0.001; % Learning rate for biases of hidden units weightcost  = 0.0002;  initialmomentum  = 0.5;finalmomentum    = 0.9;[numcases numdims numbatches]=size(batchdata);if restart ==1,  restart=0;  epoch=1;% Initializing symmetric weights and biases.  vishid     = 0.1*randn(numdims, numhid);  hidbiases  = zeros(1,numhid);  visbiases  = zeros(1,numdims);  poshidprobs = zeros(numcases,numhid);  neghidprobs = zeros(numcases,numhid);  posprods    = zeros(numdims,numhid);  negprods    = zeros(numdims,numhid);  vishidinc  = zeros(numdims,numhid);  hidbiasinc = zeros(1,numhid);  visbiasinc = zeros(1,numdims);  sigmainc = zeros(1,numhid);  batchposhidprobs=zeros(numcases,numhid,numbatches);endfor epoch = epoch:maxepoch, fprintf(1,'epoch %d\r',epoch);  errsum=0; for batch = 1:numbatches, if (mod(batch,100)==0)     fprintf(1,' %d ',batch); end%%%%%%%%% START POSITIVE PHASE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%  data = batchdata(:,:,batch);  poshidprobs = 1./(1 + exp(-data*vishid - repmat(hidbiases,numcases,1)));    batchposhidprobs(:,:,batch)=poshidprobs;  posprods    = data' * poshidprobs;  poshidact   = sum(poshidprobs);  posvisact = sum(data);%%%%%%%%% END OF POSITIVE PHASE  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%  poshidstates = poshidprobs > rand(numcases,numhid);%%%%%%%%% START NEGATIVE PHASE  %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%  negdata = poshidstates*vishid' + repmat(visbiases,numcases,1);% + randn(numcases,numdims) if not using mean  neghidprobs = 1./(1 + exp(-negdata*vishid - repmat(hidbiases,numcases,1)));    negprods  = negdata'*neghidprobs;  neghidact = sum(neghidprobs);  negvisact = sum(negdata); %%%%%%%%% END OF NEGATIVE PHASE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%  err= sum(sum( (data-negdata).^2 ));   errsum = err + errsum;   if epoch>5,     momentum=finalmomentum;   else     momentum=initialmomentum;   end;%%%%%%%%% UPDATE WEIGHTS AND BIASES %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%    vishidinc = momentum*vishidinc + ...                epsilonw*( (posprods-negprods)/numcases - weightcost*vishid);    visbiasinc = momentum*visbiasinc + (epsilonvb/numcases)*(posvisact-negvisact);    hidbiasinc = momentum*hidbiasinc + (epsilonhb/numcases)*(poshidact-neghidact);    vishid = vishid + vishidinc;    visbiases = visbiases + visbiasinc;    hidbiases = hidbiases + hidbiasinc;%%%%%%%%%%%%%%%% END OF UPDATES %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% endfprintf(1, '\nepoch %4i error %f \n', epoch, errsum);end

dofacedeepauto.m:

clear allclose allmaxepoch=200; %In the Science paper we use maxepoch=50, but it works just fine. numhid=2000; numpen=1000; numpen2=500; numopen=30;fprintf(1,'Pretraining a deep autoencoder. \n');fprintf(1,'The Science paper used 50 epochs. This uses %3i \n', maxepoch);load fdata%makeFaceData;[numcases numdims numbatches]=size(batchdata);fprintf(1,'Pretraining Layer 1 with RBM: %d-%d \n',numdims,numhid);restart=1;rbmvislinear;hidrecbiases=hidbiases; save mnistvh vishid hidrecbiases visbiases;maxepoch=50;fprintf(1,'\nPretraining Layer 2 with RBM: %d-%d \n',numhid,numpen);batchdata=batchposhidprobs;numhid=numpen;restart=1;rbm;hidpen=vishid; penrecbiases=hidbiases; hidgenbiases=visbiases;save mnisthp hidpen penrecbiases hidgenbiases;fprintf(1,'\nPretraining Layer 3 with RBM: %d-%d \n',numpen,numpen2);batchdata=batchposhidprobs;numhid=numpen2;restart=1;rbm;hidpen2=vishid; penrecbiases2=hidbiases; hidgenbiases2=visbiases;save mnisthp2 hidpen2 penrecbiases2 hidgenbiases2;fprintf(1,'\nPretraining Layer 4 with RBM: %d-%d \n',numpen2,numopen);batchdata=batchposhidprobs;numhid=numopen; restart=1;rbmhidlinear;hidtop=vishid; toprecbiases=hidbiases; topgenbiases=visbiases;save mnistpo hidtop toprecbiases topgenbiases;backpropface; 

感谢您的时间

回答：

真傻，我忘记更改反向传播微调脚本 (backprop.m) 了。 必须将输出层（重建人脸的地方）更改为实值单位。 即：

dataout = w7probs*w8;