我正在使用各种算法实现强盗问题。我遇到的问题是，对于5个手臂和2000的时域，epsilon-greedy在epsilon值为0.95时表现得比UCB更好。我知道当时域与手臂数量相当时，epsilon-greedy确实表现得更好。但由于我的手臂数量远小于时域，UCB应该表现得更好。有人知道为什么会这样吗？我附上了我的UCB实现代码。
else if(algorithm.compare("UCB") == 0){if(pulls == 0){ armpullfrequency = new int[numArms]; armRewards = new float[numArms]; armmean = new double[numArms]; UCB = new double[numArms]; for(int i=0; i<numArms; i++){ armpullfrequency[i] = 0; armRewards[i] = 0.0; armmean[i] = (double)0; UCB[i] = (double)0; }}else{ armpullfrequency[pulled_arm] = armpullfrequency[pulled_arm] + 1; armRewards[pulled_arm] = armRewards[pulled_arm] + reward;}int selected_arm = 0;//int randint = (rand() % 100);if(pulls<=6){ for(int i=0;i<numArms;i++){ if(armpullfrequency[i]==0){ selected_arm = i; return selected_arm; } }}for(int i=0;i<numArms;i++){ int freq = armpullfrequency[i]; float prize = armRewards[i]; double mean = eval_mean(freq, prize); armmean[i] = mean; }for(int i=0; i<numArms;i++){ int freq = armpullfrequency[i]; double mean = armmean[i]; double UCBval = UCBUpdate(mean, freq, pulls); UCB[i] = UCBval;}selected_arm = LargestElementIndex(UCB, numArms);return(selected_arm);我的UCB和LargestElementIndex函数如下：
int LargestElementIndex(double arr[], int size){ int max = 0; for(int i=0;i<size; i++){ if(arr[i]>max){ max = arr[i]; } } return max;}int UCBUpdate(double mean, int freq, int pulls){ double result = mean + sqrt((double)2.0 *(log(pulls))/(double)freq); return result;}UCB的结果是：-maxMean 0.5805 numTotalPulls 2000 cumulativeReward 716.308Regret = 444.692
Epsilon Greedy的结果是：-max means 0.5805 numTotalPulls 2000 cumulativeReward 823.948Regret = 337.052
回答：
我怀疑错误出在以下代码中：
int LargestElementIndex(double arr[], int size){ int max = 0; for(int i=0;i<size; i++){ if(arr[i]>max){ max = arr[i]; } } return max;}这段代码并没有返回UCB值最大的手臂的索引（这可能是你想要的）。这段代码只是返回数组中最大的UCB值，并将其转换为int类型。这可能可以通过以下方式修复：
int LargestElementIndex(double arr[], int size){ double max_val = -1000.0; int max_idx = -1; for(int i=0;i<size; i++){ if(arr[i]>max_val){ max_val = arr[i]; max_idx = i; } } return max_idx;}