我目前正在开发一个开源ANN(出于兴趣和学习目的),最近我对代码做了一个我认为相对较小的改动,但不知为何,这使得ANN的运行速度提高了16倍。(至少根据我的测试是这样)
ANN/ANN5.py:(旧的ANN)
from random import uniformclass Neuron(object): def __init__(self, parents=[]): self.parents = [{ 'neuron': parent, 'weight': uniform(-1, 1), 'slope': uniform(-1, 1), } for parent in parents] def calculate(self, increment=0): self.output = sum([parent['neuron'].output * (parent['weight'] + increment * parent['slope']) for parent in self.parents]) > 0 def mutate(self, increment): for parent in self.parents: parent['weight'] += increment * parent['slope'] parent['slope'] = uniform(-1, 1) def get_genome(self): return [parent['weight'] for parent in self.parents] def set_genome(self, value): for i, parent in enumerate(self.parents): parent['weight'] = value[i] genome = property(get_genome, set_genome)class NeuralNetwork(object): def __init__(self, inputs, outputs, hidden, rows): self.bias = Neuron() self.neurons = [] for row in xrange(rows): if row == 0: self.neurons.append([Neuron(parents=[]) for input_ in xrange(inputs)]) elif row == rows - 1: self.neurons.append([Neuron(parents=self.neurons[row - 1] + [self.bias]) for output in xrange(outputs)]) else: self.neurons.append([Neuron(parents=self.neurons[row - 1] + [self.bias]) for column in xrange(hidden)]) self.bias.output = True def calculate(self, inputs, increment=0): for i, neuron_row in enumerate(self.neurons): for j, neuron in enumerate(neuron_row): if i == 0: neuron.output = inputs[j] else: neuron.calculate(increment=increment) return [neuron.output for neuron in self.neurons[-1]] def mutate(self, increment): for neuron_row in self.neurons: for neuron in neuron_row: neuron.mutate(increment=increment) def get_genome(self): genome = [] for neuron_row in self.neurons[1:]: genome.append([neuron.genome for neuron in neuron_row]) return genome def set_genome(self, value): for i, neuron_row in enumerate(self.neurons[1:]): for j, neuron in enumerate(neuron_row): neuron.genome = value[i][j] genome = property(get_genome, set_genome)
ANN/ANN.py:(新的ANN)
from random import uniformclass Neuron(object): def __init__(self, parents=[]): self.parents = [{ 'neuron': parent, 'weight': uniform(-1, 1), 'slope': uniform(-1, 1), } for parent in parents] def calculate(self, increment=0): self.output = sum([parent['neuron'].output * (parent['weight'] + increment * parent['slope']) for parent in self.parents]) > 0 def mutate(self, increment): for parent in self.parents: parent['weight'] += increment * parent['slope'] parent['slope'] = uniform(-1, 1) def get_genome(self): return [parent['weight'] for parent in self.parents] def set_genome(self, value): for i, parent in enumerate(self.parents): parent['weight'] = value[i] genome = property(get_genome, set_genome)class NeuralNetwork(object): def __init__(self, inputs, outputs, hidden, rows): self.bias = Neuron() self.neurons = [[Neuron(parents=[]) for input_ in xrange(inputs)]] for row in xrange(rows - 2): self.neurons.append([Neuron(parents=self.neurons[-1] + [self.bias]) for output in xrange(outputs)]) self.neurons.append([Neuron(parents=self.neurons[-1] + [self.bias]) for output in xrange(outputs)]) self.bias.output = True def calculate(self, inputs, increment=0): for i, neuron_row in enumerate(self.neurons): for j, neuron in enumerate(neuron_row): if i == 0: neuron.output = inputs[j] else: neuron.calculate(increment=increment) return [neuron.output for neuron in self.neurons[-1]] def mutate(self, increment): for neuron_row in self.neurons: for neuron in neuron_row: neuron.mutate(increment=increment) def get_genome(self): genome = [] for neuron_row in self.neurons[1:]: genome.append([neuron.genome for neuron in neuron_row]) return genome def set_genome(self, value): for i, neuron_row in enumerate(self.neurons[1:]): for j, neuron in enumerate(neuron_row): neuron.genome = value[i][j] genome = property(get_genome, set_genome)
从ANN/ANN5.py到ANN/ANN.py的差异:
- self.neurons = []- for row in xrange(rows):- if row == 0:- self.neurons.append([Neuron(parents=[]) for input_ in xrange(inputs)])- elif row == rows - 1:- self.neurons.append([Neuron(parents=self.neurons[row - 1] + [self.bias]) for output in xrange(outputs)])- else:- self.neurons.append([Neuron(parents=self.neurons[row - 1] + [self.bias]) for column in xrange(hidden)])+ self.neurons = [[Neuron(parents=[]) for input_ in xrange(inputs)]]+ for row in xrange(rows - 2):+ self.neurons.append([Neuron(parents=self.neurons[-1] + [self.bias]) for output in xrange(outputs)])+ self.neurons.append([Neuron(parents=self.neurons[-1] + [self.bias]) for output in xrange(outputs)])
(都在NeuralNetwork的__init__
方法中)
tests.py:
from random import randintfrom time import timefrom ANN.ANN import NeuralNetwork# from ANN.ANN2 import NeuralNetwork as NeuralNetwork2# from ANN.ANN3 import NeuralNetwork as NeuralNetwork3# from ANN.ANN4 import NeuralNetwork as NeuralNetwork4from ANN.ANN5 import NeuralNetwork as NeuralNetwork5def test(NeuralNetwork=NeuralNetwork): time_ = time() ANNs = [] for i in xrange(10): ANNs.append(NeuralNetwork(inputs=49, outputs=3, hidden=49, rows=5)) for i, ANN in enumerate(ANNs[:1]): for j in xrange(11): for k in xrange(len(ANNs) / 2): for l in xrange(20): ANN.calculate([randint(0, 1) for _ in xrange(49)], increment=j/10) ANNs[k + len(ANNs)/2 * (i < len(ANNs)/2)].calculate([randint(0, 1) for _ in xrange(49)]) # print 'ANN {0} mutation {1:02d} opponent {2} turn {3:02d}'.format(i + 1, j + 1, k + 1, l + 1) ANN.mutate(increment=randint(1, 100)) return time() - time_if __name__ == '__main__': print 'time: {0}'.format(test()) # print 'time 2: {0}'.format(test(NeuralNetwork2)) # print 'time 3: {0}'.format(test(NeuralNetwork3)) # print 'time 4: {0}'.format(test(NeuralNetwork4)) print 'time 5: {0}'.format(test(NeuralNetwork5))
我注释掉了ANN2、ANN3和ANN4,因为它们是更旧的ANN版本,我仅在本地存储(没有在Github上)以便比较性能。目前我只关心ANN5.py和ANN.py之间的性能变化。
我之所以使用for i, ANN in enumerate(ANNs[:1]):
而不是for i, ANN in enumerate(ANNs):
,是因为后者的测试时间太长,我认为即使不重复处理10个ANN,结果仍然完全足够(我偶尔会测试所有10个以确保准确性)。
当我最后一次运行tests.py时,得到的结果是:
time: 0.454416036606time 5: 8.02504611015
结果总是大致接近这个值。
我已经进行了各种测试来比较ANN.py和ANN5.py的功能,到目前为止,在相同情况下它们表现完全相同。我使用genome属性创建了两个相同的ANN,一个使用ANN.py中的NeuralNetwork类,另一个使用ANN5.py中的NeuralNetwork类,在相同输入下,它们总是给我相同的结果。
所以我的问题是,到底发生了什么?我意识到我的问题不够精确,但我真的不知道为什么会有这么大的性能差异。我希望旧的ANN(ANN5.py)只是因为我初始化ANN的方式在后台做了什么非常低效的事情,而新的ANN(ANN.py)正确地初始化了它,但我担心新的ANN可能缺少了一些东西,而这些东西在手动测试时没有显示出来/没有造成任何差异。
回答:
我之前忘了这个问题,并假设之前发生了什么不好的事情,但最近我注意到生成的genome相当短。那时我注意到我正在循环xrange(outputs),它的长度是3,而不是xrange(hidden),它的长度是49。
在我更改了以下代码后:
for row in xrange(rows - 2): self.neurons.append([Neuron(parents=self.neurons[-1] + [self.bias]) for output in xrange(outputs)])
改为:
for row in xrange(rows - 2): self.neurons.append([Neuron(parents=self.neurons[-1] + [self.bias]) for output in xrange(hidden)])
速度又回到了与旧代码相似的水平,但至少它生成了正确大小的ANN。