Lead
Artificial intelligence (AI) technology has been gaining momentum in recent years and is playing an important role in many fields, but most of these neural networks that support AI tools are based on the neuronal computational models of the 60s of the 20th century, and it is through a large number of simplified computational models that they have paved the way for the development of today's AI systems.
But new models developed by the Center for Computational Neuroscience (CCN) at the Simmons Foundation's Flatiron Institute in the United States show that these approximate models do not fully capture the computational power of real neurons.
Researchers at the Simmons Foundation's Flatiron Institute say that while the new models they proposed are indeed more sophisticated, their new findings also provide a new way to learn more about how neurons perform cognitive functions that computation and process information, and the findings were published in the Proceedings of the National Academy of Sciences.
The old model is likely to become an obstacle to artificial intelligence, so in what ways does the new neuron computing model improve the old model and may make the AI system more powerful?
Old model.
The old model had only one free parameter in the form of mathematical equations, which meant that both the construction of neurons and the working principle of neurons were only greatly weakened, but it was a necessary choice at the time, and it was a choice that researchers made in order to make further simulations to help scientists understand the evolution and development of the brain at that time.
At the same time, this has become a burden today, this kind of living neurons can achieve very complex calculations, and a large number of details in the calculation process can not be simplified to the addition of unit modules to calculate, but the electrical signals on each next layer of neurons as input to participate in the calculation of neurons, that is to say, the relationship between the electronic signals between neurons is intricate, but in the old model this complex relationship has been simplified into a linear relationship, This brings the new model closer to how living neurons are calculated.
The old model is mathematically formulated as: S = g(w^Tx-b), which means that if the linear combination of the input signal x and the weight w minus the threshold b is greater than 0, then the output S of the neuron is activated, otherwise it is inactive.
A researcher called Gabor argues that w is not a parameter, because w is used to weight the activation level of the input cell.
In the further study of the old model, it was found that the dot product of w and x is the activation degree of the input unit, and b is a parameter used to adjust the activation level of the current neuron, so the dot product of w and x and subtract b The result is the same as the output S in the model.
Therefore, there is no parameter called w, but the relationship between the dot product of x and b minus the value of b and the input activation degree x and threshold b intuitively has some bias, although the parameter w in the old model has a certain substantive meaning, but it does not directly correspond to the activation degree and threshold of the input signal like x and b, so in the new model, the researchers believe that the two parameters should be separated, one parameter is used to represent the activation degree of the input signal x, and the other parameter is used to represent the threshold b. This is more in line with the original intent of the old model.
The mathematical formula of the new model is: S=g(x-b), it can be found that in the new model, x and b are truly understandable, x represents the activation degree of the input neuron, and b represents the threshold of the current neuron, so the new model can represent the relationship between the input signal and the threshold more accurately than the old model.
Why the new model is better
In the old model, researchers think that w is not a parameter, which is not wrong intuitively, but in reality, the dot product of w and x is not a real parameter, but an important indicator of x, and w can only be calculated as a coefficient of this indicator, not a parameter with real meaning, so in fact, w does not really reflect the degree of activation of neurons.
So the researchers are wrong to think that w is not a parameter, in fact w is a parameter, but it is not reflected, and in the new model, the researchers split x and w to represent the degree of activation of the input neuron and the coefficient between them, respectively, which is more in line with the actual meaning of the old model.
At the same time, although the parameter w in the old model is not a real parameter, the dot product between them can be used as an indicator of the degree of input activation, but in the new model, this indicator and x are duplicated, so the two parameters can be combined into one parameter.
This will make the model look more concise, but in order to make the new model and the old model more one-to-one, without changing the parameters, there is a transformation relationship between the two, and the two parameters cannot be combined.
Another parameter that cannot be combined between the old and new models is the threshold b, but in fact, this parameter is also a coefficient in the old model, which does not reflect the accurate meaning of b in the mathematical formula, so the new and old models are completely corresponding and mutually transformed.
It can be said that in fact, the old model is more practical, but it is not the case in mathematics, so the neuronal computing function shown in the new and old models is actually the same, but there are some changes in the mathematical formula, but in fact, the new model is more complex and can fully express this computing function, so the new model will be more applied to the AI system.
epilogue
The neuronal computing function expressed by the old and new models is the same, but the new model is relatively new, and from the new model, we can have a deeper understanding of the interaction between neurons, and we can better discover the mysteries, so as to make a more powerful AI system.
At the same time, the enlightenment gained from the old and new models can help us better understand the working patterns of the brain, and better discover the hidden functions in the depths of the brain, so as to point the way to the realization of more powerful AI systems.