Hybrid logical neuron

If the perceptron recognition machine responds to the elephant drawing with a “mura” signal, the image of a camel is also “mura” and a portrait of a prominent scientist is again “mura”, this does not necessarily mean that it is faulty. She may just be philosophically tuned.
K. Prutkov-engineer. Thought number 30.

Strict logical activation function

By copying the principle of action of a biological neuron when creating artificial neural networks, we don’t really think about the meaning of the activation function of the logical model of a neuron. The function is always recorded as a logical sum, a logical “AND” for a particular set of inputs, and it is the simultaneous activity of these inputs that activates our neuron. If we discard the external meaning of the inputs, we can describe the activation of the neuron as follows. For one external event consisting of a set of incoming images, a specific group of incoming images is combined into a new purely logical image - abstraction. Already for a group of such events activating a neuron, a common set is selected - a generalization. But how abstraction occurs and generalization depends on the rules used for training in our neuron. The closest to reality training for one neuron has always been training without a teacher. But even in this case, we have several guiding principles of self-study. Two extremes and a compromise between them. The first extreme is the statistical finding of the most common group of images. Each time an event occurs, the currently active images are incremented by an internal counter. The second extreme is to find the most frequently repeated pattern of all active images. The currently active images with the lowest internal counter value are enlarged. The trade-off between extremes is obvious.
After the learning stage, the logical function of neuron activation can no longer change during operation, retraining will change the results. The logical functions of neurons constitute the logical framework, by retraining it, we can solve new problems, but the old ones no longer exist. To solve new problems on the basis of a stable logical framework, you need to use existing solutions through analogies, as well as additional training by adding new neurons. The logical activation function does not have the possibility of analogy because of its severity, and there is no point in changing its severity, it will break the logic on which we will rely in the mechanism of analogies. Our neuron still has inputs from its area of ​​responsibility that do not participate in the activation function. We will use them if the activation function has not worked. The analog function will work and learn at the same time, trying to remember the last situation at the inputs of the neuron. Actually the presence of two functions within one neuron makes it a hybrid, and both of these functions can activate a neuron. But the strict logical activation function is primary, and the analogy function is secondary.

Activation function by analogy.

The function of analogies works according to the scale method, depending on which active analogies are more, positive or negative, whether the neuron is activated or silent. You can enter the weight factor for the links of analogies, to regulate the speed of retraining.
The principle of the formation of a positive analogy is that the most important is the primary function of neuron activation, only if it works, there is reason to consider the active connections of analogies as additional factors accompanying the image recognition by the neuron. The value of the analogy is enhanced only when the primary activation function is triggered. Analogy links can even be called the image context recognized by the primary activation function.
The principle of the formation of a negative analogy is that, without a positive result of the primary logical activation function, a neuron with the prevailing number of active positive analogies with time should be silent. But to reduce the weight of the active connection of the analogy is only until the moment of neuron calming down, that is, the reason for reducing the weight of the connection becomes the activation of the neuron is a function of the analogy.

Neural network with logic in time

The logic of the analogy function gives the memorization of the previous event. The result depends not only on the present event, but also on the sequence of the previous ones. At the same time, the activation function makes it impossible to link the sequence of events into a strict logical framework. This can be achieved by shifting the output of the result by a neuron by one cycle. Shifting the output of the result allows you to connect neurons in the network in an arbitrary way, since you can first calculate the functions of all neurons in the network and only then send the results to the outputs without disturbing the logic of signal propagation in the network. Such networks can memorize sequences, analyze information flows.

Finding a new network and adding new neurons

It may happen that the information processed will be completely new to our network. That is, neurons will not be activated not only by the activation function, but even the analogies will be extremely negative. This is the case when we cannot recognize the incoming information and use analogy from existing logical neurons for its recognition. This means that the presence of only active negative analogies is a sign of the discovery of new information that cannot be classified by the existing logical frame of the network. And it is precisely in the place where the greatest number of negative active analogies occurs that a new neuron must be added for learning.