This book has the unique intention of returning the mathematical tools of neural networks to the biological realm of the nervous system, where they originated a few decades ago. It aims to introduce, in a didactic manner, two relatively recent developments in neural network methodology, namely recurrence in the architecture and the use of spiking or integrate-and-fire neurons. In addition, the neuro-anatomical processes of synapse modification during development, training, and memory formation are discussed as realistic bases for weight-adjustment in neural networks. While neural networks have many applications outside biology, where it is irrelevant precisely which architecture and which algorithms are used, it is essential that there is a close relationship between the network's properties and whatever is the case in a neuro-biological phenomenon that is being modelled or simulated in terms of a neural network. A recurrent architecture, the use of spiking neurons and appropriate weight update rules contribute to the plausibility of a neural network in such a case. Therefore, in the first half of this book the foundations are laid for the application of neural networks as models for the various biological phenomena that are treated in the second half of this book. These include various neural network models of sensory and motor control tasks that implement one or several of the requirements for biological plausibility.

The expression 'Neural Networks' refers traditionally to a class of mathematical algorithms that obtain their proper performance while they 'learn' from examples or from experience. As a consequence, they are suitable for performing straightforward and relatively simple tasks like classification, pattern recognition and prediction, as well as more sophisticated tasks like the processing of temporal sequences and the context dependent processing of complex problems. Also, a wide variety of control tasks can be executed by them, and the suggestion is relatively obvious that neural networks perform adequately in such cases because they are thought to mimic the biological nervous system which is also devoted to such tasks. As we shall see, this suggestion is false but does not do any harm as long as it is only the final performance of the algorithm which counts. Neural networks are also used in the modelling of the functioning of (sub systems in) the biological nervous system. It will be clear that in such cases it is certainly not irrelevant how similar their algorithm is to what is precisely going on in the nervous system. Standard artificial neural networks are constructed from 'units' (roughly similar to neurons) that transmit their 'activity' (similar to membrane potentials or to mean firing rates) to other units via 'weight factors' (similar to synaptic coupling efficacies)."