Theory, Programming, and Simulation of Neural Networks

Abstract

Topics include: Graphical methods and game theory (backtracking, constraint propagation), analytical optimization (multidimensional extremal problems, equilibria, gradient descent),neuronal networks (biological networks, close-to-biology modeling, spin system analogies),evolutionary optimization (genetic algorithms, genetic programming), expert systems (clustering techniques)

Objective

In the introductory part, we use games to introduce the concept of a directed graph. This will provide the paradigm for understanding the different methods that are treated in the lectures. As an application of continuous systems, higher dimensional optimization, Lagrange multipliers, gradient descent and simplex optimization are briefly discussed. Iterated function systems provide an idea of how a complex energy landscape may look like and how it may be generated. In the focus part we begin with the developmentary history and the physiology of biological neuronal networks, which then leads to the biophysically detailed modeling of network elements and their mathematical idealizations on different levels. These elements will then be used to compose networks of neurons. The implementation of the most common neural network types is discussed (perceptron, Kohonen and Hopfield networks) and their efficiency characteristics are evaluated. We demonstrate that by virtue of the same principles, efficient clustering of data can be achieved, and we compare this method with the alternative methods used in the field. As concurrent alternatives to neural networks we finally discuss genetic algorithms and genetic programming. The lectures equally focus on analytical and simulation approaches. All essential aspects of the lectures are illustrated by programs written in the simulation environment Mathematica, for which we provide a short introduction. The lectures provide an understanding of the functioning, potential, limits and salient applications of neural networks and related methods, from both theoretical and practical points of view. The knowledge acquired in the lectures together with the distributed programs will enable the simple, knowlegeable and successful application of these techniques to new problems that arise in all areas of today's science and technology.

Content

Neuronal networks are an important subset of the methods of artificial intelligence. These methods have become increasingly important in the fields that with the more traditional methods of informatics are difficult to tackle, and therefore have been reserved for human intelligence. In addition to being able to replace and to support a human workforce, these methods also provide insight into the structure and methods of human reasoning. The lectures are organized as follows. Introductory topics are: - graphical methods and game theory (backtracking and constraint propagation) - analytical optimization (multidimensional extremal problems, Lagrange multipliers, equilibria, gradient descent) Focus topics are: - neuronal networks (biological networks, close-to-biology modeling, spinsystem analogies) - evolutionary optimization (genetic algorithms and programming) - expert systems (clustering techniques)

Resources