Appendix 2

Automata Networks

    The master network is a large, intricately interconnected network of programs. In this context, it is interesting to ask: what, in general, is known about the behavior of large interconnected networks of programs? The most important work in this area is Stuart Kauffmann's (1988) theory of random automata networks, which was initiated in the late 1960's. Kauffmann's method is very simple: construct networks of N random Boolean automata (i.e., programs), connect them to each other, and let them run for thousands of iterations. His results, however, are striking. Let K denote the number of other automata to which each automaton is connected.

    One of Kauffman's first questions was: what is the long-term dynamical behavior of the networks? Since the networks are finite, for any given initial condition each one must eventually converge to a stable state or lock into a repetitive "limit cycle" (say, from state 1 to state 2 to state 3, and then back to state 1). The question is, how many different limit cycles are there, and how many distinct states are involved in each cycle? Where k=1, the networks operate in a largely independent, well-understood manner. Where k=N, the case of complete connectivity, the problem is also analytically soluble. For intermediate k, there is as yet no mathematical answer to Kauffmann's question -- but in every case, the numerical results are extremely clear. The total number of attractors (steady states or limit cycles) is roughly proportional to the number of automata in the network. But the average period of a cycle is exponential in the number of automata. Thus an arbitrary automata network, supplied an arbitrary initial condition, will almost certainly display "pseudo-chaotic" behavior -- it will eventually lock into a cycle of states that is so long as not to be humanly recognizable as a cycle.

    This tells us little about the behavior of highly structured automata networks like the master network. But Kauffmann's more recent work (1988) tells us a bit more. He has set up a network of automata according to a certain scheme called the "NK model". Each automaton seeks to optimize its state according to certain criteria, and the criteria used by each automaton depend in a certain way upon the states of certain other automata. His model is very specific, but philosophically, it represents a very common situation: a network of programseach optimizing some function of an environment which is continually changed by the action of each program. His network is much simpler than the master network, but there are certain qualitative similarities: it is much closer to the master network than is a random automata network.

    Kauffman's main result regarding the NK model is that the interdependence of criteria makes a tremendous qualitative difference. If the criteria for each optimizing automaton were independent of the other automata, then as the optimization problems became more and more complex, the approximate optima found would be further and further from the true optimum. And this is true in the interdependent case, if the number of other automata to which each automaton is connected is increased proportionally to the complexity the problem. But if the complexity of the problem is increased while the connectivity is kept much smaller, the network tends to settle into a state in which each automaton can solve its problem relatively well.

    I am being vague about the exact nature of Kauffmann's NK model because it is not relevant here. The model is tied to evolutionary biology rather than psychology or neuroscience. But similar results have been obtained from models of the immune system (Kauffmann et al, 1985); and it is only a matter of time until someone attempts a similar model of the simpler aspects of neurodynamics. What interests us here is Kauffman's general conclusion:

        [I]t is a reasonable bet that low connectivity... is a sufficient and perhaps necessary feature in complex systems which must be perfected in the face of conflicting constraints... which are themselves complex adapting entities with many parts and processes which must mesh well to meet their external worlds.... [L]ow connectivity may be a critical feature of coadaptation of entities....

The mind and brain are clearly systems of the sort Kauffmann describes.

    The nAMP meets the criterion of low connectivity, because its structure is based on hierarchy, and inherently low-connectivity inter-neural-cluster connections. And, in general, the master network meets the criterion of low connectivity: in the STRAM each memory item is directly connected to only a small proportion of others; in the perceptual and motor hierarchies each processor is connected to only a few processors on the same or immediately adjacent levels; etc.

    Admittedly, the observation that the master network has low connectivity is not particularly exciting or profound. If it did not have low connectivity, it could not feasibly be constructed. But it is enchanting to think that it might somehow be possible to use the general theory of automata networks to deduce more interesting properties of the network of programs that is the brain, or the network of programs that is the mind. This would be the exact opposite of the psychological approach to mind. Instead of explaining particular behaviors with highly specific ad hoc theories, one would be analyzing the overall structure and dynamics of intelligence from a perspective inclusive of all complex self-organizing systems: minds, brains, ecosystems, immune systems, bodies,....