Three themes

Three Themes

Basically, there are three key ideas that keep reappearing throughout the book. Each idea is relatively simple in principle, but all three must be presented within the context of the entire book in order to be appreciated since all of them make some reference to the contents of the five book parts: computation, fractals, chaos, complex systems, and adaptation.

Taken in order, these three ideas roughly build on one another, so it helps to consider them in the following order:

Theme #1

The whole is greater than the sum of the parts

By this I mean that the topics of computation, fractals, chaos, complex systems, and adaptation are far more interesting considered together than by themselves. Each of the topics is related to the others in a non-trivial way; moreover, each can be seen as the result of a few simple principles such as recursion, parallelism, and nonlinearity.

Theme #2

The interesting stuff is in the middle

“Beauty,” i.e., that which makes something interesting, is related to a mixture of regularity and irregularity. When things are too regular, we usually find them to be uninteresting because they yield no surprises for us. Complementary to this, highly irregular things are often uninteresting because they make no sense. In the middle, between regularity and irregularity, lies a place where things can be understood, but not completely.

Theme #3

Science is doomed to uncertainty---but this is a good thing

The interesting things mentioned above are often found to have computationally profound qualities. For instance, even if you had a perfect model or theory of how something worked, chances are that it would still be impossible to perfectly predict the future of that which you have modeled. A related result guarantees just the opposite: regardless of how much “data” one collects, it's not always possible to build a perfect theory. Taken together, these two results insure that there will never be an end to science, or suprises.

1. The whole is greater than the sum of the parts

Let's examine this idea further. Starting from a computational framework, fractals are special “programs” that build self-similar structures.

Computation

structural ---> self-similarity

Fractals

The program in this case often involves a form of recursion that specifies how one part of a fractal looks like another portion of it (or of the whole fractal). Other times, as in the case of the Mandelbrot set and in Julia sets, structural self-similarity appears almost as a side-effect because the compact rules that describe such objects do not explicitly contain specifications for how things should look.

In either case, when one allows the rules of a fractal to be expressed ad infinitum strange mathematical properties can be found in the structural forms of the fractal. In the most extreme cases, fractals such as the Mandelbrot set can be seen as mathematical statements that are similar in complexity to incomputable problems found in computer science.

Chaotic systems are similar to fractals but contain functional self-similarity that occurs at different scales.

Fractals

functional ---> self-similarity

Chaos

More specifically, while we could visually confirm that a portion of a fractal is similar to itself, it is also possible to see how a chaotic system is functionally self-similar. Often times, this functional self-similarity shows itself as a characteristic behavior that occurs on multiple time and/or spatial scales. This is why a strange attractor (which is a characterization of a chaotic system's behavior) will usually have a fractal shape to it.

Chaos, which depends on nonlinearity, is usually studied in systems that can be described with only a few equations. However, another type of chaos is found in large collections that operate in parallel. Examples of this latter type include ecosystems, economies, and chemical soups. We refer to such collections as complex systems because even though a single agent in the collection can be understood (e.g., a predator, stock trader, or reactant) the composite behavior of the system is far more complicated.

Chaos

multiplicity ---> & parallelism

Complex Systems

Hence, by adding multiplicity and parallelism to collections of simple nonlinear agents, it is possible to produce behavior that is not only chaotic, but computationally profound as well.

One of the more interesting facts concerning complex systems is that sophisticated behavior can be found even when the interactions among the system's agents are constrained to be local. Sometimes the details of how the interaction occur can be subtly modified, as in the case when a synapse that connects two neurons is allowed to strengthen or weaken.

Complex Systems

feedback --->

Adaptation

Since agents in a complex system are all indirectly coupled to one another, it is possible for a sophisticated form of feedback to occur that allows the agents (and the system as a whole) to adapt in a seemingly intelligent manner. While an obvious form of adaptation through feedback is found in the brain, more subtle forms can be found in market economies, immune systems, and through evolution.

Since adaptation is ultimately a form of modeling (where events external to the adapted system are mirrored internally), adaptive systems that alter their environment are indirectly altering themselves.

Adaptation

self- ---> reference

Computation

This means that some adaptive systems can be self-referential (or recursive), which is a requirement for the ability to perform complex computations. Adaptive systems that contain such self-reference (such as recurrent neural networks) are capable of universal computation.

Having come full-circle in seeing how all of these topics relate in a sequential manner, it is now worthwhile to see what some of the cross-connections are between the five topics. The table below contains a list of subjects that relate to two or more of the book topics.

Computation Fractals Chaos Complex Systems Adaptation Computation - recursion incomputability cellular automata coadaptation Fractals recursion - strange attractors growth models hierarchical models Chaos incomputability strange attractors - phase transitions emergence Complex Systems cellular automata growth models phase transitions - self-organization Adaptation coadaptation hierarchical models emergence self-organization -

As can be seen, there are many subjects that are clearly related to more than one book part. By viewing all of these topics in a multidisciplinary manner, one can appreciate how the topics of computation, fractals, chaos, complex system, and adaptation are far more interesting when considered together.

2. The interesting stuff is in the middle

This idea is best illustrated with an example that appears in the book (paraphrased from Section 22.6, “Internal Representations”).

Consider the task of finding a good movie to see. You have at your disposal several movie reviews and some prior knowledge concerning the relative tastes of the individual movie reviewers. Some reviewers seem to like everything ever put on film, so what they say about any movie is useless, in that it tells you nothing at all about whether or not you will like a movie. Other movie reviewers have such complex tastes that you may be hard pressed to ever guess what kind of movie they would like or dislike. However, if you find that your taste in science fiction films is similar to one particular critic's, or that your preferences for comedy films are exactly opposite to those of another critic, then these are both useful pieces of information for you to use when picking a movie.

The key point here is that the usefulness of a movie review depends on the relative complexity of the critic: if he or she is either too simplistic or too complex, then the information they provide is negligible.

We can extend the analogy to partially describe what makes a movie entertaining. Both predictable and random plots or boring. But something in between, that contains both familiarity and surprise, makes a movie fun to watch.

Still not convinced? Then consider music. Very few people enjoy listening to simple scales, no matter how precisely they are played. Nor do people enjoy listening to random beeps. Interestingly, analysis of classical music often reveals a fractal representation for the notes, which gets to the heart of the idea that “beautiful” things contain a mixture of regularity and irregularity.

This basic idea, that neat things are at the midpoint between computability (simplicity) and incomputability (randomness) can be extended to just about every conceivable domain. The table below (adapted from a table in Chapter 24) is a partial listing of a broad range of things that seem to fit with this idea.

Computable Partially Computable Incomputable Sets recursive RE and CO-RE not RE & not CO-RE Numbers rational computable irrational incomputable Programs trivially (never) halt possibly halt --- Proofs true or false profound statements unprovable NP-Complete Problems underconstrained critically constrained overconstrained Deterministic Geometry Euclidean deterministic fractal --- Stochastic Geometry --- stochastic fractal pure noise AC compressible possibly incompressible incompressible Mandelbrot Set white regions border regions black regions Continuous Dynamics fixed point chaotic high-dimensional chaos or stochastic Discrete Dynamics regular from over-sampling complex at mid-sampling irregular from under-sampling Attractors integral dimensions strange infinite dimensional Mater solid liquid gas Wolfram CA class I or II class IV class III Langton's lambda lambda < 1/3 lambda near 1/2 lambda > 2/3 Agent Interactions globally coordinated or always cooperative locally coordinated or cooperative and cooperative uncoordinated or always competitive NK Nets K = 1 K = 2 2 < K <= N Sandpiles flat and stable critical tall and unstable Economics communism free but regulated unrestrained Governments dictatorial democracy anarchy Patterns consistent hidden order inconsistent Models not self-referential coadaptive or recurrent hopelessly self-referential Search Methods local or greedy hybrid exhaustive Search Spaces smooth complex structured pathological

Many of the topics listed in the table may not be familiar to you, but most of them are covered in considerable detail in the book. In particular, each book part is concluded with a “Postscript” chapter that reviews all of the topics covered in a part as they relate to the ideas from the theory of computation. In this way, it is possible to see how all of these diverse areas relate to the fundamental region between computability and incomputability.

3. Science is doomed to uncertainty---but this is a good thing

The bottom line here is that whenever something is so rich as to be related to something akin to a universal computer, it is impossible (in the general case) to completely understand the underlying phenomenon. This has deep implications for how we do science. Consider the figure below (from Chapter 24).

In the past, there were two different methods for how science was performed. In one method, an experimentalist fiddles with some natural phenomenon so that data can be collected in the form of observations. These data are then compared with the current state of knowledge.

Coming from the opposite direction, a theorist derives a model for how a natural process works. The new model is then used to make predictions for how the natural process will behave at a future time.

With the advent of computers, a new and hybrid method of science has emerged. This new type of scientist, known as a simulationist, simultaneously performs experimentation in a universe of theories by simulating natural phenomena within the confines of a computer.

Neither of these methods of performing science is intrinsically superior to the others. Each of them are, in a sense, incomplete. But simulation allows for a type of exploration that neither experimentation nor theorization can provide. Pure theory fails when a phenomenon does not obey models that can be analytically solved. Pure experimentation fails when complex effects cannot be correlated to simple causes. But simulation hovers about these two extremes, allowing for a new type of science that mirrors (if only approximately) the complexity and beauty that we see in the real world.

The good news here is that there will never be an end to science. For science to end, one of two things must happen: (1) we would need a Theory of Everything that could be used to predict and explain all things, or (2) we would reach a clear and identifiable point at which no further progress could be made. But because nature's most beautiful phenomenon contain computational richness, there will always be surprises and new discoveries just beyond our reach.