Showing posts with label evo-contingencies. Show all posts
Showing posts with label evo-contingencies. Show all posts

March 9, 2015

Review of "Arrival of the Fittest"

"Arrival of the Fittest" is the latest book by Andreas Wagner, a professor at the University of Zurich in Switzerland. The book [1] tackles a subtly complicated topic: the evolution and evolvability of innovations using a biochemical and computational perspective. For the most part, Wagner succeeds at presenting an elegant case for how the ability to naturally evolve innovations lies at the heart of the evolutionary process. To get there, however, Wagner must introduce us to a number of semi-obscure concepts (at least to the layman). The book can be summarized in four parts: survival of the most novel (I), the concept of innovability (II), the safe and the risky (III), and multiple origins, multiple solutions (IV). I will give a technical review of the book by highlighting these four interrelated themes.



I. Survival of the most novel.
To understand the propagation of innovative solutions throughout the tree of life, it is important to understand the difference between the force of evolution versus the role of selection. Wagner proposes that, rather than creating innovations, the role of natural selection is to preserve them. Innovations themselves are the result mutation, recombination and historical contingencies. The first two mechanisms are capable of randomly generating either very simple innovations or the components of more complex innovations. But to achieve the "tinkering" that seems to be prevalent in complex genetic pathways and phenotypes, we need to have standard meta-components that can lock in previous changes. This allows for the limited exploration of a fitness space without incurring the cost of losing previous advances.


Wagner teases historical contingencies this into two classes: building blocks and standards. In the case of building blocks, the working components that result from previous innovation are modularized into larger units. This allows for increasing complexity to be had from a stochastic process (evolution) as well as accelerating the process of finding and retaining novelties (innovation). The coordination of building blocks often leads to standards, which are much more flexible than hyper-specialized systems but are much more able to deal with hyper-complexity.

Building blocks assembled into a complex structure.

Wagner illustrates this by comparing metabolic engines (a high-tolerance biological engine) with the internal combustion engine (a high-precision mechanical innovation). While metabolic engines can use an interchangeable set of fuels and reactants, internal combustion engines require specific specifications to operate. While one might argue that this implies mechanical engines will be "optimized" and metabolic engines will be made "good enough", this is indeed the point. Rather than survival of the fittest, we observe a survival of the fit enough, with selection acting strongest on functional novelties that augment fitness.

A high-tolerance but highly-specific engine (internal combustion) type.

II. Concept of innovability.
In the case of complex metabolic networks, the basic function has not changed throughout evolutionary history. What has changed is the number of reactions, which scales with evolutionary complexity. This scaling requires very general standards. But to make these standards interoperable across divergent evolution, a diversity of building blocks regulatory mechanisms are also required. This leads us to a set of principles which can explain general trends in innovation rather than on a case-by-case basis. Another such principle involves the number of parts and identity of a specific innovation. The number of interacting parts and their configuration provides a means to compare specific innovations in a hypothetical manner. Wagner proposes that this be done using a high-dimensional structure such as a hypercube [2] or a neutral network [3].

In a hypercube representation, each node represents a specific genotype, while the edges represent pathways (or the accumulation of mutations) between genotypes. In short, the shortest paths are the most probable. In a highly-connected space, a long distance can be traveled across the space in a short number of mutations (or edges).

III. The safe and the risky.
How can the structure of a biological system make things safe for innovation? Certainly, blindly changing key components of a metabolic network or developmental scaffolding without regard for essential function can result in lethality. The presence of the building blocks and standards principles provides a failsafe means to tinker or change without lethal disruption. But there are two other principles at work: redundancy and connectivity. These principles are arguably more important in acting as gatekeepers for the fitness benefits reaped by innovations.

A machine finding its way through a maze. One way to search for novel solutions in a de novo fashion.

Redundancy involves the presence of multiple parts that play a similar or interchangeable role in the functioning of a system. For example, both genetic and metabolic networks can be robust to the removal of single components. When components get removed without having a consequence on function [4], we can say that such a component is redundant. Gene duplications can play a similar role: single copies can be knocked out without a large detriment to fitness. Related to redundant function is, of course, robustness. Robustness can be thought of as how much redundancy exists in a specific system. Wagner gives the example of phenocopying as a way in which developmental robustness can lead to redundant phenotypic configurations.

Connectivity is a less appreciated aspect of evolution. Yet in terms of functional and conceptual unity, connectivity is an essential component of evolving systems that produce innovation. Three aspects of connectivity are most important here: interaction type, density, and connection order. There are a number of interaction types in biological networks that can give rise to innovation. For example, we can look to genetic networks (interaction between genes), breeding networks (interactions between conspecifics), or even the aforementioned neutral networks (interactions between evolutionary configurations) for ways in which connectivity can yield pathways that lead to innovations of high fitness.

An example of a genetic (gene interaction) network. Sometimes this approach is called "hairball science". COURTESY: Figure 1 in [5].

Network density (or the number of interconnections between nodes) allows for a greater number of potential innovative solutions to be explored in a shorter number of steps. This can increase the number of potential solutions, which can be reached in a shorter number of evolutionary steps and/or time (depending on your perspective). In the neutral network, a number of "safe" pathways of equivalent fitness are created for the innovating pathway or organism to explore. While the network density is important in facilitating innovation, the connection order of the network is also important. In networks with a short connection order (e.g. small-world networks), even random paths through the network can yield large-scale, non-deleterious change.

An example of constituent genotypes in a neutral network with reference to robustness. COURTESY: Ricardo Azevedo, Wikipedia.

IV. Multiple origins, multiple solutions.
The final point of Wagner's book is to remind us that innovations can be achieved through multiple unique solutions, and that existing innovations may have had multiple origin points. While evolution also has no unique solution, innovation is particularly subject to parallel evolution. Since this book does not shy away from computational representations, Wagner applies the idea of evolution strategies [6] to illustrate how innovations can be modeled as a five-part process. Specifically, innovation involves trial-and-error, population-based exploration, solutions with multiple origins, a combinatorial structure, and a stochastic process with a mutation-selection structure. While these factors also define the evolutionary process, we might say that innovation is inseparable from evolution by natural selection. Thinking more broadly and considering social systems, cultural innovation might also be best characterized as an evolutionary process. Wagner's approach to innovation as evolvability is a useful and accessible introduction to the topic, and places this new set of concepts and terminology directly into the context of evolution by natural selection.

NOTES:
[1] For other, briefer reviews, please see: Hoppe, R.B.   Andreas Wagner: Arrival of the Fittest: Solving Evolution’s Greatest Puzzle. Panda's Thumb blog, November 4 (2014) AND Pagel, M.   The Neighborly Nature of Evolution. Nature, 514, 34 (2014).

[2] For an example, please see: Gavrilets, S. and Gravner, J.   Percolation on the Fitness Hypercube and the Evolution of Reproductive Isolation. Journal of Theoretical Biology, 184(1), 51–64 (1997).

[3] For an example, please see: Wagner, A.   Robustness and Evolvability in Living Systems. Princeton University Press (2005).

[4] Li, J., Yuan, Z., and Zhang, Z.   The Cellular Robustness by Genetic Redundancy in Budding Yeast. PLoS Genetics, 6(11), e1001187 (2010).

[5] Magtanong, L. et.al   Dosage suppression genetic interaction networks enhance functional wiring diagrams of the cell. Nature Biotechnology, 29, 505-511 (2011).

[6] Evolution strategies assumes that evolution is an optimizing process that results from the iterative application of a mutation-selection model. For more, please see: Schwefel, H-P.   Numerical optimazaion of computer models. Wiley Press, Chichester (1981) AND Beyer, H-G. and Schwefel, H-P.   Evolution Strategies: A Comprehensive Introduction. Journal of Natural Computing, 1(1), 3–52 (2002).

December 25, 2014

Does the Concept of "Paradigm Shift" Need a Rethink?

A faux relationship between the paradigm shift and the theoretical resynthesis. Although in terms of advancing theory, perhaps they indeed do arise from a common ancestor.

As someone who is interested in both evolutionary and "meta-" theory I read a recent comment paper in Nature [1] called "Does Evolutionary Theory Need a Rethink?" with great interest. In fact, I discussed this paper a bit in a Synthetic Daisies post from last month. There are some interesting issues here regarding the role of "extended evolutionary synthesis" ideas in making evolutionary inferences and predictions. However, the real issue here is whether theory best proceeds through soft "paradigm shifts" (in this case, extending the framework) or through "resynthesis" (in this case, relentless synthesis). As an emerging approach to evolutionary theory, the extended evolutionary synthesis includes ideas not typically embraced by the modern evolutionary synthesis [2, 3]. These might include developmental plasticity, evolvability, epigenetic phenomena, genetic assimilation, and cultural evolution. The primary argument is not just that evolution involves more than just changes in allele frequencies over time, but that such mechanisms should take a more central role in the process of evolutionary change [4].

The landscape of how evolutionary theory might be rethought. WHITE: Darwin's core contributions, LIGHT GRAY: modern evolutionary synthesis, DARK GRAY: extended evolutionary synthesis. Notice that this diagram implicitly favors the addition of rather than a shift towards new topical areas (e.g. resynthesis over paradigm shift). COURTESY: Figure 1.1 in [2].

But why do we need to rethink theories anyway? The classic observation of theoretical change comes from Thomas Kuhn [5], who advocated the dual concepts of "theoretical paradigms" and "paradigm shifts". In the evolution of a given scientific field, many new findings and concepts are introduced over time. Yet there is also an so-called essential tension between traditional and upstart concepts. Only very occasionally, a set of findings or concepts takes root that sweeps away the prevailing worldview. Paradigm shifts are thus low-frequency events that are nonetheless transformative in the way people think about a given scientific field. As events in intellectual history, paradigm shifts can often be a neccessary progression in the history of a scientific field. This is due to both the integrative nature of theory itself and conceptual inertia from the scientific establishment.

Although less appreciated by Kuhn, the integrative nature of theory thus serves to act as a form of conceptual inertia. In terms of theoretical evolution, incremental changes are hard to come by as singular findings and propositions do not often stand on their own. To really understand what is going on, the incremental progress of empirical science must coalesce into an intellectually coherent framework. According to paradigm shifters, prevailing theoretical models tend to be established through what is often called "saltationist" or non-gradualist change. Yet I would argue that whether such advances occur through paradigm shift or through resynthesis requires an underlying set of favorable conditions in the existing literature.

Yet perhaps the predominance of paradigm shifts throughout the history of science is largely based on assumption. Earlier, I mentioned a tension between "soft paradigm shifts" and "resynthesis". It may seem that paradigm shifts are neccessary in order to enable the novel insights in understanding. However, suppose that instead of acting as a neccessity for theoretical change, the paradigm shift served as a bias for those who would build theory itself. This might explain why all too often there is an expectation that new ideas either paradigm shift a field or languish insignificantly.

As an alternative to the paradigm shift, theoretical resynthesis allows for additional information to be incorporated into an existing theoretical framework. While more conservative, it may be no less transformative. In Darwin's original formulation of evolution by natural selection, the concept of heredity was without a formal mechanism [6]. The modern evolutionary synthesis was formulated in part to reconcile the ideas of evolution by natural selection and heredity by independent assortment. While a paradigm shift might give us a sorely needed new perspective, it can also live up to the idiom of throwing out the baby with the bathwater. It is worth noting that while it may be possible to achieve an extended evolutionary synthesis through theoretical resynthesis, the tone of contemporary arguments for an extended synthesis (e.g. the Altenberg 16) tend to be biased towards a soft paradigm shift.

An example of a directed conceptual network, in this case featuring the intellectual evolution (1940s-present) of cybernetics and systems science. As we can see, there are several distinct intellectual traditions that cross-fertilize the field to various degrees and at various points in time. COURTESY: Castellani, Wikimedia Commons.

To answer the question of whether or not evolutionary theory needs a rethink, a literature mining exercise might be helpful [7]. This type of approach would allow us to characterize to what extent extended synthesis concepts are being considered alongside modern evolutionary synthesis concepts and vice versa in the same context. This can be characterized using a conceptual network of empirical studies. In our conceptual network topology, the overall connectivity of (e.g. linkages between) various concepts would represent their relative conceptual integration in empirical studies and literature reviews, which in turn provides a basis for theoretical advances. Think of such pre-existing linkages as the histroical contingencies of theoretical change. This is not typically considered in the paradigm shift model, but has consequences for resynthesis and paradigm shifts alike.

To illustrate how this approach might be useful, I will give two examples from the contemporary biological literature. For example, how often does a published paper consider population genetics alongside evo-devo? Alternatively, how often does cultural evolution get characterized as part of an integrated evolutionary process? In the case of the former, relatively few studies seem to sufficiently bridge population genetics and evo-devo [8]. In the case of the latter, there are only but a few established approach for using the mathematics of population genetics to characterize both genetical and cultural evolution in the same framework [9].

In terms of a network topology, each of these examples would represent sparsely connected and a bit more densely connected concepts, respectively. This also illustrates the difference between the need for a paradigm shift and room for accomodation via resynthesis. On the other hand, perhaps the state of the literature serves to predict which outcome is more or less likely. Even in cases like the cultural evolution example, examples of resynthesis might not be representative of the dominant approach.


NOTES:
[1] Laland, K., Uller, T., Feldman, M., Sterelny, K., Muller, G.B., Moczek, A., Jablonka, E., Odling-Smee, J., Wray, G.A., Hoekstra, H.E., Futuyma, D.J., Lenski, R.E., Mackay, T.F.C., Schluter, D., and Strassmann, J.E.   Does evolutionary theory need a rethink? Nature, October 8 (2014).

[2] Pigliucci, M. and Muller, G.B.   Evolution: the extended synthesis. MIT Press, Cambridge, MA (2010).

[3] Muller, G.B.   Evo-devo: extending the evolutionary synthesis. Nature Reviews Genetics, 8(12), 943-949 (2007).

[4] Lamb, M.J. and Jablonka, E.   Evolution in Four Dimensions. MIT Press, Cambridge, MA (2006).

[5] Kuhn, T.S.   The Structure of Scientific Revolutions. University of Chicago Press, Chicago (1962).

[6] For more, please see: Charlesworth, B. and Charlesworth, D.   Darwin and Genetics. Genetics, 183(3), 757-766 (2009) AND West-Eberhard, M.J.   Toward a modern revival of Darwin's theory of evolutionary novelty. Philosophy of Science, 75(5), 899–908 (2008).

[7] I am merely offering the suggestion for a full-scale analysis rather than providing one.

[8] Based on a cursory survey of PubMed entries for the terms "evo-devo" + "population genetics" (21 results). Also:

a) Two reviews that provide a verbal analysis of evo-devo's theoretical underpinnings (circa early 2000's) can be found here: Arthur, W.   The emerging conceptual framework of evolutionary developmental biology. Nature, 415, 757-764 (2002) AND Gilbert, S.F.   The morphogenesis of evolutionary developmental biology. The International Journal of Developmental Biology, 47, 467-477 (2003).

b) There is also a burgeoning but small field called "micro evo-devo". For more, see: Nunes, M.D.S., Arif, S., Schlotterer, C., and McGregor, A.P.   A Perspective on Micro-Evo-Devo: Progress and Potential. Genetics, 195, 625-634 (2013).

[9] The approaches established by Boyd and Richerson (Culture and the Evolutionary Process) and Cavalli-Sforza and Feldman (Cultural Transmission and Evolution: a quantitative approach) are illustrative of such resyntheses. However, other models of culture are less integrative, and one might argue that these examples are not the most complete or parsimonious way to integrate of biology and culture.

June 27, 2014

Historical Contingencies at the Birthday Party

Historical contingencies are perhaps the most interesting outcomes of the evolutionary process. Stephen J. Gould spent a lot of time and energy making this idea popular, but evidence comes from both paleontology [1], extant populations [2], and experimental evolution [3]. However, the ubiquity of the contingency concept does not resolve its phylogenetic consequences. Is historical contingency highly specific (a hard constraint resulting in unique paths), or is it a softer constraint? And how can we understand the role of convergent evolution within this framework? We will approach this from a mathematical perspective, and answer the riddle of what evolution and birthdays have in common.


Definition of generative science (Wikipedia) and historical science (RationalWiki). 

Evolutionary Histories and Their Accidents
Like human history, evolutionary history is a product of many forces and causes. We often think of these factors as a series of chance events (sometimes unique) that lead to a given outcome [5]. Observers sometimes use this point to argue that history is not systematic and thus cannot be separated from context (and thus comparative history would be quite impossible) [5]. But this also assumes that the factors that make a given evolutionary history unique (its branching events) are "hard". Not only are they irreversible, but also should not have significant similarities. The outcomes of the evolutionary process (genotypes and phenotypes) are locked in to a specific trajectory. By itself, this constraint should favor some changes over others and disallow changes that resemble even closely-related lineages.

If historical contingency is a hard constraint, then this leads us to an evolutionary hypothesis: historical contingency creates irreversible paths to highly-unique phenotypes. While a bit simplistic, this nonetheless serves to understand the consequences of contingency. Recall that the evolutionary process occurs through branching, and results in a series of evolutionary outcomes (Figure 1). While these outcomes are individually different, their degree of uniqueness relies on the "hardness" of the branch that separates one outcome from another. Figure 1 not only shows the results of branching, but also assumes "hard" constraints. The contingencies generated by this model involve hard constraints that results in a unique, lineage-specific partition of the search space.

Figure 1. An example of a phylogeny with unique, non-recurrent evolutionary outcomes. The evolutionary changes act as hard constraints, and each terminal taxon occupies a distinct 1-dimensional subspace. 

In Figure 1, a conventional phylogenetic model demonstrates how a search space can be partitioned through evolutionary branching processes. However, when the constraints are softer, each branching event results in less distinction between the resulting alternative forms and increases the chances that traits or forms that resemble those of a related lineage (even distantly so) will emerge. Figure 2 demonstrates this difference using the analogy of the Plinko game [6]. In this case, the combination of the process and outcome of contingency creates an overlapping search space over time for a given lineage (see the distribution of Plinko balls at the bottom of Figure 2). 

Contingency also rests on the assumption that evolutionary randomness results in unique combinations of traits. One feature of historical contingency involves building upon previously-acquired traits. As complexity is built in this way, the total number of possibilities decreases. But while the stochastic nature of evolution is a matter of conditioned chance, branching is an assumption of theoretical intuition. Therefore, evolutionary outcomes can converge even when their forms nominally exist in different lineages. But given these constraints, shouldn't convergent evolution be impossible? Before we answer the question (and the answer is no) we must take an intellectual detour by way of birthday parties.

Figure 2. What it means to have an overlapping space of evolutionary outcomes enabled by soft historical constraints. COURTESY: Plinko Probability, version 2.02. PhET Interactive Simulations.


How are birthday parties at all relevant here? The birthday party paradox, a statistical curiosity, might help us establish a link between contingency and recurrence. But first, let us revisit our evolutionary process-as-hierarchical tree model. In this model, all possible combinations of n are classified using a tree-like structure. Given that the search space is much larger than the number of objects being classified, do they also end up in unique categories? Perhaps. But, as we will learn, it may not matter as much as does the size and complexity of the evolutionary landscape itself.

What is the Birthday Party paradox [7]? Amazingly, this did not make a Quora list of the most counterintuitive mathematical results [8]. But perhaps this result is not so intuitive after all. Say you were to survey a room of n people. Given that every day of the year has an equal chance of being a birthday, how many people will you need to sample in order to find at least two people with the same birthday? The answer you might give depends on your intuitions about randomness. With 365 days in a typical year, one might assume that you would need a lecture hall of at least 300 people. But in fact, once you reach a sample size of 47, the probability (95%) becomes asymptotic to 100%. See Figure 2 for a graphical representation.


Figure 2. Number of people surveyed (x-axis) vs. probability of at least two people having the same birthday (y-axis).

Evolutionary Histories and Their Coincidents
This outcome results from a mathematical principle called recurrence. This principle suggests that motifs and themes can recur at an unknown frequency -- it explains why you get runs of heads or tails in a series of coin flips. This recurrence has nothing to do with the outcomes being related to one another. They are merely conincidences inherent in a generative process. In the evolutionary outcome space example, this suggest that overlap can occur in the form of deep similarities. Can this be applied to the probability that n lineages will exhibit convergence?

Not exactly what we are talking about here, but an evolutionary birthday nonetheless.

Phylogenetic birthday (or contingency) paradox:
In the next few tables, I have shown how the mathematics and problem formulation of the standard birthday paradox can be used to understand a generative set of evolutionary configuration and the probability of a parallel evolutionary outcome. 

What the data should look like (standard Birthday Party paradox):

 What the data should look like (proposed evolutionary paradox):

In the case of the evolutionary paradox, an exceedingly small sample size of 60 possible configurations was used for demonstration purposes. It is of note that this model is scalable to very large numbers of distinct evolutionary configurations. However, it is clear that the probability of convergent evolution is nearly 100% well before a given lineage is locked in to a single point in the configuration space. As the number of changes increases, the number of possible configurations changes is correspondingly reduced. 

But....but.....there are assumptions!
This model makes a few general assumptions. One, while each change is assumed to be countable, there is no accounting of how hard or soft the constraint actually is. This could be resolved through using a soft classifier to characterize each change, although would not remove the effects of geographically-localized specialization. An example of this is in the supplemental Excel dataset (see Notes section below). Another is that all evolutionary configurations are countable in the same way (e.g. no modularity). Again, this can be resolved by generating a matrix for each component of an organism (e.g. phenotypic module). 

Despite these assumptions (for better or for worse), the general principle of recurrence should give us a somewhat useful model for estimating how plausible or implausible convergent evolution is for a given set of evolutionary relationships. Recurrence is a useful tool that is largely ignored in conventional discussions about evolutionary constraints and parallel evolution. Once again, recurrence (by way of Henri Poincare in Figure 3) allows us to use principles of complexity theory to better understand evolutionary phenomena [9].


Figure 3. An example of Poincare recurrence. In this example, an image of Henri Poincare has been permutated, with reconstruction of the original image (or a reasonable approximation) is reached well before the maximum number of possible combinations is reached.


UPDATE (6/30/2014):
It was pointed out to me by a reader that birthdays have a distribution of their own throughout the course of a year. For example, birthdates in the Summer months (June, July) are more common than those in the winter months. This is of course due to human mating preferences and seasonality (and so birthdays are actually a quasi-stochastic process). Hence, there is a clustering of more common (as opposed to less common) birthdates on the calendar (Figure 4).

Figure 4. Visualization of birthdate frequency (in heatmap form) distributed across the calendar year. COURTESY: VizWiz blog and NYTimes.

I imagine this type of probability density is also somewhat true for evolutionary data across the diversity of a genus, order, or domain. But this type of clustering is also an outcome of stochastic processes (and one reason why recurrence is possible). When sampled at a given point in time, the outcome of a stochastic process is often not uniformly distributed -- in fact, it reveals clusters which must be distinguished from clusters that result from non-random processes. The question would be whether or not birthdates (or confounding evolutionary processes) cluster so significantly as to override clusters that result from randomness. The birthday paradox equations don't explicitly take that into account, but that likely does not invalidate the larger pattern.

UPDATE (8/4/2014):
Here is a good recent article from Nautil.us Magazine on evolutionary contingency. Puts a lot of the contemporary support for the idea in perspective.

Zorich, Z.   If the World Began Again, Would Life as We Know It Exist? Nautil.us, June 19 (2014).


NOTES:
Mathematical notation courtesy Wolfram MathWorld (http://mathworld.wolfram.com). Implemented in Excel courtesy of eXcel eXchange (http://excelexchange.com). Excel workbook (computed using pseudo-data) located on Github (https://github.com/balicea/evo-birthdays).

[1] Vermeij, G.J.   Historical contingency and the purported uniqueness of evolutionary innovations. PNAS, 103(6), 1804-1809 (2006).

[2] Taylor, E.B. and McPhail, J.D.   Historical contingency and ecological determinism interact to prime speciation in sticklebacks, Gasterosteus. Proceedings of The Royal Society of London B, 267, 2375-2384 (2000).

[3] Blount, Z.D., Borland, C.Z., and Lenski, R.E.   Historical contingency and the evolution of a key innovation in an experimental population of Escherichia coli. PNAS, 105(23), 7899-7906 (2008).

[4] Travisano, M., Mongold, J.A., Bennett, A.F., and Lenski, R.E.   Experimental Tests of the Roles of Adaptation, Chance, and History in Evolution. Science, 267, 87-90 (1995).

[5] Fales, E.   Uniqueness and Historical Laws. Philosophy of Science, 47(2), 260-276 (1980).

[6] The Plinko analogy has also been used to describe the epigenetic landscapes of Waddington: Gordon, R. Introduction to differentiation waves Part 2. The evo-devo of epigenetic landscapes as differentiation trees. Embryogenesis Explained course (2013).

[7] Fletcher, J.   The Birthday Paradox at the World Cup. BBC News Magazine, June 15 (2014).

[8] Mathematics: what are some of the most counterintuitive mathematical results? Quora, March 27 (2014).

[9] Crutchfield, J., Farmer, J.D., Packard, N.H., and Shaw, R.S.   Chaos. Scientific American, December (1986).

Printfriendly