Reconsidering the Model as a Unit of Regulation: cybernetics and the adaptive outcome

Here is a preview of an essay Robert Stone and I have been working on as part of the Orthogonal Research initiative during the course of the last year. The formal title is: "The Foundations of Control and Cognition: The Every Good Regulator Theorem". This essay takes a classical tool from the cybernetics literature and applies it to game theoretic and other problems of our interest.

Robert Stone, cybernetics enthusiast

Robert Stone and myself, bringing cybernetics back to the "soft" but immensely-complex (social, brain, and biological) sciences. The full version (with notes, definitions, and additional references) can be found here.

A seemingly simple discrete system with feedback (which makes it not so simple during future iterations). COURTESY: intgr, Wikimedia commons.

I. Introduction

            In the history of scientific discovery, there have been examples of certain persons or facets of their work being considered ‘out of step’ with the dominant scientific or philosophical trends of the time. As such, they risk falling down a deep well in our cultural landscape, with their work’s efficacy lost to subsequent generations. If their work has merit, it may be considered ahead of its’ time by future generations. The timing of a given theory or great idea is largely determined by cultural and cognitive biases that favor the dominant paradigm [1]. In other cases, ideas at the paradigmatic vanguard end up resurrected in a more pragmatic way. The acceptance of such ideas occurs either gradually or in one fell swoop at a later point in time. Let us keep this in mind as we discuss Ronald C. Conant and W. Ross Ashby’s seminal work “Every Good Regulator Theorem” [2] (EGRT):

“[The EGRT is]….a theorem is presented which shows, under very broad conditions, that any regulator that is maximally both successful and simple must be isomorphic with the system being regulated…….Making a model is thus necessary.” [2]

The EGRT characterizes regulation with respect to cybernated control systems. In the case of Ashby and Conant [3], the EGRT developed within the context of several intersecting traditional fields. These include algorithmics, information theory, systems theory, and behavioral science. In such a context, models are exceedingly important. Given the reliance of the EGRT concept on inference and propositional thinking, there is an essential reliance on models. In fact, the EGRT exists at such a high level of abstraction that even with a high degree of specification may not be directly applicable in the real world [4]. However, there are certain advantages of cybernetic modeling that make their cross-contextual application useful.

Ashby's graphical formulation of the EGRT Theorem with original notation. COURTESY: [2].

II. Background

Let us return to the notion of modeling as phenomenology. Systems engage in modeling not simply to purposely regulate their environments, but rather to reactively respond to input stimuli in a way that maintains higher-level states [5]. This ability to model becomes part of their structure at the most basic of levels, though it would be fair to say most modeling (in the way we will use the word) is the result of cognitive processes. The constructivist might argue that such metacognitive dynamics [6] would influence one’s proposed scientific model. Like Shakespeare’s Hamlet, however, the question of whether or not to model (or be) is one of survival, whether that survival be genetic or memetic. Rather than reviewing the proof step-by-step, let’s discuss its potential significance in a variety of use-cases. In the process, we will be transcending the traditional boundaries of autonomic, ‘choice’, or even cognitive.

          Simply put, the Every Good Regulator Theorem says that regulators operate on approximations (e.g. models) of the thing they are regulating. This requires a mapping of the natural world to the model. While one might consider the activities of encoding and translation to be inherently cognitive, genomic systems also perform biological control functions in the absence of cognition [7, 8]. In the biological control example, what matters is not intent, but accuracy. Rather than an actively goal-oriented criterion, what we observe here is passively goal-oriented system output. Accuracy of the approximated model influences the quality of regulation. Thus, there need not be agency on the part of any single system component. Indeed, to survive as a unit in an interrelated system, a regulating machine must construct an interactive model that includes inputs, outputs, and feedback.

Let us consider a couple cases of regulatory dynamics, which may be valuable in understanding the importance of this theorem. We can then move on to what could this mean for both further theoretical development and practical application. A good place to begin in cognitive science is game theory [9]. One of the most simple, effective, and most explanatory strategies in the Prisoners’ Dilemma game is the tit-for-tat strategy [10]. In this 2-player, 2x2 game, the tit-for-tat strategy is simple: ‘Do unto others as they have done unto you’ after an initial good faith move of cooperation. The strategy is simply to copy your opponent's behavior. If the opposing agent cooperates, so does the tit-for-tat strategizing agent; if they defect, the tit-for-tat strategist follows suit. The intended outcome of the strategy is to move the exchange towards an equilibrium (though this is not the only possible outcome, nor is the strategy perfect).

Of specific interest here is that the mechanics of the strategy requires a model to be held in memory by the agent employing tit-for tat (a 1-bit cooperate/defect model), regardless of the strategy employed by the other agent (whether that be a more sophisticated maximizing strategy, or random selections). While an economist might view this as free-riding behavior by one of the two agents, the selection of tit-for-tat by both players can produce a cooperative equilibrium, such as in the evolution of reciprocal altruism in biological systems. The EGRT suggests that the greater the memory for an agent, and the longer it has the opportunity to observe and integrate the moves of its opponent, the greater its’ potential for effective regulation.

Over time, this can lead to greater accuracy for the agent’s cognitive model and a more stable equilibrium game outcome. Further, this equilibrium state can be long-lasting, given extended memory capacity for more detailed models, and may evolve towards ‘a conspiracy of doves’, within a game of homo lupus homini. An agent with a greater memory capacity can also employ more elaborate (or deeper) strategies over time. This development of deeper strategies may also feedback into modifying its model of the external world [11]. Overall, the capability to regulate behavior of other players depends on the inferential and predictive capacities of each player’s model: in a highly complex competitive game environment, a good regulator has a superior model, or it will find itself regulated by a competing agent in the game, especially as the behaviors get more complex.

“The theorem has the interesting corollary that the living brain, so far as it is to be successful and efficient as a regulator for survival, must proceed, in learning, by the formation of a model (or models) of its environment.” [2]

An example of a basic 2x2 payoff matrix characterizing the Prisoner's Dilemma. COURTESY: "Extortion in Prisoner's Dilemma", Blank on the Map blog, September 19 (2012).

III. Further Considerations

Let us now consider a more complicated scenario where we might be able to uncover the universal components of the EGRT phenomenology. The context will be two people on a blind date (this can actually be a complicated scenario). If one has been in one of these (terrifying) contexts, then one can already see where we are going. The cognitive agents are continually competing to increase the efficacy of their models of the other agent, while also attempting to constrain the modeling of the other agent towards a compact image they prefer. Although rarely implemented successfully, winning strategies include accurately modeling the other actor and influencing the state of their mental model. This can include both elaborate, multi-step strategies, and simpler strategies, the complexity of which is does not indicate their effectiveness. If the goal is a continuation of relations, the acquisition and intentional obfuscation of information occurs at appropriate times and in appropriate ways. Furthermore, this information has contextual value. As in most scenarios involving imperfect or asymmetrical information [12], your model must be superior to become the leader of the interaction [13], and thus control of regulation.

Does regulation even require what we would call cognition? This of course depends on our definition of cognition and regulation. However, let us consider that a bacterium does not have a “cognitive” or mental model of its environment, yet appears to have little trouble getting around and controlling some aspects of its landscape. The similarities between chemotactic sensation and mental models built upon multisensory stimuli serve as evidence for the universal character of the EGRT. In fact, Heylighen [14] has proposed that cybernetic regulation is a highly-generalized form of cognition. Yet do thermostats or other mechanical systems possess anything approaching what we consider cognition? While none of these has the cognitive capacity of a brain, they do have information processing capabilities from their physical or electronic structure, memory states, and crude models of how things ‘should’ be, towards which they regulate conditions. Non-cognitive systems possessing these characteristics are obviously still capable of rudimentary communication, control, decision making, and regulation, at least abstractly. We should also expect some degree of continuity that crosses the boundary of the cognitive and non-cognitive, since cognitive systems evolved from less intentional ones with more rudimentary forms of behavioral control.

“...success in regulation implies that a sufficiently similar model must have been built, whether it was done explicitly, or simply developed as the regulator was improved.” [2]

IV. Conclusion

Earlier, we had touched upon the history of scientific discovery, and contextual model building. A scientific theory is simply a model, and its value lies in its efficacy and repeatability (thus its’ trustworthiness and ability to aid in regulation). Theoretical models have tended, historically, to shift from informal, conceptual models towards formal mathematical ones (consider Comte’s Philosophy of Science). As a given model acquires more data, and as those data create ever-more accurate model revisions with higher fidelity. The overall capacity to aid regulation increases via feedback. Thus, the model’s value to humans increases. However, as noted by the example of ahead of their time thinking, scientific thought does not exist in a vacuum, and the landscape conditions need to be aligned so that the model can prove fruitful. Consider how we are witnessing an explosion in robust formal mathematical and/or computer models either aiding or besting human cognitive efforts [15, 16]. Informational revisions of the model often occur faster than the landscape conditions change, so adaptive cross-contextual models may prove more successful in dynamic situations, such as ones which are developed by human thought and human cultural systems.

            This ability to cross the boundary between cognitive and non-cognitive with models may challenge either our informal, colloquial conception of cognition or the universality criterion of the formal EGRT. As both features of cognition and more universal mechanisms, information processing, memory, communication, and selection can occur without any kind of cognitive superstructure. Perhaps the context of what we call “cognition” is too limiting. What about human cognition then is truly universal, and what is unique to a certain set mechanisms and representational models? For example, are models of so-called cellular decision-making [17] an unduly anthropomorphic representation of cellular differentiation and metabolism, or is it drawing upon a common set of universal properties that can only be abstracted from the system by an appropriate model?

Rather than trying to solve this philosophical puzzle now, let us take leave to consider that a deep truth like the one perhaps contained within the formalism of the EGRT should make us question scientific knowledge in a manner akin to reconsidering our firmly-held beliefs. It should make us reconsider how well we understand the relationship between nature and our own conceptual models. In that, it kindles the same spark from which all great scientific theories alight: It leads us to more questions, new ways of thinking about things, and guides us towards more accurate, repeatable, and otherwise ‘good’ models.

“Now that we know that any regulator (if it conforms to the qualifications given) must model what it regulates, we can proceed to measure how efficiently the brain carries out this process. There can no longer be question about whether the brain models its environment: it must.” [2]

References:

[1] Kuhn, T.   Structure of Scientific Revolutions. University of Chicago Press (1962). 

[2] Conant, R.C. and Ashby, W.R.   Every good regulator of a system must be a model of that system. International Journal of Systems Science, 1(2), 89–97 (1970).

[3] Ashby, W.R.   Introduction to Cybernetics. Chapman and Hall (1962).

[4] Fishwick, P.   The Role of Process Abstraction in Simulation. IEEE Transactions on Systems, Man, and Cybernetics, 18(1), 18-39 (1988).

[5] Brooks, R.   Intelligence Without Representation. Artificial Intelligence, 47, 139-159 (1991).

[6] Kornell, N. Metacognition in Humans and Animals. Current Directions in Psychological Science, 18(1), 11-15 (2009).

[7] Ertel, A. and Tozeren, A.   Human and mouse switch-like genes share common transcriptional regulatory mechanisms for bimodality. BMC Genomics, 23(9), 628 (2008).

[8] Gormley, M. and Tozeren, A.   Expression profiles of switch-like genes accurately classify tissue and infectious disease phenotypes in model-based classification. BMC Bioinformatics, 9, 486 (2008).

[9] Gintis, H.   Game Theory Evolving. Princeton University Press (2000).

[10] Imhof, L.A., Fudenberg, D., and Nowak, M.A.   Tit-for-tat or Win-stay, Lose-shift? Journal of Theoretical Biology, 247(3), 574–580 (2007).

[11] Liberatore, P. and Schaerf, M.   Belief Revision and Update: Complexity of Model Checking. Journal of Computer and System Sciences, 62(1), 43–72 (2001).

[12] Rasmussen, E.   Games and Information: an introduction to game theory. Blackwell Publishing (2006).

[13] Simaan, M. and Cruz, J.B.   On the Stackleberg Strategy in Nonzero-Sum Games. Journal of Optimization Theory and Applications, 11(5), 533-555 (1973).

[14] Heylighen, F.   Principles of Systems and Cybernetics: an evolutionary perspective. CiteSeerX, doi:10.1.1.32.7220 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.32.7220 (1992).

[15] LeCun, Y., Bengio, Y., and Hinton, G. Deep Learning. Nature, 521, 436-444 (2015).

[16] Ferrucci, D., Brown, E., Chu-Carroll, J., Fan, J., Gondek, D., Kalyanpur, A.A., Lally, A., Murdock, J.W., Nyberg, E., Prager, J., Schlaefer, N., and Welty, C.   Building Watson: an overview of the DeepQA project. AI Magazine, Fall (2010).

[17] Kobayashi, T.J., Kamimura, A.   Theoretical aspects of cellular decision-making and information-processing. Advances in Experimental Medicine and Biology, 736, 275-291 (2012).


UPDATE (9/30): During the editorial process, Rob and I had a discussion about using the word "alight" (in the final paragraph). I was not sure about the correct word usage, but Rob assured me that it was being used correctly in this context. But to back this up even further (and to gratuitously insert an informatics Easter Egg), here is the Google Ngram history of "alight" usage since 1800. 

Contributions to the bioRxiv, Summer 2014

I have been busy finishing up some work done in the Cellular Reprogramming Lab between 2010 and 2013. These two papers were submitted to and rejected as a single paper from PLoS One (after two rounds of revision). They were subsequently split them into their wet-lab molecular biology (written as an extended protocol) and computational components (written as a more conventional manuscript) for publication on bioRxiv.

The first paper (wet-lab molecular biology) is called "Using Polysome Isolation with Mechanism Alteration to Uncover Transcriptional and Translational Dynamics in Key Genes", in which we explore the world of mRNA regulation during adaptive cellular processes. The first part of the title (polysome isolation) involves harvesting mRNA from the polysome (translation-related mRNA). Harvesting this in tandem with mRNA associated with transcription provide us with a direct comparison between the transcriptome (TST) and translatome (TLT).

The second part of the title involves administering drug treatments to fibroblast populations which have systematic effects on transcription and protein production. These treatments are called "mechanism alteration" because they mimic changes that occur in a dying or transforming cell.

 

The third part of the title involves looking at transcriptional and translational dynamics for key genes. One criticism of the combined paper involved the use of candidate genes instead of high-throughput data. High-throughput data is great if one can afford it. On the other hand, large datasets can leave you with more questions than answers, which might be particularly true of this work. 

These figures demonstrate the analysis of experiments which validate the polysome recovery technique and the effects of drug treatments (mechanism disruption) on both transcriptome- and translatome- related mRNA (TST and TLT, respectively).

The second paper (computational) is called "Modeling Cellular Information Processing Using a Dynamical Approximation of Cellular mRNA". There is also a Github repository that contains associated Matlab code and simulations. Work from the first and second papers have been presented previous to their bioRxiv release, notably at the Stem Cell and Regenerative Medicine Conference held on the Oakland University (MI) campus in 2012.

We will go through this title in backwards order this time. The last part (cellular mRNA) refers to its connection to the first paper. Gene expression measured at both the transcriptome (TST) and translatome (TLT) will be used to model the cell's general response to mechanism alteration. In this case, an assumption is made: fluctuations of both mRNA fractions and at multiple points in time represents a regulatory process. 

Thus, a first-order feedback model can be constructed, with a simplified set of feedforward, feedback, and decay components. While there are a multitude of mRNA decay pathways and processing functions, this model focuses on a much simpler abstraction: the path from DNA to protein with single sources of decay and feedback. Each fraction of mRNA can be represented as a point process controlled by inputs and outputs.

The middle part of the title (dynamical approximation) then refers to the simulation of mRNA dynamics using the model, its components, and biological data. The idea is to approximate meaningful trends at certain points in a biological process, which is expected to differ by gene and by model component. This is where the proof-of-concept nature of this paper is most evident. 

While a somewhat contrived means to approximate a complex biological process is used in both papers, the original application was to be for understanding the early stages of cellular reprogramming. However, it proved to be exceedingly difficult to go from iPS cultures to meaningful computational inference.

An example of the first-order feedback model. A: a graphical example of the model components. B: an example of activity among the components over time.

Finally, the first part of the title (modeling cellular information processing) is based on the interpretation of the model output. the notion of cellular information processing treats the regulation of mRNA as an information processing problem. That is, you have an input, a process, and an output. systematic noise can also be added to the model, depending on the application. The process itself (mRNA processing from DNA transcription to RNA translation) is a transformation of information. 

When a cell is challenged by an environmental stimulus or the need to change phenotype, information provided by mRNA can be operated upon in a number of ways (linear responses, accumulation, delayed responses). Three information processing principles are used to interpret phenomena such as linear decay, the sequestration of mRNA at either the TST or TLT, and the differential response among individual genes.

Finally, I must point out how cool the altimetric support is on the bioRxiv. Here is a screen shot showing the number of tweets, abstract views, and .pdf downloads for the wet-lab paper:

Almost as functional as the analytics data on Blogger (which is not saying much, but for a formal publication venue, it's pretty impressive). The people at Cold Spring Harbor Lab have done a good job on this, and are ahead of the people at arXiv on this. As it turns out, however, the arXiv has a principled policy on this. Whether viewership stats are a sign of vanity and worthy of scare quotes is another matter.

New Directions in Space, Time, and Thought

Here is the latest news from the realm of Tumbld Thoughts. All features are interesting. In this post, we move from the latest episodes of Cosmos to new directions in economic value and the arXiv, to new directions in practicing research.

I. Making Amphibians Out Of Quarks and Other Tales of Scale

Here are the supplementary readings for Episode 6 of the Cosmos reboot, called “Deeper, Deeper Still”. These are organized by theme. I am not responsible for any groans my puns may cause.

(Episode) Origins…..

A sneak peek for this week. Daily Galaxy blog, April 12 (2014).

Ziggy Stardust and the Extra Dimensions (on Mars?):

Berkowitz, J.   The Stardust Revolution. Prometheus Books (2012).

Greene, B.   The Search for Hidden Dimensions. Richard Dawkins Foundation for Reason and Science. YouTube, May 17 (2010).

A human = 10^30 quarks?

Wolchover, N.   A Jewel at the Heart of Quantum Physics. Quanta Magazine, September 17 (2013).

Carroll, S.   Jaroslav Trnka on the Amplituhedron. Preposterous Universe blog, March 31 (2014).

Filmer, J.   New Discovery Simplifies Quantum Physics. From Quarks to Quasars blog, September 19 (2013).

Huang, C.   Scale of the Universe II. Scaleofuniverse.com.

Tardigrages and Angiosperms:

Stromberg, J.   How Does the Tiny Waterbear Survive in Outer Space?Smithsonian.com, September 11 (2012).

Nichols, P.B., Nelson, D.R., and Garey, J.R.   A family-level analysis of tardigrade phylogeny. Hydrobiologia, 558, 53-60 (2006).

Soltis, P., Soltis, D., and Edwards, C.   Angiosperms. Tree of Life (2005).

Plants Move Towards the Light and Make Food:

Wyatt, S.E. and Kiss, J.Z.   Plant tropisms: from Darwin to the International Space Station. American Journal of Botany, 100(1), 1-3 (2013).

Artificial Photosynthesis. Wikipedia, April 13 (2014).

Carbon is Versatile:

Buckminsterfullerene. Wikipedia, April 13 (2014).

Carbon Nanotube. Wikipedia, April 13 (2014).

Wall of Forever:

Wall, M.   Major Discovery: ‘Smoking Gun’ for Universe’s Incredible Big Bang Expansion Found. Space.com, March 17 (2014).

Tate, K.   How Gravitational Waves Work (Infographic). Space.com, March 17 (2014).

IMAGES:

Third from top: Book Cover, You are Stardust. Elin Kelsey and Soyeon Kim.

Fourth from top: Ichetucknee Springs, North Florida, USA.

Bottom Image: Evidence for Cosmic Inflation following the Big Bang, COURTESY:BICEP2 Group.

II. Clean Room Redux

Here are the supplemental readings for the seventh episode of the Cosmos reboot entitled "The Clean Room". A bit of a departure from the previous episodes in that the focus was on the social consequences of scientific findings. As usual, readings are thematic.

Meteors, Sediments, and Early Earth:
Scientists Building Asteroid Threat Early-Warning System. Space.com, February 20 (2013).

Diverging evolution of early Earth and Mars revealed by meteorites. The Daily Galaxy blog, April 17 (2014).

Appenzeller, T.   Early Earth. National Geographic, December (2006).

Clean Rooms and Isotopes:
Radioactive Decay: a sweet simulation of a half-life. AAAS Science NetLinks.

Radioactive Dating Game. PhET Interactive Simulations.

Lewington, R.   A virtual tour of Applied Materials' clean room. Applied Materials blog.

Chemophobia vs. Public Relations and the role of science:

How corporations corrupt science at the public's expense. Union of Concerned Scientists, Center for Science and Democracy.

Washburn, J.   Science's Worst Enemy: corporate funding. Discover Magazine, October (2007).

"Silent Spring" at 50. The credit, and the blame, it deserves. Big Think blog, June 19 (2012).

Lead poisoning and health. World Health Organization, Fact sheet #379. September (2013).

Needleman, H.L.   The removal of lead from gasoline: historical and personal reflections. Environmental Research, 84(1), 20-35 (2000).

III. Pushing the Boundaries of the arXiv

I guess I am literally pushing the boundaries of the arXiv. On Tuesday, March 25, I submitted a paper called "Contextual and Structural Representations of Market-mediated Economic Value". While they normally announce the paper at Midnight (GMT) the following weekday, this paper was not announced until two days later (Friday morning).

Usually, when a paper is delayed, it means there is an issue with classification. Ultimately, the paper was placed in the q-fin.GN category. Then, 12 days later, arXiv introduced two new categories: q-fin.EC (economics) and q-fin.MF (mathematical finance). While this could be a coincidence, I still like to think that my paper broke their system. Hopefully, it ends up breaks new ground and old paradigms as well.

IV. The New, Potentially Paradigm-busting Paper on the arXiv

How do we assign value to economic transactions? In my latest paper, now available at the arXiv, I approach this problem using a computational and evolutionary approach. "Contextual and Structural Representations of Market-mediated Economic Value" is my first paper in the "q-fin" category (1403.7021, q-fin.GN).

Culturally-mediated biological markets are used to model several aspect of object valuation. Contextual Geometric Structures (CGSs) [1] are used to model individual minds in an agent-based simulation. Read the paper to fully appreciate what this means. While it is a purely computational study, it might also be of interest to behavioral economists and evolutionary anthropologists.

[1] Alicea, B.   Contextual Geometric Structures: modeling the fundamental components of cultural behavior.

Proceedings of Artificial Life, 13, 147-154 (2012).

IV. Orthogonal Research: slouching towards research enterprise

In lieu of a formal academic position, I am now publishing and conducting work under the affiliation "Orthogonal Research". This is (currently) a money-less start-up, focused on research in mathematical modeling and data analysis. Right now, this just involves myself. However, potential collaborators, co-PIs, and funders are welcome to contact me.

Things are a great deal more serious than this.

The Orthogonal Research Q1 activity report is now available. "Q1" refers to the first quarter of the calendar year, not financial.

On Bet-hedging and Evolutionary Futures

When someone mentions "bet-hedging", the first thing that comes to mind is an economic investment or gambling strategy, one that maximizes a human's return on their investment. This necessitates cognitive mechanisms for decision-making and valuation. For example, a bet-hedger might keep their investments in two places (e.g a rowboat and a hang-glider). When the return on one potential source of income is exhausted (e.g. rowboat goes over the falls), the other investment can be drawn upon to pick up the slack. Overall, total losses are minimized and potential gains are maximized.

Bet-hedging as a human investment strategy. Hence the rowboat and hang-glider analogy.

But bet-hedging has also been used as a means to explain biological "decision-making" with respect to adaptive changes in genotype and/or phenotype. In this case, decision-making refers to directed changes in a lineage that emerge from stochastic mechanisms. There is no need for a set of formal cognitive mechanisms (or a designer), because in this case natural selection plays the role of a brain (e.g. information processing). In the case of biological bet-hedging, an organism hedges between two or more phenotypic/genotypic states (or behavioral strategies), one becoming predominant only when encouraged by environmental conditions.

Figure 1. A statistical view of biological bet-hedging. COURTESY: Discussion of bet-hedging and adaptation to environmental stresses in [1].

To understand how biological bet-hedging might work, we can use a Venn diagram to take a statistical view of the process (Figure 1). Some form of switching (phenotypic, genotypic) is induced by a subset of environmental stimuli. A smaller subset of positive responses are triggered by environmental stress. This relatively small resulting set of adaptive responses (switching due to an environmental stress signal) should (in theory) be the outcomes of highest fitness value. 

There are many ways this bet-hedging can be accomplished, as the existing literature is quite diverse. I will focus on two candidate mechanisms from the literature: selection on random noise and selection on standing variation.

Selection on random noise is driven by inherent oscillations in gene expression. Gene expression differences (e.g. differential gene expression) are usually thought to define distinct biological processes and cell types. However, gene expression also exhibits wide-band fluctuations [2] and "bursty" changes over time [3], even within the same process or cell type. A recent review in Science on oscillating gene expression over time (e.g. gene expression noise) and biological bet-hedging [4] discusses these mechanisms in more detail.

Figure 2. Type A transcriptional oscillations, sensu [4]. One example of genotypic bet-hedging.

The first type of bet-hedging (type A) is indirect, and involves retaining two or more context-dependent mechanisms for the expression of a single gene. In Figure 2, the successful transcription of a downstream target gene depends on the synchronized activity of two upstream transcription factors. In this case, each upstream gene fluctuates with respect to time. Only when their fluctuations become coordinated in-phaseare they capable of activating a downstream target. In fact, in order to exhibit this selective synchronization, the upstream gene expression must be oscillatory. Otherwise, the downstream gene would not be sensitive to environmental context.

Figure 3. Type B transcriptional oscillations, sensu [4]. Another example of genotypic bet-hedging.

By contrast, type B bet-hedging (direct) is a matter of selecting between alternating genotypic or phenotypic states. In this case, upstream genetic mechanisms create a bistable switch which allows the organism to switch between states given an environmental signal. In the case of stochastic bet-hedging, switching can be like a roulette wheel, in which all possible states are generated spontaneously. The most (as opposed to transient) stable states are those which are most strongly supported by the environment. This is a candidate model for how cells acquire and maintain their identity in microenvironmental niches. Yet it is also relevant to evolutionary change.

Instead of individual phenotypes being the focus of selection, perhaps the ability to switch between phenotypes itself is selected as a trait. For example, stochastic phenotype switching in bacteria results in persistence in the face of rapid environmental variation [5]. Using Pseudomonas fluorescens lineages, phenotypic switching can be experimentally evolved. In fact, the capacity for switching can be isolated to a single gene mutation (CarB), which is both sufficient and necessary for the colony switching trait.

Although stochastic switching is a single trait, it is by no means a simple one. Not only does CarB enable colony switching, but lineages that carry this mutation have a higher fitness compared to those that do not among mixed cultures in a static environment. In the experiments of [5], it took multiple rounds of selection to evolve the CarB mutant. This may also be due to enabling mutations and lineage-specific dependencies which set up the transition to a CarB mutant phenotype.

CarB mutation, examined within a single bet-hedging lineage. TOP: fitness, MIDDLE: genotypes, BOTTOM: phenotypes.

Now we will jump to a highly-speculative infographic [6] on future events in Earth's history. "Timeline of the Far Future" proposes events from the present to Earth's best-case scenario astrological death. In 100 quadrillion (10²⁰) years, the Earth's orbit is predicted to fall into the Sun, if the red giant Sun does not engulf the Earth before then.

In any case, five events on this timeline are relevant to the future of life and worthy of discussion. Keep in mind that this infographic is not sequentially consistent, as it is based on multiple sources of data and overlapping potential scenarios. These events include:

* end of Eukaryotic life, in 1.3 billion years.

* end of Prokaryotic and Archean life, in 2.8 billion years.

* end of C₃ photosynthesis, in 600 million years.

* end of C4 photosynthesis, in 800 million years.

* Earth's temperature rises to 47C due to an increase in solar luminosity, in 1 billion years.

Some of the predictions are provocative, such as the extinction of the Y chromosome [7]. However, one set of predictions intersect with the literature on bet-hedging: the end of photosynthesis. What we know about the possible end of photosynthesis comes from studies [8, 9] that examine the instability and breakdown of photosynthetic reactions at high temperatures. A primary producer's photosynthesis rate becomes unstable above a threshold temperature [8]. When temperature is lowered, this rate returns to normal.

When applied in a laboratory or experienced during heat waves, severe heat stress can cause cellular damage. In fact, large-scale process-based models of photosynthesis assumes that the rate returns to normal after environmental shocks [9]. However, what happens in cases where the average temperature reaches or exceeds this threshold? In such cases, extreme temperatures are not experienced as shocks, but as the physiological set-point.

View of a "red giant" Sun from a now-barren (far future) Earth. Humans, the oceans, and perhaps all extremophiles are all gone at this point.

According to the infographic's predictions, in 0.6 to 2.8 billion that will be that (although I'm not sure why primary producers would be the first type of life to die out). Yet there might be three fundamental changes to life's complexity as we reach a red giant Sun that enable it to survive until perhaps the physical end of the Earth (and/or Sun): 

* the radical restructuring of organismal physiology to suit temperatures that will approach the boiling point of water. This might include hard insulating shells, smaller areas of exposed surface, and "interesting" changes to metabolism. The hedging concept could come into play here, as the expression of genes and phenotypes associated with metabolic function in all organisms (not just single-celled ones) could fluctuate significantly across life-history.

* the radical restructuring of food webs so as to reduce any one source of primary or secondary production. Bets would be hedged in order to survive ever increasing extreme conditions. The increase of energy in the biosphere could lead to more energy being available in general. But of course there could be biospheric tradeoffs (atmospheric composition) which could limit ecological complexity. The sources of primary production would of course need to adapt to take advantage of this situation.

* the evolution of primary production itself. Like complex organisms, we should not expect photosynthesis to simply disappear. Recall that C4 photosynthesis was dominant in the Paleozoic era, only to be eclipsed by the C₃ variety during the Mesozoic era. And this transition occurred despite C4 photosynthesis being more efficient than C₃ in times of heat stress and draught [10]. It might turn out that a new type of hyper-efficient and multiphasic photosynthesis could evolve that takes advantage of late solar system conditions.

These points are speculative as well, but remember -- fully-functioning ecosystems that are gradually exposed to extreme conditions will likely adapt instead of coming to an abrupt end. It would be hard to transplant existing organisms (even extremophiles) into these conditions, but it might not be as hard to evolve responses to the extremities of late Earth. 

NOTES:
[1] Mostowy, R.   Evolution of Stress Response in the Face of Unreliable Environmental Signals. Rafal Mostowy blog, August 20 (2012).

[2] Eldar, A. and Elowitz, M.B.   Functional roles for noise in genetic circuits. Nature, 467, 167-173 (2010).

[3] Goh, K-I. and Barabasi, A-L.   Burstiness and Memory in Complex Systems. arXiv:0610233.

[4] Levine, J.H., Lin, Y., and Elowitz, M.B.   Functional Roles of Pulsing in Genetic Circuits. Science, 342, 1193 (2013).

[5] Beaumont, H.J.E., Gallie, J., Kost, C., Ferguson, G.C., and Rainey, P.B.   Experimental evolution of bet hedging. Nature, 462, 90-93 (2009).

[6] Timeline of the Far Future. BBC Future, January 6 (2014).

[7] This is an example of a scientific debate disguised as persistent myth in the popular press. For two perspectives (the former Y-optimist, the latter Y-pessimist), please see:

a) Hughes, J.F. et.al   Strict evolutionary conservation followed rapid gene loss on human and rhesus Y chromosomes. Nature, 483, 82-86 (2012).

b) Aitken, R.J. and Marshall-Graves, J.A.   Human Spermatozoa: the future of sex. Nature, 415, 963 (2002).

[8] Sage, R.F. and Kubien, D.S.   The temperature response of C₃ and C₄ photosynthesis. Plant, Cell, and Environment, 30, 1086-1106 (2007).

[9] Huve, K., Bichele, I., Rasulov, B., and Niinemets, U.   When it is too hot for Photosynthesis: heat-induced instability of photosynthesis in relation to respiratory burst, cell permeability changes and H₂O₂ formation. Plant, Cell, and Environment, 34, 113-126 (2011).

[10] Liu, Z., Sun, N., Yang, S., Zhao, Y., Wang, X., Hao, X., and Qiao, Z.   Evolutionary transition from C3 to C4 photosynthesis and the route to C4 rice. Biologia, 68(4), 577-586 (2013).