I like the visual layout of scoop.it (they offer magazine-style templates for organizing content), and further like the idea of having the updates archived all in one place. Check it out if interested.
March 19, 2012
New Look for "Complexity Digest"
Complexity Digest (a.k.a. ComDig), the weekly update on all things complexity-oriented run by Carlos Gershenson, has a new look. Once distributed to subscribers via e-mail, it is now being hosted on the scoop.it servers.
I like the visual layout of scoop.it (they offer magazine-style templates for organizing content), and further like the idea of having the updates archived all in one place. Check it out if interested.
I like the visual layout of scoop.it (they offer magazine-style templates for organizing content), and further like the idea of having the updates archived all in one place. Check it out if interested.
March 18, 2012
Methods of Controlling Intelligence
This blog post will focus on the recent (and not so recent) attempts to quantify, control, and augment intelligent performance-related behavior in human beings. The intersection of human intelligence and artificial intelligence by way of human performance goes by the name of Augmented Cognition. Augmented Cognition, generally regarded as a domain of Human Factors engineering, also has broad applications to human-machine systems. Relevant application domains could range from automotive and transportation performance to human interactions with information technologies and bioengineered prosthetic devices.
Augmented Cognition is distinct from traditional artificial intelligence, in which a general purpose intelligence is constructed de novo to control all aspects of intelligent behavior. Rather than machine intelligence compensating for the shortcomings of human intelligence, human intelligence compensates for the shortcomings of machine intelligence. Academic interest in this set of problems began in the 1950's [1], while contemporary approaches have included information technologies and DARPA's Augmented Cognition project. As applied to technology, this work falls into the broader category of human-assisted intelligent control.
There are two main components of augmenting human intelligence using computational means. The first is a closed-loop system which involves a feedforward and a feedback component between the individual and a technological system that enabled augmentation. This could be a heads-up display, a mobile device, or a brain-computer interface controlled by a real-time algorithm. The second is a model of human performance for a given set of cognitive and physiological functions which determines a control policy. Examples of both are provided below, along with a consideration of open problems in this field.
In an article from the 1950's [2], W.R. Ashby took a cybernetic approach to first-order (e.g. no intermediate variables) intelligence augmentation (Figures 1-4). While somewhat crude by modern standards (by which we use sensors to gain real-time measurements of physiological state), it does lay out a simple theoretical model for augmenting cognitive and neural function.
In the Ashby model, the feedforward component (G) was the intelligence of the user applied to performance captured by the device. This might be driving performance, or accuracy in moving an object. While the idea that intelligence can be distilled to a single variable is controversial, modern applications have used variables such as accuracy counts or a specific electrophysiological signal to "drive forward" the system. The amplifier (S) itself gathers the feedforward elements of G and operates on them in a selective manner. This can be treated as either an optimization problem [3] or an inverse problem [4], and defines the control policy imposed on the performance data. In the Yerkes-Dodson example shown later on, a minimax-style optimization method is used. The feedback element (U) is a signal taken from the information in G and should contribute to an improvement in performance, or subsequent measurements of G.
More contemporary models for augmenting human performance [5,6] have involved mapping closed-loop control to a physiological response function. Figures 5 through 7 show how this works in the context of the Yerkes-Dodson curve. The Yerkes-Dodson curve is an inversely U-shaped function that characterizes arousal in the context of some physiological measurement. At both low and high values of the physiological indicator, the level of arousal is low. At moderate values of the physiological indicator, the level of arousal is high. The goal of an amplifier (also called a mitigation strategy) is to maintain performance (defined as measured arousal) among the highest range of arousal values.
There are two potential challenges to this control policy: a reliance on convexity and complete measurement of a physiological state. The example shown here has relevance to arousal and attention. It has attracted attention because of its relative ease of mitigation. The development of brain-machine interfaces has likewise focused on simple-to-characterize physiological signals (such as population vector codes for movement [7] or spectral bands of an EEG [8]). However, not all physiological response functions are so simple to characterize. In cases of significant non-convexity (or cases where the response function does not form smooth, convex gradients), it may be quite difficult to mitigate suboptimal behavior or physiological responses [9]. In such cases, there could be multiple optimal points each with very different performance characteristics.
The complete measurement of physiological state is another potential problem with this method. While fully characterizing a physiological or behavioral process is the most obvious difficulty, the adaptability of a physiological system to repeated mitigation is a more subtle but important problem. In some cases, the physiological response will habituate to the mitigation treatments and render them ineffective. In the case of presenting information on a heads-up display, users might simply tend to ignore the presented cues over long periods of time. It might also be that encouraging rapid changes in arousal level is more effective than encouraging a fixed level of performance over time. In both strength training regimens and more general physiological responses to the environment, switching between stimuli of alternating intensities can have a complex and ultimately adaptive consequences on the long-term response.
Incorporation of intelligence augmentation into the design of a technological system is an ongoing challenge. In a future post, I will focus on why certain aspects of human and animal intelligence are fundamentally different from and can potentially aid and complement current approaches to machine learning and artificial intelligence.
[1] Ashby, W.R. (1952). Design for a Brain. Chapman and Hall, London.
[2] Ashby, W.R. (1958). Design for an Intelligence Amplifier. In Automata Studies. Shannon, C.E. and Ashby, W.R. Princeton University Press, Princeton, NJ.
[3] an optimization method uses some objective criterion to select a range of values thought to either minimize or maximize system properties.
[4] an inverse problem is one where the solution is known, but the route to that solution is not.
[5] Schmorrow, D. D. & Stanney, K.M. (Eds) (2008). Augmented Cognition: A Practitioner's Guide. HFES Publications.
[6] Fuchs, S., Hale, K.S., Stanney, K.M., Juhnke, J., and Schmorrow, D.D. (2007). Enhancing Mitigation in Augmented Cognition. Journal of Cognitive Engineering and Decision Making, 1(3), 309-326.
[7] Jarosiewicz, B., Chase, S.M., Fraser, J.W., Velliste, M., Kass, R.E., and Schwartz, A.B. (2008). Functional network reorganization during learning in a brain-computer interface paradigm. PNAS, 105(49), 19486-19491.
[8] Lotte, F., Congedo, M., Lecuyer, A., Lamarche, F., and Arnaldi, B. (2007). A review of classification algorithms for EEG-based Brain-Computer Interfaces. Journal of Neural Engineering, 4, 1-24.
[9] Alicea, B. The adaptability of physiological systems optimizes performance: new directions in
augmentation. arXiv Repository, arXiv:0810.4884 [cs.HC, cs.NE] (2008).
Augmented Cognition is distinct from traditional artificial intelligence, in which a general purpose intelligence is constructed de novo to control all aspects of intelligent behavior. Rather than machine intelligence compensating for the shortcomings of human intelligence, human intelligence compensates for the shortcomings of machine intelligence. Academic interest in this set of problems began in the 1950's [1], while contemporary approaches have included information technologies and DARPA's Augmented Cognition project. As applied to technology, this work falls into the broader category of human-assisted intelligent control.
There are two main components of augmenting human intelligence using computational means. The first is a closed-loop system which involves a feedforward and a feedback component between the individual and a technological system that enabled augmentation. This could be a heads-up display, a mobile device, or a brain-computer interface controlled by a real-time algorithm. The second is a model of human performance for a given set of cognitive and physiological functions which determines a control policy. Examples of both are provided below, along with a consideration of open problems in this field.
Closed-loop System Design
In an article from the 1950's [2], W.R. Ashby took a cybernetic approach to first-order (e.g. no intermediate variables) intelligence augmentation (Figures 1-4). While somewhat crude by modern standards (by which we use sensors to gain real-time measurements of physiological state), it does lay out a simple theoretical model for augmenting cognitive and neural function.
Figure 1. Highlight for the X component.
Figure 2. Highlight for the G component.
Figure 3. Highlight for the S component
Figure 4. Highlight for the U component.
In the Ashby model, the feedforward component (G) was the intelligence of the user applied to performance captured by the device. This might be driving performance, or accuracy in moving an object. While the idea that intelligence can be distilled to a single variable is controversial, modern applications have used variables such as accuracy counts or a specific electrophysiological signal to "drive forward" the system. The amplifier (S) itself gathers the feedforward elements of G and operates on them in a selective manner. This can be treated as either an optimization problem [3] or an inverse problem [4], and defines the control policy imposed on the performance data. In the Yerkes-Dodson example shown later on, a minimax-style optimization method is used. The feedback element (U) is a signal taken from the information in G and should contribute to an improvement in performance, or subsequent measurements of G.
Contemporary Models from Human Performance
More contemporary models for augmenting human performance [5,6] have involved mapping closed-loop control to a physiological response function. Figures 5 through 7 show how this works in the context of the Yerkes-Dodson curve. The Yerkes-Dodson curve is an inversely U-shaped function that characterizes arousal in the context of some physiological measurement. At both low and high values of the physiological indicator, the level of arousal is low. At moderate values of the physiological indicator, the level of arousal is high. The goal of an amplifier (also called a mitigation strategy) is to maintain performance (defined as measured arousal) among the highest range of arousal values.
Figure 5. Example of a physiological response function (e.g. Yerkes-Dodson curve).
Figure 6. Example of a mitigation strategy.
Figure 7. Keeping performance within an optimal range.
Two outstanding problems
There are two potential challenges to this control policy: a reliance on convexity and complete measurement of a physiological state. The example shown here has relevance to arousal and attention. It has attracted attention because of its relative ease of mitigation. The development of brain-machine interfaces has likewise focused on simple-to-characterize physiological signals (such as population vector codes for movement [7] or spectral bands of an EEG [8]). However, not all physiological response functions are so simple to characterize. In cases of significant non-convexity (or cases where the response function does not form smooth, convex gradients), it may be quite difficult to mitigate suboptimal behavior or physiological responses [9]. In such cases, there could be multiple optimal points each with very different performance characteristics.
The complete measurement of physiological state is another potential problem with this method. While fully characterizing a physiological or behavioral process is the most obvious difficulty, the adaptability of a physiological system to repeated mitigation is a more subtle but important problem. In some cases, the physiological response will habituate to the mitigation treatments and render them ineffective. In the case of presenting information on a heads-up display, users might simply tend to ignore the presented cues over long periods of time. It might also be that encouraging rapid changes in arousal level is more effective than encouraging a fixed level of performance over time. In both strength training regimens and more general physiological responses to the environment, switching between stimuli of alternating intensities can have a complex and ultimately adaptive consequences on the long-term response.
Incorporation of intelligence augmentation into the design of a technological system is an ongoing challenge. In a future post, I will focus on why certain aspects of human and animal intelligence are fundamentally different from and can potentially aid and complement current approaches to machine learning and artificial intelligence.
References:
[1] Ashby, W.R. (1952). Design for a Brain. Chapman and Hall, London.
[2] Ashby, W.R. (1958). Design for an Intelligence Amplifier. In Automata Studies. Shannon, C.E. and Ashby, W.R. Princeton University Press, Princeton, NJ.
[3] an optimization method uses some objective criterion to select a range of values thought to either minimize or maximize system properties.
[4] an inverse problem is one where the solution is known, but the route to that solution is not.
[5] Schmorrow, D. D. & Stanney, K.M. (Eds) (2008). Augmented Cognition: A Practitioner's Guide. HFES Publications.
[6] Fuchs, S., Hale, K.S., Stanney, K.M., Juhnke, J., and Schmorrow, D.D. (2007). Enhancing Mitigation in Augmented Cognition. Journal of Cognitive Engineering and Decision Making, 1(3), 309-326.
[7] Jarosiewicz, B., Chase, S.M., Fraser, J.W., Velliste, M., Kass, R.E., and Schwartz, A.B. (2008). Functional network reorganization during learning in a brain-computer interface paradigm. PNAS, 105(49), 19486-19491.
[8] Lotte, F., Congedo, M., Lecuyer, A., Lamarche, F., and Arnaldi, B. (2007). A review of classification algorithms for EEG-based Brain-Computer Interfaces. Journal of Neural Engineering, 4, 1-24.
[9] Alicea, B. The adaptability of physiological systems optimizes performance: new directions in
augmentation. arXiv Repository, arXiv:0810.4884 [cs.HC, cs.NE] (2008).
March 8, 2012
The art of a lifetime (so far)......
Even when I was a child, I had a strong interest in science and engineering. I always had a talent for drawing. As a teenager, I got interested in becoming an artist (I also got interested in becoming an economist, but that didn't have worked out purely on principle). I wanted to go to art school, but my art was not "critically acclaimed" enough to impress people. I have a very technical style of drawing that I freely interchange with "cartoonish" features and obscure references to create what I consider art. It is very much the opposite of the good aesthetic form and practice typically learned in art school.
My art avocation has been expressed in two periods. I have posted most of these works on my personal website. The first was from 1998-2003. During this period, I focused on using mathematical/ technical concepts and hand-drawn (digitized) cartoons to create abstract art. The following works are examples of this:
Some of these works (e.g. Van Gogh meets the Samurai) resemble the narrative structure of conventional sequential art. Others (e.g. Poincare Recurrence) feature made-up characters as actors in scientific theories and real-life situations. Still others (e.g. Chaz Rodders II) feature made-up characters as actors in purely fantastical scenarios.
The is a gap comprising the years 2003 through 2008. The reason: PhD programs (e.g. coursework, finding a research voice) are time-consuming. Since I spent time in a lab that developed and experimented with virtual worlds, there is probably something in this experience to inspire some interesting artwork that I will explore over time.
The second period was from 2009-present. During this period, I started focusing more on merging pop culture and images from the internet into composite, abstract images. The following works are examples of this:
These works take less time to create, and also reflect a more conventional "mash-up" style. I have also taken to interesting juxtapositions that have little semantic value but have other, more subtle relationships. For example, in "Crazy Eyes", there are three characters (Egon Spengler from "Ghostbusters", a robot avatar from Second Life, and Kramer from Seinfeld) that share a certain physical set of congruities. Other works (e.g. Razor-faced Spy, Obscure References 3x) are simply modifications to references and icons from pop culture.
Having said that, I feel that art is about more than pure subjectivity. I feel that artistic creativity has an underappreciated value in science and engineering. I had a professor at the University of Florida (Dr. Paul Fishwick) who has developed an approach to programming called Aesthetic Computing. Aesthetic Computing uses artistic representations to map out data structures, algorithms, and algebraic relationships. I have found that this approach is somewhat useful in conveying complicated scientific concepts and theoretical advances to audiences from diverse backgrounds (e.g. biologists and mathematicians). I am also a fan of using devices such as the Feynman diagram to describe advanced mathematical concepts. In the future, I would like to take my early period style of art and apply it to problems I have encountered as an academic scientist. This is fertile territory for the communication and popularization of science, mathematics, and advanced technologies, so if anyone out there is interested in developing this further, contact me.
My art avocation has been expressed in two periods. I have posted most of these works on my personal website. The first was from 1998-2003. During this period, I focused on using mathematical/ technical concepts and hand-drawn (digitized) cartoons to create abstract art. The following works are examples of this:
"Meltdown meets the Domino Effect"
"Sequential Spike"
"Chaz Rodders II"
Some of these works (e.g. Van Gogh meets the Samurai) resemble the narrative structure of conventional sequential art. Others (e.g. Poincare Recurrence) feature made-up characters as actors in scientific theories and real-life situations. Still others (e.g. Chaz Rodders II) feature made-up characters as actors in purely fantastical scenarios.
The is a gap comprising the years 2003 through 2008. The reason: PhD programs (e.g. coursework, finding a research voice) are time-consuming. Since I spent time in a lab that developed and experimented with virtual worlds, there is probably something in this experience to inspire some interesting artwork that I will explore over time.
The second period was from 2009-present. During this period, I started focusing more on merging pop culture and images from the internet into composite, abstract images. The following works are examples of this:
"Judgement Day"
"Obscure References, obscure references, obscure references!"
"Crazy Eyes"
"Razor-faced Spy"
"Decapod vs. Cephalopod"
These works take less time to create, and also reflect a more conventional "mash-up" style. I have also taken to interesting juxtapositions that have little semantic value but have other, more subtle relationships. For example, in "Crazy Eyes", there are three characters (Egon Spengler from "Ghostbusters", a robot avatar from Second Life, and Kramer from Seinfeld) that share a certain physical set of congruities. Other works (e.g. Razor-faced Spy, Obscure References 3x) are simply modifications to references and icons from pop culture.
Having said that, I feel that art is about more than pure subjectivity. I feel that artistic creativity has an underappreciated value in science and engineering. I had a professor at the University of Florida (Dr. Paul Fishwick) who has developed an approach to programming called Aesthetic Computing. Aesthetic Computing uses artistic representations to map out data structures, algorithms, and algebraic relationships. I have found that this approach is somewhat useful in conveying complicated scientific concepts and theoretical advances to audiences from diverse backgrounds (e.g. biologists and mathematicians). I am also a fan of using devices such as the Feynman diagram to describe advanced mathematical concepts. In the future, I would like to take my early period style of art and apply it to problems I have encountered as an academic scientist. This is fertile territory for the communication and popularization of science, mathematics, and advanced technologies, so if anyone out there is interested in developing this further, contact me.
February 27, 2012
Turing Centenary Year features
In this post, I will be discussing some recent features in the popular press highlighting the life and work of Alan Turing. This year is Turing's posthumous 100th birthday. Turing contributed a lot to modern computing science, with many people calling him the "father" of computer science. Alan Turing is known for two major results (both developed in the 1930s):
1) discoverer of the Turing machine, which is the basis of both the Church-Turing thesis and modern algorithm design.
Example of a Turing Machine (COURTESY: Wikipedia).
2) characterized Turing (chemical) morphogenesis, which is a leading model for explaining pattern formation in animal development and "spontaneous" pattern generation.
Example of Turing Morphogenesis (LEFT: striping patterns on fish, COURTESY: Wired Science, RIGHT: equations that govern pattern formation, COURTESY: Johannes Wilbert Blog).
This week's issue of Nature (Volume 482, Issue 7386) features a special section on Turing's legacy (see below). There are several interesting articles in this issue contemplating how Turing's work is also relevant to a number of scientific fields. In one article, Sydney Brenner draws parallels between biological cells and Turing machines. In another set of essays, four scientists (including Rodney Brooks) re-assess the brain as a model for machine intelligence. Check it out if you can.
The latest issue of Communications of the ACM also features Turing on the cover (see below), and contains an interesting article by Moshe Vardi on the current state of algorithm design. The major issue highlighted in this article involves the legacy of the Turing machine. While contemporary algorithms are used for diverse purposes, the following question is still outstanding: are algorithms most effective as state machines, or are they recursion engines? While I will not attempt, to answer that question in this post, it is a question worth pondering.
February 25, 2012
Fictitious life-history.........
February 23, 2012
Evolution for free? Self-organization as driver of natural selection
One of the central components of current evolutionary theory, particularly the modern evolutionary synthesis, is population thinking [1]. Of the oft-cited four forces of evolution (drift, selection, recombination, and mutation), two of these are explicitly linked to populations. And the most commonly used species concepts (reproductive isolation) is also dependent on population-level phenomena. Population thinking has not only provided sufficient explanations for understanding the distribution of natural variation, but has also provided us with advanced computing tools such as genetic algorithms.
Yet any individual organism in a population can exhibit traits that are deviate from the population norm. For example, mutation and recombination occur within individuals, and genetic drift (or neutrality) can often involve very small subpopulations. While the modern synthesis does a very good job of describing phenomena that define a species or variation across phylogeny, I propose that a model that unifies aberrant, individualistic behaviors with more normative and aggregate population-level phenomena is needed.
What is the missing piece then? A natural fit to this is emergence theory, which comes in a number of varieties [2]. One of the main predictions of emergence theory is self-organization, or the notion of order from chaos. Self- organization should be expected in natural populations because individuals act both in competition and collectively to produce unsupervised patterns at the population level. These are the very same patterns that we identify when we say that a population has undergone selection or some other form of differential reproduction.
In his book "The Origins of Order", Stuart Kauffman [3] uses the term "order for free". Order for free refers to the seeming lack of thermodynamic cost to the spontaneous generation of order observed in self-organizing systems. Of course, self-organizing processes must conform to thermodynamic constraints, but nevertheless result in highly ordered patterns. Two common examples of self- organization in biology are morphogenesis (a developmental process - [4]) and insect nest building (a behavioral process - [5]). In the former example, highly-parallel gene expression and intercellular signaling result in a highly patterned and repeatable cellular architecture across organisms. In the latter example, highly-parallel interactions between organisms produces a highly patterned but variable nest architecture across subpopulations in the same species.
Yet any individual organism in a population can exhibit traits that are deviate from the population norm. For example, mutation and recombination occur within individuals, and genetic drift (or neutrality) can often involve very small subpopulations. While the modern synthesis does a very good job of describing phenomena that define a species or variation across phylogeny, I propose that a model that unifies aberrant, individualistic behaviors with more normative and aggregate population-level phenomena is needed.
What is the missing piece then? A natural fit to this is emergence theory, which comes in a number of varieties [2]. One of the main predictions of emergence theory is self-organization, or the notion of order from chaos. Self- organization should be expected in natural populations because individuals act both in competition and collectively to produce unsupervised patterns at the population level. These are the very same patterns that we identify when we say that a population has undergone selection or some other form of differential reproduction.
In his book "The Origins of Order", Stuart Kauffman [3] uses the term "order for free". Order for free refers to the seeming lack of thermodynamic cost to the spontaneous generation of order observed in self-organizing systems. Of course, self-organizing processes must conform to thermodynamic constraints, but nevertheless result in highly ordered patterns. Two common examples of self- organization in biology are morphogenesis (a developmental process - [4]) and insect nest building (a behavioral process - [5]). In the former example, highly-parallel gene expression and intercellular signaling result in a highly patterned and repeatable cellular architecture across organisms. In the latter example, highly-parallel interactions between organisms produces a highly patterned but variable nest architecture across subpopulations in the same species.
Figure 1. Examples of self-organization that involve animal populations. UPPER LEFT: examples of a honeybee hive and a wasp's nest, UPPER RIGHT: example of epidermal morphogenesis in worm (Courtesy, wormbook.org - Chapter on Epidermal Morphogenesis), LOWER LEFT: example of emergent states of activation in the human brain (Courtesy, Figure 2, Chialvo, Nature Physics, 6, 744–750 -- 2010), LOWER RIGHT: example of fish shoaling (Courtesy, Wikipedia).
The question that naturally arises from this is how evolution by natural selection over multiple generations is related to these examples, since development and behavior both shape and constrain evolution. And while the answer is not straighforward, we can learn much from the structure of these examples. The first lesson is that while evolution is a population-dependent process, it is also dependent upon highly-parallel interactions between individuals. We can see this in many of the competitive and cooperative processes that define mating and social interaction. While this may seem to require no paradigm shift, the role of these processes in regulating the aggregate properties of the population is not a consideration of modern theory.
What are the regulatory processes that influence the evolution of a population? To do this, we will take a complex adaptive systems approach [6] to emergence. This implies that there are top-down, bottom- up, and hierarchical components of evolution in addition to the four factors proposed by current theory. An example of a top-down force in evolution is constraint by historical contingency. Historical contingency acts upon all members of a population more or less uniformly, and provides a organizational scheme to developmental and physiological processes such as gene expression. By contrast, a bottom-up force of evolution is embodied in the randomness produced by mutation and recombination. This is brought to the population by individual organisms. This randomness is not totally independent across individuals, but does provide a mechanism for individuality.
Neither top-down nor bottom-up mechanisms are particularly different from what is accounted for in current perspectives on evolution. Yet there is a third component (hierarchy) that unifies top-down and bottom-up components into the context of a complex system. The hierarchical component of evolution by natural selection is related to hierarchical structure of a population. By this I not only mean relationships between individuals in a population, but also the trophic levels of organismal organization (e.g. cells, tissues, organs - [7]). Hierarchical organization, particularly the multiscale nature (e.g. relationship between scales of organization) of evolving populations, is key to driving self- organization in individual animal body [8], and may provide a means to understand variability across instances of emergence in long-term evolution.
In considering the difference between developmental emergence (in which deviations are often deleterious) and social insect emergence (where deviations across nest and hive designs are often observed), evolution is much more like the latter than the former. This may be due to robustness mechanisms specific to biological evolution (e.g. modularity and evolvability) which are somewhat beyond the scope of the current essay, but have interesting implications on the emergence of evolutionary systems.
What are the regulatory processes that influence the evolution of a population? To do this, we will take a complex adaptive systems approach [6] to emergence. This implies that there are top-down, bottom- up, and hierarchical components of evolution in addition to the four factors proposed by current theory. An example of a top-down force in evolution is constraint by historical contingency. Historical contingency acts upon all members of a population more or less uniformly, and provides a organizational scheme to developmental and physiological processes such as gene expression. By contrast, a bottom-up force of evolution is embodied in the randomness produced by mutation and recombination. This is brought to the population by individual organisms. This randomness is not totally independent across individuals, but does provide a mechanism for individuality.
Neither top-down nor bottom-up mechanisms are particularly different from what is accounted for in current perspectives on evolution. Yet there is a third component (hierarchy) that unifies top-down and bottom-up components into the context of a complex system. The hierarchical component of evolution by natural selection is related to hierarchical structure of a population. By this I not only mean relationships between individuals in a population, but also the trophic levels of organismal organization (e.g. cells, tissues, organs - [7]). Hierarchical organization, particularly the multiscale nature (e.g. relationship between scales of organization) of evolving populations, is key to driving self- organization in individual animal body [8], and may provide a means to understand variability across instances of emergence in long-term evolution.
In considering the difference between developmental emergence (in which deviations are often deleterious) and social insect emergence (where deviations across nest and hive designs are often observed), evolution is much more like the latter than the former. This may be due to robustness mechanisms specific to biological evolution (e.g. modularity and evolvability) which are somewhat beyond the scope of the current essay, but have interesting implications on the emergence of evolutionary systems.
Besides acting to regulate the current population, these alternate components of evolution also operate on multiple generations of individuals. However, to observe the emergence of features in long-term processes such as evolution, we must consider a time scale between that of a single reproducing organism and the traditional signatures of evolution by natural selection. Think of emergent natural selection as a series regulatory processes as acting upon a small number of generations. This allows us to see the origins of long-term evolutionary changes.
Understanding the links between ultimate outcomes and proximal events in this way allows us to talk about "evolution for free", a play on the coinage "order for free" and a phrase I have encountered informally amongst colleagues. Specifically, an emergent view allows us to place evolution for free in a less evanescent context. In addition, placing evolution in this emergent context allows us to build sets of models that more explicitly link complex behaviors, brain function, and developmental processes to evolutionary outcomes.
Understanding the links between ultimate outcomes and proximal events in this way allows us to talk about "evolution for free", a play on the coinage "order for free" and a phrase I have encountered informally amongst colleagues. Specifically, an emergent view allows us to place evolution for free in a less evanescent context. In addition, placing evolution in this emergent context allows us to build sets of models that more explicitly link complex behaviors, brain function, and developmental processes to evolutionary outcomes.
Examples from the Nervous System:
One example of how this might be useful is in what is typically referred to as exaptation. Evolution for free might explain the evolution of color vision in primates on top of an existing circuit [9, 10]. This may also be true in cases where the primate color vision system utilizes existing cortical areas for purposes of processing. In terms of a fitness landscape, this could lead to a fitness amplifier, or perhaps an evolutionary “ratchet” that moves a population towards fitness peaks more quickly. Another example may involve the evolution of ocular dominance columns, which self-organize in development but are also similar across phylogenetically-distant taxa.
The authors of [11] refer to the self-organization of ocular dominance columns as canalization, a concept which has many analogies with evolutionary dynamics. Canalization [12] occurs when organisms are constrained to the same developmental pathway, and common developmental pathways are roughly equivalent to canals (hence canalization). These canals might be thought of as a series of minima with respect to energy required or changes in gene expression, or as linkages in an nk-boolean network [13]. This can be short-circuited through stresses such as heat shock, which uncover a lot of deleterious variants. Self-organization, on the other hand, is quite different. Self-organization is related to emergence, which is the production of higher-order patterns from disorderly interactions among cells or organisms (e.g. "stripes" from white noise). The way in which you might get to an emergent structure (such as a termite's nest) is not constrained in "development". Rather, there are many alternate pathways and patterns of interaction to the same structure (in this case, it is the structure which is "canalized", not the means of getting there).
The authors of [11] refer to the self-organization of ocular dominance columns as canalization, a concept which has many analogies with evolutionary dynamics. Canalization [12] occurs when organisms are constrained to the same developmental pathway, and common developmental pathways are roughly equivalent to canals (hence canalization). These canals might be thought of as a series of minima with respect to energy required or changes in gene expression, or as linkages in an nk-boolean network [13]. This can be short-circuited through stresses such as heat shock, which uncover a lot of deleterious variants. Self-organization, on the other hand, is quite different. Self-organization is related to emergence, which is the production of higher-order patterns from disorderly interactions among cells or organisms (e.g. "stripes" from white noise). The way in which you might get to an emergent structure (such as a termite's nest) is not constrained in "development". Rather, there are many alternate pathways and patterns of interaction to the same structure (in this case, it is the structure which is "canalized", not the means of getting there).
You might say that while canalized phenotypes are products of path-dependence (e.g. developmental contingency), self-organized aspects of the phenotype are path-invariant but structure-dependent. In the visual cortex, interactions between inputs might produce an interference pattern that creates spatial boundaries and, yes, maximally efficient patterns of information storage. Much like a box of Neopolitan ice cream (which has NOT undergone canalization), there is competition for space among multiple types of output. As long as those inputs are mapped to a cortical-like structure, self- organization is the predominant driving force.
Figure 2. LEFT: Neopolitan ice cream, an intentionally striped form. MIDDLE: Striping in an ocular dominance column (courtesy of [14]), RIGHT: D-Stat mRNA expression in Drosophila (courtesy of [15]).
Conclusion
I have provided a very rough outline of what I believe to be a necessary component of evolutionary theory largely overlooked by contemporary theorists. It is not so much a matter of being "overlooked" as is a more explicit grounding of complexity theory in the relationship between individuals and populations. There has been much spirited discussion regarding the merits and shortcomings of group vs. individual selection, but that is not what I am proposing here. This alternative view still champions population thinking -- but is done so in a way that does not obscure the role of individualistic, non-normal events that occur in the course of natural history. Based on observations of assortative mating and differential reproduction in nature, we might ask: if a trait is rare in the population is it also rare with regard to the individual? As with most posts on this blog, this is a work in progress. Suggestions for future directions are welcome.
I have provided a very rough outline of what I believe to be a necessary component of evolutionary theory largely overlooked by contemporary theorists. It is not so much a matter of being "overlooked" as is a more explicit grounding of complexity theory in the relationship between individuals and populations. There has been much spirited discussion regarding the merits and shortcomings of group vs. individual selection, but that is not what I am proposing here. This alternative view still champions population thinking -- but is done so in a way that does not obscure the role of individualistic, non-normal events that occur in the course of natural history. Based on observations of assortative mating and differential reproduction in nature, we might ask: if a trait is rare in the population is it also rare with regard to the individual? As with most posts on this blog, this is a work in progress. Suggestions for future directions are welcome.
References:
[1] see Sober, E. (1980). Evolution, population thinking, and essentialism. Philosophy of Science, 47(3), 350-383 for more information on population thinking.
[2] Reid, R.G.B. (2007). Biological Emergences: evolution by natural experiment. MIT Press, Cambridge, MA.
[3] Kauffman, S.A. (1993). Origins of Order: self organization and selection in evolution. Oxford University Press, Oxford, UK.
[4] Wartlick, O., Mumcu, P., Julicher, F., and Gonzalez-Gaitan, M. (2011). Understanding morphogenetic growth control: lessons from flies. Nature Reviews Molecular Cell Biology, 12, 594-604.
[5] Camazine, S., Deneubourg, J-L., Franks, N.R., Sneyd, J., Theraulaz, G., and Bonabeau, E. (1992). Self-Organization in Biological Systems.
[6] Holland, J.H. (1992). Adaptation in Natural and Artificial Systems. MIT Press, Cambridge, MA.
[7] Alicea, B. (2008). Hierarchies of Biocomplexity: modeling life's energetic complexity. arXiv Repository, arXiv:0810.4547.
[8] Hunter, P.J. and Borg, T.K. (2003). Integration from proteins to organs: the Physiome Project. Nature Reviews Molecular and Cellular Biology, 4(3), 237-43.
[9] Surridge, A.K., Osorio, D., Mundy, N.I. (2003). Evolution and selection of trichromatic vision in primates. Trends in Ecology and Evolution, 18(4), 198-205.
[10] Barton, R.A. (2010). Evolutionary specialization in mammalian cortical structure. Journal of Evolutionary Biology, 20(4), 1504-1511.
[11] Kaschube, M., Schnabel, M., Lowel, S., Coppola, D.M., White, L.E., and Wolf, F. (2010). Universality in the Evolution of Orientation Columns in the Visual Cortex. Science, 330, 1113-1116.
[12] Waddington, C.H. (1960). Experiments on canalizing selection. Genetical Research, 1, 140-150.
[1] see Sober, E. (1980). Evolution, population thinking, and essentialism. Philosophy of Science, 47(3), 350-383 for more information on population thinking.
[2] Reid, R.G.B. (2007). Biological Emergences: evolution by natural experiment. MIT Press, Cambridge, MA.
[3] Kauffman, S.A. (1993). Origins of Order: self organization and selection in evolution. Oxford University Press, Oxford, UK.
[4] Wartlick, O., Mumcu, P., Julicher, F., and Gonzalez-Gaitan, M. (2011). Understanding morphogenetic growth control: lessons from flies. Nature Reviews Molecular Cell Biology, 12, 594-604.
[5] Camazine, S., Deneubourg, J-L., Franks, N.R., Sneyd, J., Theraulaz, G., and Bonabeau, E. (1992). Self-Organization in Biological Systems.
[6] Holland, J.H. (1992). Adaptation in Natural and Artificial Systems. MIT Press, Cambridge, MA.
[7] Alicea, B. (2008). Hierarchies of Biocomplexity: modeling life's energetic complexity. arXiv Repository, arXiv:0810.4547.
[8] Hunter, P.J. and Borg, T.K. (2003). Integration from proteins to organs: the Physiome Project. Nature Reviews Molecular and Cellular Biology, 4(3), 237-43.
[9] Surridge, A.K., Osorio, D., Mundy, N.I. (2003). Evolution and selection of trichromatic vision in primates. Trends in Ecology and Evolution, 18(4), 198-205.
[10] Barton, R.A. (2010). Evolutionary specialization in mammalian cortical structure. Journal of Evolutionary Biology, 20(4), 1504-1511.
[11] Kaschube, M., Schnabel, M., Lowel, S., Coppola, D.M., White, L.E., and Wolf, F. (2010). Universality in the Evolution of Orientation Columns in the Visual Cortex. Science, 330, 1113-1116.
[12] Waddington, C.H. (1960). Experiments on canalizing selection. Genetical Research, 1, 140-150.
[13] Bassler, K.E., Lee, C. and Lee, Y. (2004). Evolution of developmental canalization in networks of competing boolean nodes. Physical Review Letters, 93(3), 038101.
[14] Stiles, J. and Jernigan, T.L. (2010). The Basics of Brain Development. Neuropsychology Review, 20(4).
[15] Yan, R., Small, S., Desplan, C., Dearolf, C.R., Darnell, J.E. (1996). Identification of a Stat gene that functions in Drosophila development. Cell, 84(3), 421-430.
[14] Stiles, J. and Jernigan, T.L. (2010). The Basics of Brain Development. Neuropsychology Review, 20(4).
[15] Yan, R., Small, S., Desplan, C., Dearolf, C.R., Darnell, J.E. (1996). Identification of a Stat gene that functions in Drosophila development. Cell, 84(3), 421-430.
February 21, 2012
Official Host of the Carnival of Evolution #46
Hooray! Synthetic Daisies will be the official host for the Carnival of Evolution (CoE) #46. The publish date is April 1 (April Fools' Day). Carnival of Evolution features a wide range of submissions in the area of biological evolution (although cultural evolution, evolutionary psychology, biomimetics, and evolutionary computing would also be welcome). Please submit your relevant blog posts (dated March 1-March 31, 2012) here.
February 3, 2012
Hard-to-define Events Workshop
I just found out that the workshop I am trying to organize for Artificial Life XIII on hard-to-define phenomena was accepted by the program committee. Hard-to-define events are events in a complex system that set up major transitions or obvious features. Signatures of hard-to-define events are related to natural fluctuations, embedded patterns, and rare events of large magnitude. Unlike the underlying patterns and information revealed by machine learning techniques, hard-to-define techniques require alternative approaches not yet formalized.
It should be a good session, but I need to procure a lineup of participants. I am currently recruiting people to present their work in the context of hard-to-define events: the plan is to think about how one's research might involve hard-to-define events, and then consider how we might design analytical tools and/or measurement techniques to discover them.
I am interested in having people participate from any number of fields. Of particular interest is how this idea might apply to the biological, cognitive, and social sciences. I have launched a webpage devoted to the latest news on the session. Please check it out, and if you are interested in participating contact me for more information. If you are in the Midwest on the weekend of July 19-22, you should try to attend (see previous blog post for more information).
It should be a good session, but I need to procure a lineup of participants. I am currently recruiting people to present their work in the context of hard-to-define events: the plan is to think about how one's research might involve hard-to-define events, and then consider how we might design analytical tools and/or measurement techniques to discover them.
I am interested in having people participate from any number of fields. Of particular interest is how this idea might apply to the biological, cognitive, and social sciences. I have launched a webpage devoted to the latest news on the session. Please check it out, and if you are interested in participating contact me for more information. If you are in the Midwest on the weekend of July 19-22, you should try to attend (see previous blog post for more information).January 28, 2012
Representing rare events
In this post I will be discussing the occurrence of rare events [1] and how they might be represented in computational systems. Rare events are intriguing precisely because they occur infrequently. We are all familiar with rare events in nature: avalanches, rogue waves, freak storms, and developmental mutations are but a few examples. Furthermore, while these events are rare, they are not improbable. I posted a few months back on the context of such processes (overproductive systems) and their potential relevance to biology. While the occurrence of rare events themselves is sparsely distributed in time, they occur against a background process filled with many events we like to call "normal". This is related to a normal distribution: background events occur within a certain range of expected outcomes.
One way in which rare events have been addressed from a statistical standpoint is to model them as a distribution of noise. In processes where we expect no rare events, we can assume that events will unfold according to a white noise distribution [2]. For processes where these rare events occur at increasingly larger magnitudes, a model of colored (or 1/f) noise can be used to model the expectation of rare events embedded in a process. So-called pink noise provides a series of rare events with a near-uniform size many orders of magnitude above the size of background events, while black noise provides a series of rare events occurring against a background of near-inactivity.
| The comparison of extreme 1/f noise to white noise can be made using a thought experiment. Suppose you enjoy sleeping with a white noise machine running in the background. Why do you enjoy this? One reason might be that a white noise machine provides a stochastic but uniform auditory stimulus that allows your relax enough to fall asleep. Yet suppose that you wanted to awake from your slumber. Most people use an alarm clock, which presents bursts of auditory stimuli at different magnitudes, depending on your preference.
This transition from a uniform stream of sound to a series of bursts embedded in a more uniform background is the transition from common events to rare events. Not only do rare events capture your attention, but they also provide a basis for large-scale transitions in coupled processes. In this case, your alarm clock is coupled to your sensory systems, which force you to wake up abruptly. |
One also can think of a noisy distribution as the null model for the expectation and occurrence of rare events. Yet during a given natural process, not only are rare events expected to occur at a certain rate, but also recur at a certain rate. This is because rare events tend to be stochastic, and as a result are not evenly distributed in time. This connection to recursion suggests that such events are computable, if we only had the proper data structure for these events.
From the time of the difference engine to today, representing something for a computational system has involved using a discrete binary model that conforms to a logical data structure (e.g. logic gate, tree, matrix). Yet "natural" computation (or computable behavior) has been observed in systems such as chemical (B-Z and Turing) reactions [3], collective behavior, and gene expression, which may be neither discrete and binary nor explicitly logical. However, one could argue that rare and extreme events occur in such systems, and that it is a phenomenon very poorly captured by contemporary hardware and software.
LEFT: an example of a B-Z reaction. RIGHT: examples of reaction-diffusion seen in so-called Turing reactions (a key mechanism in biological development).
Why aren't contemporary data structures suitable for what I am describing? It is true that much of the work on fractal geometry was made possible using standard, existing computing logic [4]. Yet most of modern artificial intelligence and machine learning is based on the notion of normalization. For example, in machine learning and pattern recognition, a series of exemplars are used to identify discrete groups and isolated objects. A basis function used to delimit these groups is often based on the normal distribution, and the margin of these functions are generally associated with a deterministic amount of variance. This leads to a high rate of true positives for classifying objects, but can lead to catastrophic failures from time to time.
An alternate approach is to use a genetic algorithmic or artificial life representation. While these techniques are capable of generating rare and/or extreme events (through mutation and recombination), we must still understand the context of rare events. By observing systems such as rogue waves and avalanches, we can see that fluctuations are an important ingredient for generating rare events. Thus, systems that exhibit flux (where there are undulations in the acceleration and movement of individual particles) should be abstracted to a formal data structure in order for commercially-available computers to completely model rare events. Incorporating these features into a genetic algorithm or artificial life application would greatly help us understand natural instances of non-uniformity.
But can this even be done using conventional computational abstractions? It is hard to say. I have thought about this problem, and remain convinced that we need fundamentally new forms of computation to truly address these features of nature. One option might be "natural" computing, which is the use of natural phenomena (such as chemical reactions or bubble formation) to execute computations [5]. Another option might be in the area of biomimetics, where the goal is to abstract functional features from biological and other natural systems for purposes of computing and developing engineering applications.
An interesting area indeed. More news as it develops.
References
[1] Rare events have been characterized by the Black Swan phenomenon and the Poisson distribution.
[2] A white noise distribution is a normally-distributed Gaussian process.
[3] Adamatzky, A. (2001). Computing in Nonlinear Media and Automata Collectives. Institute of Physics, Bristol, UK.
[4] Mandelbrot, B. (1982). The Fractal Geometry of Nature. W.H. Freeman, New York.
[5] I highly recommend checking out Andrew Adamatsky's work on reaction-diffusion computing and Manu Prakash's work on microfluidic logic using bubbles.
Subscribe to:
Posts (Atom)
