NOTES:
[1] shown here is a short animation of the process. I tend to be a bit more nonlinear when writing research papers, so this is pretty impressive.
[2] Courtesy of Timothy Weninger, UIUC.
November 28, 2012
Research Paper "Emergence"
NOTES:
[1] shown here is a short animation of the process. I tend to be a bit more nonlinear when writing research papers, so this is pretty impressive.
[2] Courtesy of Timothy Weninger, UIUC.
November 27, 2012
Topological references, courtesy of Futurama
I am re-posting this from my microblog Tumbld Thoughts, as it has been one of my most popular posts there. The theme fits in well with this blog as well.
Currently catching up on TV. In the Futurama episode "Benderama", there was a reference to the Banach-Tarski paradox (in the form of a duplicator machine) [1]. The machine is used to duplicate things (ultimately Bender) at progressively smaller scales [2]. All this in 23 minutes. Great stuff.
NOTES:
[1] the point was not to be technically accurate. The point was to make an analogical and highly obscure reference so that people like me could talk about it.
[2] the "shrinker" eventually leads to the potential destruction of the world's drinking supply (nice subreference to the gray goo hypothesis/scenario in the field of nanotechnology).
November 21, 2012
Artificial Life meets Geodynamics (EvoGeo)
For the past few years, I have been working on a novel approach to modeling biological evolution. It utilizes a computational tool from fluid mechanics called Lagrangian Coherent Structures (LCS). This method was developed by George Haller [1] at ETH Zurich, and has been previously used in biology to study the biomechanics of locomotion in fluids [2] and the relationship between predation and ocean currents
[3]. I presented on this at ALife XIII (Contextual Geometric Structures - CGS, [4]) and have a related paper (LCS-like structures, [5]) on the arXiv.
The premise of this model is simple: a population of automata, with either a neural (CGS) or genomic (LCS-like) representation, diffuses across an n-dimensional flow field. As they diffuse according to local flow conditions (which can mimic environmental selection), they form various clusters called coherent structures. While they can be identified using qualitative means, there are actually highly-complex differential equations that can be used to estimate evolutionary distance and perhaps even reconstruct evolutionary trajectories [6]. However, much more work needs to be done on the simulation capabilities of this hybrid [7] evolutionary model, and as of now exists as somewhere between conceptual metaphor and bona-fide simulation.
Figure 2. LEFT: Internal representations for automata in the CGS paper. RIGHT: Internal representations for automata in the LCS-paper.
In my application, some of the equations and data structures are borrowed, and some are uniquely "evolutionary". This allows me flexbility in terms of applying the model to many different kinds of evolutionary scenarios. One of these (identified in the LCS-like paper) is biogeography, particularly island biogegraphy. For the uninitiated, biogeography [8] is the study of population processes in a geographic context. As organisms migrate and geomorphology changes, population genetics and demography are affected in corresponding fashion.
In my LCS-inspired evolutionary model, recall that the environment consists of a generic flow field (and in CGS paper, this is already exploited as a quasi-geography). If this substrate could be replaced by a dynamic topology (e.g. a more explicit geography), the LCS-inspired model might provide insights into evolutionary "deep" time. When I say deep time, I mean a period of time long enough for uplift, continental drift, and seafloor spreading to occur and effect the distribution of populations and species.
How do we go about establishing this dynamic topology? There are a number of options here, two of which I will discuss in detail. The first is the terraforming engine used in virtual worlds such as SimCity, Spore, and Second Life [9]. The second involves using a mathematical tool called plate motion vectors to predict tectonic drift [10]. Figure 3 shown examples of each. While this has not been formally worked out, the basic goal is to create kinematics (underlying movements that govern environmental constraints) much as the quasi-flow regimes provide in the CGS and LCS-like models.
Once the kinematics of geomorphology have been established as an evolutionary field, the kinetics (e.g. how these movements unfold in time) must then be accounted for. A model of plate tectonics (the Burridge-Knopoff model, [11]) can be used to approximate tectonic activity as a series of sliding blocks that interact according to local rules. A simpler method might be to represent the buildup and release of stresses between tectonic units as a non-uniform probability density function (PDF). In any case, examples of the concept on a three-dimensional space can be seen in Figure 4.
Figure 4. Single generation examples of organisms (automata, black balls) and populations (clusters of black balls) on a dynamic topology. LEFT: Two separate landmasses, one with mountains. UPPER RIGHT: Newly-uplifted mountains and an isthmus (land bridge). LOWER RIGHT: Pre-uplifted mountains and continental drift. Simulated using pseudo-data.
What I am describing, then, is a three-level hybrid model (see Figure 5): a geomorphological model to partition and add dimensionality to the underlying substrate of evolution, a general model of particle diffusion (genetic drift), and a genomic representation that provide diversity and relatedness to the automata population. This is only a quick sketch from the Fluid Models of Evolutionary Dynamics project -- if you interested in collaborating or otherwise helping develop this model, let it be known. Other comments are also welcome.
[1] Haller, G. (2007). Uncovering the Lagrangian Skeleton of Turbulence. Physical Review Letters, 98, 144502. Here is a tutorial.
[2] Nawroth, J.C., Feitl, K.E., Colin, S.P., Costello, J.H., and Dabiri, J.O. (2010). Phenotypic plasticity in juvenile jellyfish medusae facilitates effective animal-fluid interaction. Biology Letters, 6(3), 389-393.
[3] Tew Kai, E., Rossi, V., Sudre, J., Weimerskirch, H., Lopez, C., Hernandez-Garcia, E., Marsac, F., and Garcon, V. (2009). Top marine predators track Lagrangian coherent structures. PNAS, 106(20), 8245–8250.
[4] Alicea, B. Contextual Geometric Structures: modeling the fundamental components of cultural behavior.
Proceedings of Artificial Life, 13, 147-154 (2012).
[5] Alicea, B. Lagrangian Coherent Structures (LCS) may describe evolvable frontiers in natural populations.
arXiv Repository, arXiv:1101.6071 [nlin.AO, physics.bio-ph, q-bio.PE] (2011).
Supplementary Information for [4] and [5], and slides for [4].
[6] For examples, please see: Lipinski, D. and Mohseni, K. (2010). A ridge tracking algorithm and error estimate for efficient computation of Lagrangian coherent structures. Chaos, 20, 017504.
[8] There are many relevant reviews. For an example, please see: Ronquist, F. and Sanmartin, I. (2011). Phylogenetic Methods in Biogeography. Annual Review of Ecology, Evolution, and Systematics, 42, 441-464.
[9] Here is the terraforming and ground textures documentation page from Second Life support. They use something called a .raw file, which converts a height field into a georeferenced surface (similar to digital elevation maps or digital elevation models).
[10] For more information on plate motion vectors, please see this link to "Teaching Geosciences in the 21rst Century". For tutorial on measuring relative plate motion, please see this tutorial. For tectonic drift methods (e.g. plate motion vectors) used to create predictions (maps) of continental drift, see the research of Ronald Blakely, Northern Arizona University.
[11] Rundle, J.B., Tiampo, K.F., Klein, W., and Sa Martins, J.S. (2002). Self-organization in leaky threshold systems: The influence of near-mean field dynamics and its implications for earthquakes, neurobiology, and forecasting. PNAS, 99(S1), 2514-2521.
[12] An important distinction: the surfaces shown in Figure 4 are not traditional fitness landscapes. Rather, these surfaces are based on elevation, which can change as the model evolves. Any fitness parameters are calculated independently of elevation, although they can be effected by elevation (e.g. uplift) or general topographic changes (new mountains, seafloor).
[3]. I presented on this at ALife XIII (Contextual Geometric Structures - CGS, [4]) and have a related paper (LCS-like structures, [5]) on the arXiv.
The premise of this model is simple: a population of automata, with either a neural (CGS) or genomic (LCS-like) representation, diffuses across an n-dimensional flow field. As they diffuse according to local flow conditions (which can mimic environmental selection), they form various clusters called coherent structures. While they can be identified using qualitative means, there are actually highly-complex differential equations that can be used to estimate evolutionary distance and perhaps even reconstruct evolutionary trajectories [6]. However, much more work needs to be done on the simulation capabilities of this hybrid [7] evolutionary model, and as of now exists as somewhere between conceptual metaphor and bona-fide simulation.
Figure 1. RIGHT: An example of LCS models as applied to fluid dynamics. LEFT: An example of the LCS-inspired model (from CGS paper).
In my application, some of the equations and data structures are borrowed, and some are uniquely "evolutionary". This allows me flexbility in terms of applying the model to many different kinds of evolutionary scenarios. One of these (identified in the LCS-like paper) is biogeography, particularly island biogegraphy. For the uninitiated, biogeography [8] is the study of population processes in a geographic context. As organisms migrate and geomorphology changes, population genetics and demography are affected in corresponding fashion.
In my LCS-inspired evolutionary model, recall that the environment consists of a generic flow field (and in CGS paper, this is already exploited as a quasi-geography). If this substrate could be replaced by a dynamic topology (e.g. a more explicit geography), the LCS-inspired model might provide insights into evolutionary "deep" time. When I say deep time, I mean a period of time long enough for uplift, continental drift, and seafloor spreading to occur and effect the distribution of populations and species.
How do we go about establishing this dynamic topology? There are a number of options here, two of which I will discuss in detail. The first is the terraforming engine used in virtual worlds such as SimCity, Spore, and Second Life [9]. The second involves using a mathematical tool called plate motion vectors to predict tectonic drift [10]. Figure 3 shown examples of each. While this has not been formally worked out, the basic goal is to create kinematics (underlying movements that govern environmental constraints) much as the quasi-flow regimes provide in the CGS and LCS-like models.
Once the kinematics of geomorphology have been established as an evolutionary field, the kinetics (e.g. how these movements unfold in time) must then be accounted for. A model of plate tectonics (the Burridge-Knopoff model, [11]) can be used to approximate tectonic activity as a series of sliding blocks that interact according to local rules. A simpler method might be to represent the buildup and release of stresses between tectonic units as a non-uniform probability density function (PDF). In any case, examples of the concept on a three-dimensional space can be seen in Figure 4.
Figure 3. LEFT: Examples of terraforming (SimCity) in a virtual world. RIGHT: Predictions of tectonic drift for the Jurassic period [see 9].
In an artificial life context, a fitness function is used to disallow diffusion of organisms and thus gene flow across cells that are either too high or too low (representing oceans and mountains, respectively). Since there is a geodynamic component to the model, these fitnesses and regions colonized can both change over time [12]. Organisms can colonize a patch of land that becomes an island or valley over time, only to be cut off from the main population with uplift or ocean floor spreading. By contrast, members of the population can evolve the ability to either live in or traverse these new environments, leading to some potentially interesting scenarios. It should be noted that agents (automata) in the LCS-inspired models reproduce asexually and have unique evolutionary dynamics, so any conclusions raised using this model should not necessarily be extrapolated to paleobiological scenarios.
Figure 5. Diagram of the three-level model structure in context.
References and Notes:
[2] Nawroth, J.C., Feitl, K.E., Colin, S.P., Costello, J.H., and Dabiri, J.O. (2010). Phenotypic plasticity in juvenile jellyfish medusae facilitates effective animal-fluid interaction. Biology Letters, 6(3), 389-393.
[3] Tew Kai, E., Rossi, V., Sudre, J., Weimerskirch, H., Lopez, C., Hernandez-Garcia, E., Marsac, F., and Garcon, V. (2009). Top marine predators track Lagrangian coherent structures. PNAS, 106(20), 8245–8250.
[4] Alicea, B. Contextual Geometric Structures: modeling the fundamental components of cultural behavior.
Proceedings of Artificial Life, 13, 147-154 (2012).
[5] Alicea, B. Lagrangian Coherent Structures (LCS) may describe evolvable frontiers in natural populations.
arXiv Repository, arXiv:1101.6071 [nlin.AO, physics.bio-ph, q-bio.PE] (2011).
Supplementary Information for [4] and [5], and slides for [4].
[6] For examples, please see: Lipinski, D. and Mohseni, K. (2010). A ridge tracking algorithm and error estimate for efficient computation of Lagrangian coherent structures. Chaos, 20, 017504.
[7] Fromentin, J., Eveillard, D., and Roux, O. (2010). Hybrid modeling of biological networks: mixing temporal and qualitative biological properties. BMC Systems Biology, 4, 79.
[10] For more information on plate motion vectors, please see this link to "Teaching Geosciences in the 21rst Century". For tutorial on measuring relative plate motion, please see this tutorial. For tectonic drift methods (e.g. plate motion vectors) used to create predictions (maps) of continental drift, see the research of Ronald Blakely, Northern Arizona University.
[11] Rundle, J.B., Tiampo, K.F., Klein, W., and Sa Martins, J.S. (2002). Self-organization in leaky threshold systems: The influence of near-mean field dynamics and its implications for earthquakes, neurobiology, and forecasting. PNAS, 99(S1), 2514-2521.
[12] An important distinction: the surfaces shown in Figure 4 are not traditional fitness landscapes. Rather, these surfaces are based on elevation, which can change as the model evolves. Any fitness parameters are calculated independently of elevation, although they can be effected by elevation (e.g. uplift) or general topographic changes (new mountains, seafloor).
November 14, 2012
Paper of the Week (Detecting Causality)
Figure 1. Three instances of mirage correlations between two time series (which can occur in a range of systems, from ecological to financial). COURTESY: Figure 1 in [2].
However, in cases where coupling between the two systems under analysis is weak [6], a new approach called Convergent Cross Mapping (CCM) can be used to detect causality. The CCM approach measures the extent to which the historical record (e.g. a prior time-series) can predict a current time-series [see 7]. This approach relies on the principle of cross-prediction: the current time-series must causally influence the prior time series via feedback or transitive couplings (see Figure 2). The success of this approach also depends heavily on convergence within complex systems [8] and the ability to reconstruct the state space for both time-series (Figure 3) using historical and current information [9].
Figure 2. Cases and examples of coupling between dynamical systems and/or variables. COURTESY: Figure 4 in [2].
Figure 3. LEFT: Example of the CCM method for three time-series sharing the same attractor basin manifold. RIGHT: an example of the Simplex projection method (see notes [3] and [9]).
In my opinion, this is a very interesting and perhaps even landmark paper. You should read it and save it to your Mendeley (or similar application) library immediately.
NOTES:
[2] Formal citation: Sugihara, G., May, R., Ye, H., Hsieh, C-H., Deyle, E., Fogarty, M., and Munch, S. (2012). Detecting Causality in Complex Ecosystems. Science, 338, 496.
[3] Both are legends in the field of ecology. Figure 3 shows an example of their previously introduced (and theory-based) method called Simplex analysis.
[4] one example: an n-dimensional phase space trajectory such as a Lorentz attractor.
[6] MATLAB code for Granger causality can be found here (basic analysis) and here (toolbox for inferring network connectivity).
[7] What type of complex systems are most amenable to the CCM method? Nonseperable, weakly connected dynamical systems. Which are, according to the authors, something beyond the scope of (linear) Granger causality analysis.
[8] convergence is to be contrasted with Lyapunov divergence (characterized by an exponent), where two systems begin at the same initial condition and diverge over time.
[9] YouTube animation of this process from the Sugihara Lab. In implementing the CCM method, an algorithm based on the simplex projection is used to generate a nearest-neighbor solution for kernel density estimation is used. More details can be found in the Supplementary Materials.
November 5, 2012
5 years or 20,000 visits, whichever comes first....
Five years ago next month, I founded Synthetic Daisies. The name [1] is based on a merger of my interest in artificial and synthetic life (Synthetic) with my interest in systems thinking (Daisies, based on the Daisyworld experiments of James Lovelock). And yet historically at least, the blog have covered much more. The first post was a review of the book “The Complementary Nature”, followed by two sets of posts: one post on non-optimality [2] and another on YouTube videos from the grand opening of Dickinson Hall at the University of Florida [3]. Since then, I have posted on a mix of biological, computational, and innovation-related topics (broadly construed). I try to keep it balanced between these three areas, with some pop culture and geeky technology fun thrown in.
My blogging style has evolved since starting Synthetic Daisies. For one thing, I have optimized the template design and layout towards something that is attractive to look at as it is to read (not sure I have totally succeeded at this, but I’ve tried). I have also used a number of devices to convey what is sometimes highly complicated information to a general audience. One of these is a preponderance of diagrams and images. Another is a moving a lot of technical detail to footnotes. Being true to the interactive nature of a blog, I sometimes update posts with retrospective information (after the original event or thoughts that inspired the post). This is a good way to keep your posts from becoming "archival" (and perhaps even embarrassing) in feel after a few months or years.
After posting sporadically for the first three years, I made the commitment to keep up the frequency of new posts (at least one post every 1-2 weeks). If you are starting a blog, don’t be afraid to post a large number of posts without immediate readership or feedback. If it is good and you promote them, the traffic will follow. I must also say that blogging on a regular basis has improved my writing skills. Although it takes a lot of effort, I hope that people find this blog both useful and entertaining.
Not bad for the first 4.917 years. However, according to the blogspot.com analytics, it appears that I have reached 20,000 [4] views (this includes all posts and pages). Many of these views are due to a handful of posts, as the number of views per post follows a power-law distribution [5]. One thing that helped my blog along is participating in the Carnival of Evolution (CoE) [6]. Another helper in getting readership is thinking about innovative topics to cover (e.g. thinking outside the blog template, so to speak). I also publicize my posts on Facebook, Tumblr [7], and my research website where appropriate. Finally, I have integrated the blog with my research projects and teaching activities, which helps along the topical innovation.
[1] See the explanation page for more information.
[2] This was a very rough idea at the time. It is also quite unconventional. In engineering and economics, there are whole subfields, library stacks, and conferences devoted the notion of “optimization”. However, in a number of fields (such as Molecular Biology, Evolutionary Ecology, and Cultural Anthropology), models of optimality do not explain the data well. I eventually worked this idea into an arXiv paper called “The ‘Machinery’ of Biocomplexity: understanding non-optimal architectures in biological systems”.
[3] Dickinson Hall was opened in the 1960s as the original home of the Florida Museum of Natural History (FLMNH). It is now used as a research facility and to house the collections not on exhibition. Link to post.
[4] To be precise, probably sometime tomorrow. Most of these visits are legitimate (e.g. not spambots or other redundant counts). What people are getting out of each visit, however, is not known.
[5] Notably, the traffic with respect to time is bursty, especially on a day-to-day basis (as expected).
[6] I hosted CoE #46 with the theme of evolutionary trees (which has earned roughly 2500 views). See it here. I plan to host again next year. CoE is the longest-running blog carnival (which is a monthly review of the blogosphere for a certain topic). Thanks to The Genealogical World of Phylogenetic Networks for this analysis.
[7] I keep a Tumblr site (Tumbld Thoughts) for some of my shorter concepts, observations, and sets of hyperlinks. This “microblogging” platform is particularly good for this purpose. Analytics are currently being collected.
[8] Rank order distribution plotted on a log-log plot. The distribution of views per post follows a power-law distribution, assuming that all traffic flow to the site over time is a Poisson process.......you know, all the good stuff.......
Images from the first Synthetic Daisies posts (using current template). LEFT: Complementary Nature review, CENTER: non-optimality post, RIGHT: YouTube videos of the Dickinson Hall opening.
My blogging style has evolved since starting Synthetic Daisies. For one thing, I have optimized the template design and layout towards something that is attractive to look at as it is to read (not sure I have totally succeeded at this, but I’ve tried). I have also used a number of devices to convey what is sometimes highly complicated information to a general audience. One of these is a preponderance of diagrams and images. Another is a moving a lot of technical detail to footnotes. Being true to the interactive nature of a blog, I sometimes update posts with retrospective information (after the original event or thoughts that inspired the post). This is a good way to keep your posts from becoming "archival" (and perhaps even embarrassing) in feel after a few months or years.
After posting sporadically for the first three years, I made the commitment to keep up the frequency of new posts (at least one post every 1-2 weeks). If you are starting a blog, don’t be afraid to post a large number of posts without immediate readership or feedback. If it is good and you promote them, the traffic will follow. I must also say that blogging on a regular basis has improved my writing skills. Although it takes a lot of effort, I hope that people find this blog both useful and entertaining.
An example of the previous blog template (circa May 2012).
Not bad for the first 4.917 years. However, according to the blogspot.com analytics, it appears that I have reached 20,000 [4] views (this includes all posts and pages). Many of these views are due to a handful of posts, as the number of views per post follows a power-law distribution [5]. One thing that helped my blog along is participating in the Carnival of Evolution (CoE) [6]. Another helper in getting readership is thinking about innovative topics to cover (e.g. thinking outside the blog template, so to speak). I also publicize my posts on Facebook, Tumblr [7], and my research website where appropriate. Finally, I have integrated the blog with my research projects and teaching activities, which helps along the topical innovation.
Power-law-distribution [8] of views per post (left) and time-series of views for all time and by week (top right, subsample) and all-time (lower right).
The ten most viewed posts and their rates of accession (data collated on 11/5/2012).
Post
Title
|
Views
|
Viewing
Rate (per day)
|
2522
|
11.57
|
|
814
|
3.21
|
|
702
|
2
|
|
614
|
2.15
|
|
297
|
1.16
|
|
293
|
0.79
|
|
292
|
0.93
|
|
277
|
0.43
|
|
267
|
5.45
|
|
264
|
0.88
|
* = featured in an edition of CoE.
NOTES:[1] See the explanation page for more information.
[2] This was a very rough idea at the time. It is also quite unconventional. In engineering and economics, there are whole subfields, library stacks, and conferences devoted the notion of “optimization”. However, in a number of fields (such as Molecular Biology, Evolutionary Ecology, and Cultural Anthropology), models of optimality do not explain the data well. I eventually worked this idea into an arXiv paper called “The ‘Machinery’ of Biocomplexity: understanding non-optimal architectures in biological systems”.
[3] Dickinson Hall was opened in the 1960s as the original home of the Florida Museum of Natural History (FLMNH). It is now used as a research facility and to house the collections not on exhibition. Link to post.
[5] Notably, the traffic with respect to time is bursty, especially on a day-to-day basis (as expected).
[6] I hosted CoE #46 with the theme of evolutionary trees (which has earned roughly 2500 views). See it here. I plan to host again next year. CoE is the longest-running blog carnival (which is a monthly review of the blogosphere for a certain topic). Thanks to The Genealogical World of Phylogenetic Networks for this analysis.
[8] Rank order distribution plotted on a log-log plot. The distribution of views per post follows a power-law distribution, assuming that all traffic flow to the site over time is a Poisson process.......you know, all the good stuff.......
November 1, 2012
Merging electronics and biology: the future of touch
The sense of touch, or haptic perception, is a key feature of animal behavior. For humans, haptic perception and proprioception is a subtle but fundamental sensory modality that helps us move, plan, and interact with our environment. Recent developments in virtual environments and conformal electronics may allow us to build artificial sensory systems that fully integrate into the user’s neural function. But first, we must bring together a number of emerging themes in the scientific literature.
Multisensory Perception and the Sensory Milieu
To understand how the nervous system will deal with inputs from artificial touch sensors, we need to understand how the touch systems of the body behave when perturbed. Virtual reality systems specialized for creating sensory illusions are needed to uncover how visual and haptic information are collated at the body’s surface and merged in the brain (for example, see Figure 1). The integration of visual and touch cues can be explored in an active touch task [1]. In this example (Figure 1), force feedback is provided that corresponds with presented visual stimuli. Multisensory integration can also be explored and decoupled by examining changes in postural sway [1a] when the frequency of visual and touch cues are incongruent [2]. The integration of these signals in the mammalian brain is a stochastic process, involving integration sites such as the superior colliculus (SC) and posterior parietal cortex (PPC) [2a].
Studies examining the cell biology of cutaneous touch receptors at the skin surface have found that different populations of cells have different thresholds for activation [3], which is likely related to the complexity of information that the touch sensory modality provides. Much as the brain abstracts away the key pieces of information needed to perform specific tasks, robotic models have also been constructed to understand the core contributions of touch to multisensory perception [4]. This issue of minimal bandwidth requirements will be an important consideration in the design of future artificial touch systems. While relatively low amounts of information may actually be required for straightforward tasks (such as picking up a cup), a much greater amount of information (and perhaps noise as well) may be required for outcomes such as self-awareness or balance.
Multisensory Perception and the Sensory Milieu
To understand how the nervous system will deal with inputs from artificial touch sensors, we need to understand how the touch systems of the body behave when perturbed. Virtual reality systems specialized for creating sensory illusions are needed to uncover how visual and haptic information are collated at the body’s surface and merged in the brain (for example, see Figure 1). The integration of visual and touch cues can be explored in an active touch task [1]. In this example (Figure 1), force feedback is provided that corresponds with presented visual stimuli. Multisensory integration can also be explored and decoupled by examining changes in postural sway [1a] when the frequency of visual and touch cues are incongruent [2]. The integration of these signals in the mammalian brain is a stochastic process, involving integration sites such as the superior colliculus (SC) and posterior parietal cortex (PPC) [2a].
Studies examining the cell biology of cutaneous touch receptors at the skin surface have found that different populations of cells have different thresholds for activation [3], which is likely related to the complexity of information that the touch sensory modality provides. Much as the brain abstracts away the key pieces of information needed to perform specific tasks, robotic models have also been constructed to understand the core contributions of touch to multisensory perception [4]. This issue of minimal bandwidth requirements will be an important consideration in the design of future artificial touch systems. While relatively low amounts of information may actually be required for straightforward tasks (such as picking up a cup), a much greater amount of information (and perhaps noise as well) may be required for outcomes such as self-awareness or balance.
Figure 1. Experiment demonstrating the importance of coordinating and
integrating vision and touch in exploration of unfamiliar objects. COURTESY: Figure 1 of [1].
The sensory milieu refers to myriad interactions between the major sensory systems of the brain (e.g. visual, touch, auditory) and the physiological mechanisms which underlie perception and action. The term "milieu" implies that homeostatic mechanisms are at play in these processes. Considering the emergent order and role of autoregulation among sensory inputs during integration and information processing is a novel way to view the neurobiology of sensation and perception. However, it may be useful to the design and maintenance of artificial systems for two reasons:
1) while it is clear what environmental information is required for statistically optimal perceptual performance [4a], in engineering contexts we may want to achieve sub-optimal performance or robust responses.
2) the perturbation (e.g. decoupling of visual and haptic information) of a multisensory system does not result in a total loss of perceptual integration, particularly over time. Therefore, there must be some type of homeostatic mechanism at play with respect to incoming sensory information [4b].
Artificial Touch is a Matter of Compliance
The interaction with the body’s surface is another key feature of artificial touch systems. To this extent, research that characterizes interactions between the physical properties of surfaces and touch systems is important (see Figure 2). Recent investigations into interactions between the epidermal ridges on the fingertip and the peaks and troughs of a rough surface show that surface forces become more variable with increases in roughness [5]. This leads to an increase in friction during sliding and other fine manipulation behaviors. This has consequences not only for the design of artificial touch systems, but for the tribological design of materials at multiple scales as well [5a].
Figure 2. The interaction between surface reaction forces and the dynamic biomechanics of touch (e.g. the interaction between the finger’s skin and the surface roughness). COURTESY: Figure 1 of [5].
Artificial touch systems require a range of novel material types. In robotics, it has become increasingly clear that the use of compliant (e.g. soft, biocompatible) materials are required for effective interaction [6]. Figure 3 demonstrates how the results shown in Figure 2 can be modulated through the manipulation of surface texture. By controlling the scale and geometry of surface features, friction due to sliding can be minimized. Looking forward (and to animal models), specialized surface features at multiple size scales can enable quite amazing functional behaviors [7].
Figure 3. An example of how surface texture can modulate surface reaction forces encountered at the fingertip. The phenomenon leads to our perception of different surface qualities (e.g. edges, orientation, smoothness). COURTESY: Figures 1 and 2 in [8].
In [8], an experiment using a sliding cantilever and a patterned elastomer demonstrated that the sliding motion of a fingertip can act to modulate the surface reaction force encountered during finger strike. The forces involved are small relative to forces encountered while walking or running [9]. However, the fingertip encounters strains which, when coupled with the complex curves of fingertip geometry, makes for an interesting design challenge. What is needed is a thin film that conducts electrical potentials, and that can bend or buckle without damage to the surface. The group in [10] have developed a carbon nanotube (CNT)-based solution to this problem. Embedded in their thin film piezoelectric device are CNT structures which store strain forces applied to the surface like a spring - and releases this potential after loading. The result is a stretchable capacitor that has the characteristics of a biological viscoelastic (e.g. skin-like) surface.
Figure 4. An summary of piezoelectric characterization for an electronic sensor device that covers the hand. COURTESY: Figure 5 in [11].
Figure 4 shows how such a thin, stretchable film might be used as a skin surrogate. The authors of [11] have engaged in extensive characterization of these types of surfaces. When the thumb is bent, a piezoelectric signal is generated that could be interfaced with the nervous system.
Conformal Electronics
In the past few years, compliant materials have been effectively used as a substrate for nanoelectronic arrays. One of the leading innovators in this field is the Rogers group at UIUC, who have published a number of papers on fabricating (see Figures 5 and 6) and demonstrating the function (Figure 7) of these materials. One commonly used fabrication technique involves the use of the silicon polymer Polydimethylsiloxane (PDMS) as a substrate. PDMS has many desirable properties for a device that will be embedded into the human body. That being said, there are other (and perhaps better) organic materials that could be used as a substrate. However, PDMS is the most common. For a presentation on the promise of tissue fabrication and potential methods, see [12].
Figure 5. An overview of how conformal electronics are fabricated. COURTESY: Figure 1 in [13].
Figure 6. The fabrication of microstructured PDMS films. COURTESY: Figure 1 in [14].
The fabrication process of a conformal electronic device can be seen in Figure 6, and shows how the thin film is molded and etched (embedded with features). In the end, we have a fully-functional and bendable array. Bendable electronics can be used for a range of applications, and is the enabling technology behind flexible OLED displays (perhaps a staple of third and fourth generation e-readers). Theoretically at least, a similar process could be used to fabricate multi-layered embeddable replacement skins for humans, which could not only restore the function of touch but also augment our sensory capabilities. A number of challenges would have to be overcome, including interfaces with the body's endocrine (hormones), circulatory (blood), and excretory (sweat) systems, the importance of which is discussed in [15].
Figure 7. An overview of the deformational and functional properties of conformal electronics. COURTESY: Figure 2 in [13].
The Future
Through the use of compliant and biocompatible materials, algorithms that can decode nervous systems activity, and a good understanding of the role touch and proprioception play in human self-awareness, artificial touch systems might become as common a technology as touch screen computers are today.
References:
[1] Drewing, K. and Ernst, M.O. Integration of force and position cues for shape perception through active touch. Brain Research, 1078, 92–100 (2006).
[1a] For some interesting examples, please visit the Jeka Lab at University of Maryland-College Park and the Multisensory Perception and Action group at Bielefeld University.
[2] Kiemel, T., Oie, K.S., and Jeka, J.J. Multisensory fusion and the stochastic structure of postural sway.
Biological Cybernetics, 87, 262-277 (2002).
[2a] For examples of multisensory integration in the SC, please see: Stein, B.E. and Meredith, M.A. The Merging of the Senses. MIT Press, Cambridge, MA (1993).
For an example of multisensory integration in the PPC, please see: Pasalar, S., Ro, T., and Beauchamp, M.S. TMS of posterior parietal cortex disrupts visual tactile multisensory integration. European Journal of Neuroscience, 31(10):1783-1790 (2010).
[3] Li, L., Rutlin, M., Abraira, V.E., Cassidy, C., Kus, L., Gong, S., Jankowski, M.P., Luo, W., Heintz, N., Koerber, H.R., Woodbury, C.J., and Ginty, D.D. The Functional Organization of Cutaneous Low-Threshold Mechanosensory Neurons. Cell, 147, 1615–1627 (2011).
[4] Duenas, J., Chapuis, D., Pfeiffer, C., Martuzzi, R., Ionta, S., Blanke, O., and Gassert, R. Neuroscience robotics to investigate multisensory integration and bodily awareness. Proceedings of the IEEE Engineering and Medicine in Biology Society, 8348-8352 (2011).
[4a] For more information, please see: Ernst, M.O. and Banks, M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429-433 (2002).
[4b] For an outline of this idea (with experiments!) please see: Alicea, B. Performance Augmentation in Hybrid Systems: techniques and experiment. arXiv, 0810. 4629 [q-bio.NC, q-bio.QM] (2008).
[5] Mate, C.W. and Carpick, R.W. A sense for touch. Nature, 480, 189-190.
[5a] For reflections on the use of hierarchical material scaffolds to realize multiscalar material design, please see: Cranford, S.W. and Buehler, M.J. Shaky foundations of hierarchical biological materials. Nano Today, 6, 332-338 (2011).
[6] Quake, S.R and Scherer, A. From Micro- to Nanofabrication with Soft Materials. Science, 290, 1536-1540 (2000).
[7] Gao, H., Wang, X., Yao, H., Gorb, S., and Arzt, E. (2005). Mechanics of hierarchical adhesion structures of geckos. Mechanics of Materials, 37, 275-285.
[8] Wandersman, E., Candelier, R., Debregeas, G., and Prevost, A. Texture-Induced Modulations of Friction Force: The Fingerprint Effect. Physical Review Letters, 107, 164301 (2011).
[7] Gao, H., Wang, X., Yao, H., Gorb, S., and Arzt, E. (2005). Mechanics of hierarchical adhesion structures of geckos. Mechanics of Materials, 37, 275-285.
[8] Wandersman, E., Candelier, R., Debregeas, G., and Prevost, A. Texture-Induced Modulations of Friction Force: The Fingerprint Effect. Physical Review Letters, 107, 164301 (2011).
[9] Dixon, S.J., Collop, A.C., and Batt, M.E. (2000). Surface effects on ground reaction forces and lower extremity kinematics in running. Medicine and Science in Sports and Exercise, 32(11), 1919-1926.
[10] Lipomi, D.J., Vosgueritchian, M., Tee, B.C-K., Hellstrom, S.L., Lee, J.A., Fox, C.H., and Bao, Z. Skin-like pressure and strain sensors based on transparent elastic films of carbon nanotubes. Nature Nanotechnology, (2012).
[11] Ying, M., Bonifas, A.P., Lu, N., Su, Y., Li, R., Cheng, H., Ameen, A., Huang, Y., and Rogers, J.A. Silicon nanomembranes for fingertip electronics. Nanotechnology, 23, 344004 (2012).
[12] Alicea, B. Nano-enabled Biological Materials. Nature Precedings, npre.2010.5448.1 (2010).
[12] Alicea, B. Nano-enabled Biological Materials. Nature Precedings, npre.2010.5448.1 (2010).
[13] Kim, D-H., Ghaffari, R., Lu, N., and Rogers, J.A. Flexible and Stretchable Electronics for Biointegrated Devices. Annual Review of Biomedical Engineering, 14, 113-128 (2012).
The Rogers lab has come up with a number of interesting advanced applications in the area of confomal electronics, including wearable (tattooable) transistors and dissolvable devices (implantable that dissolve harmlessly once no longer needed) For more information on this application, please see:
Hwang, S-W., Tao, H., Kim, D-H., Cheng, H., Song, J-K., Rill, E., Brenckle, M.A., Panilaitis, B., Won, S.M., Kim, Y-S., Song, Y.M., Yu, K.J., Ameen, A., Li, R., Su, Y., Yang, M., Kaplan, D.L., Zakin, M.R., Slepian, M.J., Huang, Y., Omenetto, F.G., and Rogers, J.A. A Physically Transient Form of Silicon Electronics. Science, 337, 1640 (2012).
The Rogers lab has come up with a number of interesting advanced applications in the area of confomal electronics, including wearable (tattooable) transistors and dissolvable devices (implantable that dissolve harmlessly once no longer needed) For more information on this application, please see:
Hwang, S-W., Tao, H., Kim, D-H., Cheng, H., Song, J-K., Rill, E., Brenckle, M.A., Panilaitis, B., Won, S.M., Kim, Y-S., Song, Y.M., Yu, K.J., Ameen, A., Li, R., Su, Y., Yang, M., Kaplan, D.L., Zakin, M.R., Slepian, M.J., Huang, Y., Omenetto, F.G., and Rogers, J.A. A Physically Transient Form of Silicon Electronics. Science, 337, 1640 (2012).
Dissolvable thin film transistor partially immersed in solvent. COURTESY: iMedicalApps.com.
[14] Mannsfeld, S.C.B., Tee, B.C-K., Stoltenberg, R.M., Chen, C.V.H-H., Barman, S., Muir, B.V.O., Sokolov, A.N., Reese, C., and Bao, Z. Highly sensitive flexible pressure sensors with microstructured rubber dielectric layers. Nature Materials, 9(9), 1-6 (2010).
[15] Slominski, A.T., Zmijewski, M.A., Skobowiat, C., Zbytek, B., Slominski, R.M., and Steketee, J.D. (2012). Sensing the environment: regulation of local and global homeostasis by the skin's neuroendocrine system. Advances in Anatomy, Embryology, and Cell Biology, 212, 1-115.
[15] Slominski, A.T., Zmijewski, M.A., Skobowiat, C., Zbytek, B., Slominski, R.M., and Steketee, J.D. (2012). Sensing the environment: regulation of local and global homeostasis by the skin's neuroendocrine system. Advances in Anatomy, Embryology, and Cell Biology, 212, 1-115.
Subscribe to:
Posts (Atom)