Human Augmentation Short Course -- Part II

I have been continuing to introduce an area of science and engineering called human augmentation to a broader audience using the "flash lecture" format discussed a few posts back. I have using my micro-blog Tumbld Thoughts as a test site for posting these lectures, and the social networking function of Tumblr (e.g. Tumblr radar) has garnered some sporadic direct interest (in the form of likes and re-posts). 

In this portion of the course (four lectures), we move beyond the basics and towards both more detailed phenomena related to augmentation and practical implementations of the technology.

I. Extending the Phenotype

In previous #human-augmentation posts, I briefly touched on the potential role of augmentation (e.g. wearing a prosthetic limb or see-though, head-mounted display) on physiological regulation. This may leave some readers puzzled, because often times these technologies do not directly interface with the nervous system. Nevertheless, once a technology provides a stand-in (or enhancer) of something the body does, it becomes incorporated into the body's physiology and representation of the world [1].

While technologies from rakes to nests have been found to represent an extended phenotype, an intelligent technology (one that includes an adaptive mitigation strategy) can actually serve to enhance or work in concert with an individual's ability for environmental adaptation [2]. Much like the diet and exercise regimen that helps a person lose weight, this ability exhibits great variation across individuals, which may be explained by pre-existing phenotypic or even genotypic differences.

The extent to which the technology participates in the physiological millieu depends of course on how much the augmentation assists in or takes over function. To understand this better, we can turn to the ergonomics definition of symbiosis, which defines "ergonomy" as the degree of coupling between human and mechanical device [3]. This ranges from tightly-coupled systems (implants that are seamlessly integrated into normal function) to ill-fitting systems (poorly-designed interfaces or computer mice).

II. Instrumented Motorcycle Helmet

Here is an example of performance augmentation in the form of an intelligent, see-through, and heads up display integrated into a motorcycle helmet. Brought to you by a start-up called LiveMap. The information presented in the field of view enables the wearer to improve their navigation ability and improve their riding experience. 

The first article (from Mashable - [4]) highlights the components of the helmet, which includes ambient information from multiple types of sensor (e.g. light sensor, microphone, GPS). This information is then fused and presented in a single location (in this case, the helmet) [5].

II. Wired Science Live Chat on Bionic Augmentation

Last week, I attended a live chat called "Our Cyborg Future", hosted by Wired Science. This was a live chat with two scientists in the field: John Rogers from UIUC and Michael McAlpine from Princeton [6]. These researchers work in a field called "bionics", where biological systems are augmented with electronics or other technology to either restore function or provide new sensory or performance capabilities.

The talk featured a number of visions for the future of bionic technologies and technologies for human augmentation. Fundamentally, the major challenge is to merge the language of electronics (e.g. electrons, phonons, and heat) with the language of biology (e.g. ions, proteins, and enzymes). While John Rogers is working towards electronically-augmented organs [7], Michael McAlpine is working towards using piezoelectric materials to harvest energy from biological motion and print 3-D structures such as tissue scaffolds. 

Three of the most interesting ideas [8] discussed during the talk:

* advances such as flexible [7] and bio-compatible electronics might be used to infuse a biological system with distributed electronics at the cellular and subcellular scale. This could include a range of components from silicon diodes to LEDs.

* the increases in brain-machine interface bandwidth due to flexible, laminated skin-like devices.

* the development of emerging technologies such as stretchable batteries, glucose fuel cells, implantable micro-heaters, and mechanical energy harvesting.

IV. The Role of Attention, Training, and critical Meta-Analysis

In a previous #human-augmentation post, I pointed to one experimental paradigm (environmental switching) that may serve as a natural (e.g. non-computational) filter for eliminating (e.g. mitigating) non-optimal performance due to environmental stresses or other challenges. This case is illustrative for two reasons: 

1) in cases where performance response curves are very complex and cannot be characterized by a simple mathematical function (e.g. a "U" shaped curve), a mitigation strategy involving physical chaos (rather than computational control) or other environmental manipulations may be more effective. In the first image (top), the potential dynamic effects of perturbation on attentional shifts during an episode of "Star Trek" is used as an example.

2) fully understanding the effects of mitigation may require more systematic experimental evaluation. One example of this involves the claim that long-term expertise with action video games improves cognitive abilities [9]. A meta-analysis of such studies [10] questions this assumption on several grounds, particularly with respect to the magnitude of improvement (e.g. effect size).

In this case of action video game expertise, two types of effect have been reported [10, 11]. The first involves relative expertise based on cross-sectional comparisons, which evaluate differences between gamers and non-gamers. The second involves acquired expertise (training in action video game play) as having numerous cognitive benefits.

The second type of effect is partially due to an effect called transfer of training (see image, lower left), in which skills acquired in one context can be transferred to another context. This effect also plays a role in human augmentation, and may exhibit a large degree of individual variation. However, there are three caveats raised in [10] that must be kept in mind not only for future action video game studies, but human augmentation studies as well:

* in studies that evaluate the effects of training, an adequate baseline for untrained performance must be used. 

* results should be generalizable to different settings and population (e.g. exhibit a high degree of experimental reproducibility).

* while there may be several improvements to performance/cognition attributable to prior training or experience with the activity in question, there may be many more outcomes that are unaffected by the treatment (action video games) or mitigation (human augmentation). 

NOTES:

[1] A few examples of this extension of the phenotype includes examples from humans (i), social insects (ii, iii), and animals (iv):

(i) Maravita, A. and Iriki, A.   Tools for the body (schema). Trends in Cognitive Science, 8(2), 79-86 (2004).

(ii) Turner, J.S.   The Extended Organism: The Physiology of Animal-Built Structures. Harvard University Press, Cambridge, MA (2002).

(iii) Turner, J.S.   Extended Phenotypes and Extended Organisms. Biology and Philosophy, 19, 327–352 (2004).

(iv) Schaedelin, F.C. and Taborsky, M.   Extended phenotypes as signals. Biological Reviews of the Cambridge Philosophical Society, 84(2), 293-313 (2009). 

[2] This ability, or adaptability, can be characterized using a parametric landscape as shown in a previous post.

[3] Licklider, J.C.R.   Man-Computer Symbiosis. IRE Transactions on Human Factors in Electronics, HFE-1, 4-11 (1960).

[4] Murphy, S.   A Motorcyclist's Dream: Google Glass in helmet form. Mashable, June 17 (2013).

[5] Mike E. New motorcycle helmet with augmented reality and navigation technology built in. VR-Zone, June 17 (2013).

[6] A transcript of the talk can be found here. Also see the McAlpine Research YouTube channel.

[7] For more on flexible robotics, please see this: Zheng, Y., He, Z., Gao, Y., and Liu, J.   Direct Desktop Printed-Circuits-on-Paper Flexible Electronics. Scientific Reports, 3, 1786 (2013).

[8] For more interesting ideas related to cyborgs and bio-inspired robotics, see these:

Rowe, A.   Top 10 Cyborg Videos. Wired Science, November 15 (2009)

When Animals Learn to Control Robots, You Know We’re in Trouble. Wired Science, March 21 (2013).

[9] Green, C. and Bavelier, D.   Learning, Attentional Control, and Action Video Games. Current Biology, 22(6), R197-R206.

[10] Boot, W.R., Blakley, D.P., Simons, D.J.   Do action video games improve perception and cognition? Frontiers in Psychology, 2, 226 (2011).

[11] Simons, D.   Think video games make you smarter? Not so fast..... Daniel Simons blog, December 30 (2012).

Adventures in #SciFund-land, with Synthetic Daisies contributions

As you may have noticed, this blog regularly features posts that are 1) submitted to something called "Carnival of Evolution", and 2) reposts from my micro-blog Tumbld Thoughts (which has become somewhat of a proving ground for featured content and "bigger" ideas). Beginning soon (perhaps within the month), I will be writing roughly one post per month for a new group blog hosted by the SciFund Challenge featuring various topics on science outreach.

This new collaboration is a product of the first SciFund outreach course, which took place during the last week of April and though the month of May. The course focused on tailoring your message (e.g. describing you research) to a broader audience (e.g. journalists and non-scientists). Even with my blogging experience, I found this to be challenging. We met weekly and conducted exercises in small discussion groups. As a result, I was able to make many new connections and be exposed to a number of new perspectives on research and outreach.

A scene from the SciFund outreach course (conducted in GoogleHangouts).

We were also introduced to non-traditional tools for scientific communication such as cartoons and podcasting, which may make an appearance on this blog in the future. There was also discussion about crowdfunding, which was the original impetus for the #SciFund Challenge. For more information on future outreach courses and other activities sponsored by SciFund, check out their website. More to come.

A caricature of Jai Ranganathan (one of the course organizers), courtesy of Jorge Cham, PhD Comics.

As a bonus, here is a list of papers and blog posts that were recommended reading during the five-week course:

Bik, H.M. and Goldstein, M.C.   An Introduction to Social Media for Scientists. PLoS Biology, 11(4), e1001535 (2013).

Clancy, K.   Defensive Scholarly Writing and Science Communication. Context and Variation blog, April 24 (2013).

Darling, E.S., Shiffman, D., Cote, I.M., and Drew, J.A.   The role of Twitter in the life cycle of a scientific publication. PeerJ PrePrints, 1, e16v1 (2013).

Kwok, R.   Two minutes to impress. Nature, 494, 137-138 (2013).

Somerville, R.C. and Hassol, S.J.   Communicating the science of climate change. Physics Today, 64(10), 48 (2011).

Plait, P.   SciFund Challenge: Why  Do Outreach? Bad Astronomy blog, May 20 (2013).

Weinersmith, K.   Comics as a tool for science outreach. SciFund Challenge blog, May 15 (2013).

New Views on Evolution in the Reading Queue

In this post, I would like to bring attention to two recent review papers that advance the application of evolutionary theory to brain and behavior in bold new ways. One is involves re-evaluating the phylogenetic origins of learning and memory, while the other presents an alternate hypothesis for neocortical (or more properly, isocortical) expansion in humans.

Paper #1: Allen, T.A. and Fortin, N.J.   The evolution of episodic memory. PNAS, doi:10.1073/pnas.1301199110.

Figure 1. The deep phylogeny of the proto-episodic memory architecture.

The first review, brought to my attention by the Olduvai Gorge tumblr site, is a re-interpretation of Endel Tulving's declaration that episodic memory has no "Darwinian" explanation. Episodic memory is the kind of memory for individuals, places, and things that enables phenomena such as "mental time travel" and the reconstruction of sophisticated scenarios [1]. While episodic memory involves several brain structures, their shared evolutionary history has not been a primary concern in making sense of their function.

To understand the evolution of episodic memory, the authors attempt to reconstruct the deep evolutionary history of episodic memory (Figure 1). To do this, a series of homologous structures are proposed that are shared by birds and mammals. These structures (hippocampus, parahippocampal region, and prefrontal cortex) are then proposed as the seat of proto-episodic memory. Proto-episodic memory, which is shared by birds and mammals, has been thought to be convergent. In this paper, however, this capability is shown to be conserved, and present in the bird-mammalian ancestor.

While proto-episodic memory is highly conserved, not every species has an equivalent capacity. Indeed, as the prefrontal cortex (e.g. isocortex) is enlarged in humans, capabilities related to this structure are enhanced. This is due to species-specific functional differences that have emerged from the same basic architecture (Figure 2). In light of this, an intriguing comparison is made between episodic memory and fear conditioning. Fear conditioning (or emotional memory) is thought to be highly conserved [2] and characterized by one-trial encoding with inefficient recall. What distinguishes episodic memory from other types of memory are:

* content of the memory trace (the what, where, and when of events),

* structure of the memory trace (complex organization of task and information recall)

* flexibility of memory implementation (the adaptive capacity of a brain).

Figure 2. Episodic memory architecture in mammals (A) and birds (B) as compared to the proto-architecture (fundamental circuit, C).

Paper #2: Raichlen, D.A. and Polk, J.D.   Linking brains and brawn: exercise and the evolution of human neurobiology. Proceedings of the Royal Society B, 280, 20122250(2012).

The second review is an alternative to the social brain hypothesis [3]. The expansion of human isocortex has been linked to a number of causal factors, including an increase in social network size amongst hominids. The social brain hypothesis predicts that isocortical expansion can be linked to social demands (e.g. the need to remember an increasing number of distinct faces). But what the social brain hypothesis does not provide is an account of the proximal mechanisms that enable this change (Figure 3).

Figure 3. Several candidate proximal mechanisms are thought to be responsible for brain enlargement in hominids.

The authors of this paper propose aerobic physical activity (APA) as the driver of such proximal (e.g. not explicitly evolutionary) mechanisms. From studies examining the effect of exercise on the brain, we know that APA generates and protects new neurons, increases the volume of brain structures, and improves cognition in living human brains [4]. In cross-species comparisons based on artificial selection experiments, APA regimens administered to primates and rodents initiates activity-induced neurogenesis, which appears to improve memory and spatial learning. APA regimens also result in the upregulation of neurotrophins such as BDNF, IGF-1, and VEGF (e.g. growth and angiogenesis factors) in mice and humans alike. Across evolutionary time, it is proposed that selection for endurance running [5] also acted to increase baseline neurotrophic and growth factor signaling, which in turn enables the metabolic background for isocortical expansion (Figure 4).

Figure 4. An excellent rendition of brain evolution (for several structures) across the Order Primates. COURTESY: Robert Dahnke of the Structural Brain Mapping Group, University of Jena, Germany.

NOTES:
[1] This old view may have simply been an artifact of the view that human behavior is somehow special or "distinct" from other animal species. For example, Machiavellian-type behavior has been well-characterized in apes and monkeys, and caching behavior is well-known in birds.

More information on mental time-travel can be found here: Suddendorf, T.   Mental time travel: continuities and discontinuities. Trends in Cognitive Sciences, 17(4), 151-152 (2013).

[2] This statement is in reference to evolutionary conservation of the main structure (amygdaloid) involved in fear consolidition. Paper (i) that proposes the amygdaloid structure was present in ancestral tetrapods, while paper (ii) demonstrates (through meta-analysis) that the amygdala is key in regulating emotional forms of memory:

(i) Moreno, N. and Gonzalez, A.   Evolution of the amygdaloid complex in vertebrates, with special reference to the anamnio-amniotic transition. Journal of Anatomy, 211(2), 151–163 (2007).

(ii) Sergerie, K., Chochol, C., and Armony, J.L.   The role of the amygdala in emotional processing: a quantitative meta-analysis of functional neuroimaging studies. Neuroscience Biobehavioral Reviews, 32(4), 811-830 (2008).

[3] Dunbar, R.I.M.   The social brain hypothesis. Evolutionary Anthropology, 6(5), 178-190 (1998) AND Dunbar, R.I.M.   The Social Brain: mind, language, and society in evolutionary perspective. Annual Review of Anthropology, 32, 163-181 (2003).

Here is a video (courtesy of Neuroanthropology blog) that critiques the Social Brain hypothesis from a social neuroscience perspective.

[4] Voss, M.W., Nagamatsu, L.S., Liu-Ambrose, T., and Kramer, A.F.   Exercise, brain, and cognition across the life span. Journal of Applied Physiology, 111(5), 1505-1513 (2011).

[5] Bramble, D.M. and Lieberman, D.E.   Endurance running and the evolution of Homo. Nature, 432, 345-352 (2004).

Human Augmentation Short Course -- part I

I have been posting many odds and ends I have in my files on the topic on the science of human performance augmentation to my micro-blog, Tumbld Thoughts. After a few posts in this area, I decided to try a Tumblr-based short course composed of "flash" lectures (an innovation I am experimenting with). 

Below is a series of posts that constitute the first part of this short course (mostly introductory concepts). These short lessons can also be found (in their original context) under the #human-augmentation tag on Tumblr. Comments would be appreciated.

I. Basics of Performance Mitigation

In a previous post, I discussed an area of science and engineering called Augmented Cognition. But how does one “augment” cognition? In the modern version, a mitigation strategy is used to enforce optimal performance in a manner similar to supervised learning.

* this requires a well-characterized response function. One example is theYerkes-Dodson law (inverted curve) that characterizes arousal.

* mitigation proceeds using a first-order linear control strategy with feedback. This is similar to the pole-balancing task widely used in robotics.

* to make these judgments, measurements of physiological state are used. In these cases, the measurements determine when there is “too much” or “too little” of a physiological state. For example, “too much” activity in a certain part of the brain leads to an unacceptable amount of arousal, which can then be mitigated using a visual stimulus on a heads-up display.

* but what about more complex responses and the role of adaptation? This will be covered in the next post.

* all images taken from the following lecture: Alicea, B.   Behavioral Engineering and Brain Science in Virtual Reality. Figshare, doi:10.6084/ m9.figshare.155710 (2013).

II. Performance Mitigation vs. Optimization

In my last post on human augmentation, I discussed what a mitigation strategy is. Now I would like to discuss the role of natural variation in human augmentation and mitigation. While some people might view the products of natural selection to be “optimal”, natural variation actually works against engineering optimization in a number of key ways.

* to fully account for natural variation, we must open up the black box of physiological regulation. To do this, we must understand the process of humans interacting with technology as a homeostatic or allostatic process [1].

* physiological regulation as a result of technological interaction occurs at multiple biological scales. These include the molecular bases of learning and memory, tissue-specific gene expression, cognitive memory consolidation, and populations in their environment.

* the nature and potential outcomes of this process can be captured using a fitness landscape or related type of n-dimensional phase space [2]. This allows us to understand the adaptability of specific genotypes or populations of individuals [3].

* the use of fitness landscapes allows us to characterize allostatic regulation as a hill-climbing (or quasi-optimal) process. However, in doing so, we must account for certain regularities of training such as the power law of practice.

III. Role of Natural Variation in Performance Augmentation

In the last two #human-augmentation posts, I discussed the role of mitigation strategies and natural variation in human augmentation. In this post, we will explore an experimental paradigm for training in novel environments that produces a chaotic output via stochastic resonance [4]. As a chaotic output, it can be controlled by various types of feedback [5].

We can use a motion-controlled sports videogame to simulate various human movement regimes (e.g. swinging, reaching). A weighted instrument (e.g. misshapen baseball bat with a forcing chamber [6]) was used to introduce chaotic motion during performance. By switching between this distortion and normal gameplay, we have created an environmental switch that can induce natural variation and the biological substrates that underlie performance. 

This type of environmental switch is found in nature as brain-related preconditioning [7] in humans, or as a means to speed up adaptation in a given population [8]. By presenting each type of movement regime in different sequences, we can augment performance under normal circumstances or control chaotic fluctuations [9].

IV. Review of Performance Mitigation Architectures

Two posts ago in the human augmentation thread, we were introduced to the role of allostasis and first-order linear control in correcting (e.g. mitigating) sub-optimal behaviors related to human performance. In this post, we will explore this theme further using architectures that adaptively control (e.g. augment) optimal levels of cognition and human performance.

The first architecture is the simple feedback with band-pass filter. This is often used to mitigate performance profiles that conform to the inverted U (e.g. arousal). The bandpass filter implements a simple rule used as feedback that reinforces parameter values within a certain range. This first-order linear control manages a unimodal response function as a signal-to-noise problem without excessive computational overhead.

But what about cases where our measure exhibits a greater number of measures? The second architecture demonstrates the simple feedback motif as a parallel array (in this case, two arrays) that contribute to a global assessment of performance (long rectangle). In this case, mitigation is treated as an optimization problem rather than a signal-to-noise problem. This allows us to search for optimal mitigation configurations on a n-dimensional landscape rather than extracting one-dimensional signal from noise.

In cases where the contributions of physiological variance are great, from example in systems which are not well-understood, we can use something called I call an allostatic control architecture. This type of model accounts for a dynamic physiological background as it interacts with performance embedded in its environment. To enforce this type of control, environmental switching [10] can be used. In this case, there is no feedback, but there is a linear filter that enforces a threshold on the response to both sets of environmental conditions. Levels of performance that are robust in both environments are selected for using the filter, and treats mitigation as a problem of stability during adaptation. 

NOTES:

[1] For the concept of allostatic regulation, please see: Schulkin, J.   Rethinking Homeostasis: allostatic regulation in physiology and pathophysiology. MIT Press (2003).

For the concept of technology being an environmental challenge, please see: Alicea, B.    Performance Augmentation in Hybrid Systems: techniques and experiment. arXiv Repository, arXiv:0810.4629 [q-bio.NC, q-bio.QM] (2008).

[2] For more information on the geometry of fitness landscapes, please see: Gavrilets, S.   Fitness Landscapes and the Origin of Species. Monographs in Population Biology, 41. Princeton University Press (2004).

[3] Populations can consist of special needs populations, different ethnic groups, and even people of differing body shape and athletic ability. They carry unique molecular, physical, and cognitive features to specific types of interaction.

For more information, please see: 

Alicea, B.   Natural Variation and Neuromechanical Systems. Cogprints, 6698 (2009).

Alicea, B.   The adaptability of physiological systems optimizes performance: new directions in augmentation. arXiv Repository, arXiv: 0810.4884 [cs.HC, cs.NE] (2008).

[4] Alicea,  B.    Stochastic  Resonance  (SR)  can  drive  adaptive  physiological  processes.  Nature  Preceedings, npre.2009.3301.1 [http://precedings.nature.com/documents/3301/version/1] (2009). 

[5] Hunt and Johnson   Keeping Chaos at Bay. IEEE Spectrum, 30(11), 32-36 (1993).

[6] A forcing chamber is a container filled with a liquid or other material (with a specific gravity) to create a distorted radius of gyration during a swing, a reach, or a stroke.

[7] Gidday, J.M   Cerebral preconditioning and ischemic tolerance. Nature Reviews Neuroscience, 7, 437-448 (2006).

[8] Kashtan, N., Noor, E., Alon, U.   Varying environments can speed up evolution. PNAS USA, 104(34), 13711-13716 (2007).

[9] Ott, E., Grebogi, C., and Yorke, J.A.   Controlling Chaos. Physical Review Letters, 1196-1199 (1990).

Ott, E.   Controlling Chaos. Scholarpedia, 1(8), 1699 (2006).

[10] Shifting between environments continually or in a patterned way. See the last #human-augmentation post for more.

Human Augmentation tag on Tumbld Thoughts

Here are the posts that constitute the #human-augmentation tag on my micro-blog, Tumbld Thoughts (as of 6/6/2013). I have been posting short features to organize my thoughts on how to communicate the topic. This includes features on augmentation-induced sensory illusion, Augmented Cognition, and the relationship between natural variation and augmentation. Enjoy.

I. Augmentation-induced Sensory Illusion

Here is a guided tour of human augmentation and the sensory illusions it often produces. The first example comes from Neil Harbisson and his third eye [1]. In this case, the third eye is a camera that is permanently worn, perceptually integrating this with his other two biological eyes [2]. This is a bit different than the augmented reality (AR) heads-up augmentation afforded by Steve Mann's early AR prototypes and Google Glass. In some ways, this sensory illusion mimics the prism adaptation from Neuropsychology [3]. The visual system adapts to the extra information, so that when it is removed, perception and even balance can be severely affected.

The second example is that of a third hand connected to a (second or) third arm. In 2011, a set of experiments [4] was published that demonstrate this effect. When the torso is shielded from view and a third (rubber) arm with hand is placed parallel to the other two arms (in full view of the subject), stroking or otherwise stimulating the third arm results in an emotional response that corresponds to this touching. It is not the sense of touch in a conventional sense [5], but a form of pseudo-touch enabled solely by the visual anticipation. But if you actually had a third hand, how would you control its movement? Fortunately, the artist Stelarc answered that question for us in the 1980s, when he used myoelectric control [6] to move his extraneous hand. And recent advances have been made to the design of prosthetic arms that may require us to rethink the role of dynamic touch in sensation and perception [7].

Review of closed-loop control of perception and action with relevance to prosthetic limb design. COURTESY: Reference [7].

II. Augmented Cognition

Augmented Cognition (Aug Cog) is an emerging research area that combines cognitive engineering, neuroimaging, and human-computer interaction. The vision of augmenting cognition goes back to early cybernetics work by W. Ross Ashby. Modern work in this area has grown out of a DARPA project funded in the early-to-mid-2000s. The idea is to augment or otherwise improve cognitive resources (e.g. attention, arousal, memory) using measurements of brain activity (e.g. EEG, fNIR) and computational models. 

The first three images (from top) are from the video short called AUG, written and directed by Tam Morris and Alistair Patterson. In these examples, cognition is being augmented using heads-up displays. The two images below are from a film called "The Future of Augmented Cognition", directed by Alexander Singer. These images show profiles of the pioneering scientists (Herbert Simon and Britton Chance) whose work serves as inspiration for modern AugCog efforts.

III. Natural Variation and Augmentation

Here [8] is a profile of one of my "lost" papers: "Range-based techniques for discovering optimality and analyzing scaling relationships in neuromechanical systems". This paper was published in 2009 at Nature Preceedings [9], but I'm not sure it ever got much exposure. The paper introduces something called the rescaled range technique, which I will describe below. The objective of this study was to take data collected from experiments [10] conducted in virtual environments, and then compare metrics of performance (movement behavior and muscle activity) to something called a "morphological scaling" (e.g. the proportion of forelimb length or volume to humeral length or volume). The application of mathematical modeling provides us with a tool called the "rescaled range".

What if we were to manipulate the length of these limbs well beyond their natural range? Could the tasks still be performed? Or would there be significant performance gains at certain scalings of size? The idea of the rescale range technique is to simulate this possibility based on empirical observations and mathematical modeling. The second picture from the top summarizes this idea in terms of physical implementation and the theoretical concept of hypo- and hyper-allometry [11]. 

By scaling each component of observed limb measurements by a certain factor, we effective also resample the performance data (as shown in the third and fourth figures from top). Indeed, this resampling shows that there are large-scale increases and decreases in performance for various manipulations. 

How are these findings useful? The first point is that these modeled manipulations of limb lengths serve as an analogue for what might be possible in the use and design of tools, devices used for teleoperation, and virtual representations of touch and the body. There are also a number of interesting relationships between physical perturbations and performance in such tasks [12]. This work has potentially great relevance to the design of immersive virtual environments, or touch- and movement-based virtual environments that have a significant real-world component.

IV. Towards a Cross-species Perspective

It is well-known in the field of human-robot interaction that humans recognize robots as having human-like qualities according to a non-linear function that results in the uncanny valley effect [13]. While this judgement of wheter or not a robot is human-like may be based on qualitative factors, recent neuroscience research suggests that this effect has roots in the perception-action system of the brain [14]. 

But do other species exhibit a similar effect? In [15], it was found that real fish can be attracted to the locomotion of a robotic fish. This attraction (or set of sensory cues) depends on hydrodynamic advantages [16] created by the swimming itself. In isolation, this finding could simply be a curious experimental artifact. However, the research in [17] demonstrates that fish sensorimotor learning can be manipulated using a virtual environment. So is the uncanny valley effect found in other species? A strange but fascinating world with more to come.....

NOTES:

[1] Photo courtesy Cyborg Foundation and Eveleth, R.   Ask a Cyborg. Nautil.us, Issue 001 (2013).

[2] For more information on the incorporation of cameras into the visual system, please see this Synthetic Daisies post on Steve Mann: Alicea, B.   Steve Mann, Misunderstood. Synthetic Daisies blog, July 18 (2012).

[3] Rock, I.   Orientation and Form. Academic Press, New York (1973).

[4] for more information on the science behind the supernumary hand illusion, please see: Guterstam, A., Petkova, V.I., and Ehrsson H.H.   The Illusion of Owning a Third Arm. PLoS One, 6(2), e17208 (2011).

Kunert, R.   Three fun ways to have three hands - for you at home. Brain's Idea blog, July 31 (2012). 

Parker-Pope, T.   Need an Extra Hand? NYT Well blog, February 24 (2011).

[5] For more information on the role of vision in touch (and the source of the diagram at lower right of the above image), please see: Ernst, M.O. and Bulthoff, H.H.   Merging the senses into a robust percept. Trends in Cognitive Science, 8(4), 162-169 (2004).

[6] For more information about the interplay between myoelectric control of a hand and the resulting neural responses, please see: Maruishi, M. et.al   Brain activation during manipulation of the myoelectric prosthetic hand: a functional magnetic resonance imaging study. NeuroImage, 21, 1604-1611 (2004).

[7] Kwok, R.   Once more, with feeling. Nature, 497, 176-178 (2013).

[8] More information on Figure 1: Picture at lower right from the Gritsenko Lab Website, University of West Virginia. This lab does research on the neural mechanisms behind the online correction of sensorimotor control using virtual environments. Somewhat similar to what I am getting at here -- the difference is that I am focusing more on the distortion capabilities/potential of the virtual environment itself.

Picture at lower left from the following article: Pappas, S.   Machine That Feels May Usher in 'Jedi' Prosthetics. LiveScience, October 5 (2011).

[9] Alicea, B.    Range-based techniques for discovering optimality and analyzing scaling relationships in neuromechanical systems. Nature Precedings, npre.2009.2845.2 (2009).

[10] The experiments involved reaching for and manipulating virtual objects (e.g. arm swinging and touching) with feedback. The feedback was distorted not only by the virtuality of the task, but also by distorting the tools (e.g. physics, shape) used to perform these tasks.

[11] Hypo- and hyper-allometry normally occur in the development of organisms, and usually describe evolutionary changes that unfold across related taxa. For example, in the case of the order Primates, the forearm:humerus relationship is highly variable across species, but not so much across individuals within a species.

[12] See the following papers for other work that provide more detail about these type of experiments:

Alicea, B.    Naturally Supervised Learning in Motion and Touch-driven Technologies. arXiv, arXiv: 1106:1105. [cs.HC, q-bio.NC] (2011).

Alicea, B.    Performance Augmentation in Hybrid Systems: techniques and experiment. arXiv, arXiv:0810.4629 [q-bio.NC, q-bio.QM] (2008). 

[13] For more information, please see: Moore, R.K.   A Bayesian explanation of the ‘Uncanny Valley’ effect and related psychological phenomena. Scientific Reports, 2, 864 (2012).

Guizzo, E.   Who's Afraid of the Uncanny Valley? IEEE Spectrum, April 2 (2010).

[14] Saygin, A.P., Chaminade, T., Ishiguro, H., Driver, J., and Frith, C.   The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions. Social, Cognitive, and Affective Neuroscience, 7(4), 413-422 (2012).

[15] Marras, S. and Porfiri, M.   Fish and robots swimming together: attraction towards the robot demands biomimetic locomotion. Journal of the Royal Society Interface, doi:10.1098 (2012). Video.

[16] Salumae, T. and Kruusmaa, M.   Flow-relative control of an underwater robot. Proceedings of the Royal Society A, 469, 20120671 (2013).

[17] Engert, F.   Fish in the matrix: motor learning in a virtual world. Frontiers in Neural Circuits, 6, 125 (2013).

See the following references for more information on Augmented Cognition:

[i] Schmorrow, D.   Foundations of Augmented Cognition. CRC Press (2005).

[ii] Costandi, M.   Augmented Cognition: science fact or fiction? Neurophilosophy blog, January 3 (2007).

[iii] Izzetoglu, K., Bunce, S., Onaral, B., Pourrezzaei, K., and Chance, B.   Functional Optical Brain Imaging Using NIR during Cognitive Tasks. International Journal of Human-Computer Interaction, 17(2), 211-227 (2004).

[iv] Alicea, B.   Behavioral Engineering and Brain Science in Virtual Reality. Figshare, (2012).

[v] Alicea, B.   The adaptability of physiological systems optimizes performance: new directions in augmentation. arXiv Repository, arXiv:0810.4884 [cs.HC, cs.NE] (2008).