Human Augmentation tag on Tumbld Thoughts

Here are the posts that constitute the #human-augmentation tag on my micro-blog, Tumbld Thoughts (as of 6/6/2013). I have been posting short features to organize my thoughts on how to communicate the topic. This includes features on augmentation-induced sensory illusion, Augmented Cognition, and the relationship between natural variation and augmentation. Enjoy.

I. Augmentation-induced Sensory Illusion

Here is a guided tour of human augmentation and the sensory illusions it often produces. The first example comes from Neil Harbisson and his third eye [1]. In this case, the third eye is a camera that is permanently worn, perceptually integrating this with his other two biological eyes [2]. This is a bit different than the augmented reality (AR) heads-up augmentation afforded by Steve Mann's early AR prototypes and Google Glass. In some ways, this sensory illusion mimics the prism adaptation from Neuropsychology [3]. The visual system adapts to the extra information, so that when it is removed, perception and even balance can be severely affected.

The second example is that of a third hand connected to a (second or) third arm. In 2011, a set of experiments [4] was published that demonstrate this effect. When the torso is shielded from view and a third (rubber) arm with hand is placed parallel to the other two arms (in full view of the subject), stroking or otherwise stimulating the third arm results in an emotional response that corresponds to this touching. It is not the sense of touch in a conventional sense [5], but a form of pseudo-touch enabled solely by the visual anticipation. But if you actually had a third hand, how would you control its movement? Fortunately, the artist Stelarc answered that question for us in the 1980s, when he used myoelectric control [6] to move his extraneous hand. And recent advances have been made to the design of prosthetic arms that may require us to rethink the role of dynamic touch in sensation and perception [7].

Review of closed-loop control of perception and action with relevance to prosthetic limb design. COURTESY: Reference [7].

II. Augmented Cognition

Augmented Cognition (Aug Cog) is an emerging research area that combines cognitive engineering, neuroimaging, and human-computer interaction. The vision of augmenting cognition goes back to early cybernetics work by W. Ross Ashby. Modern work in this area has grown out of a DARPA project funded in the early-to-mid-2000s. The idea is to augment or otherwise improve cognitive resources (e.g. attention, arousal, memory) using measurements of brain activity (e.g. EEG, fNIR) and computational models. 

The first three images (from top) are from the video short called AUG, written and directed by Tam Morris and Alistair Patterson. In these examples, cognition is being augmented using heads-up displays. The two images below are from a film called "The Future of Augmented Cognition", directed by Alexander Singer. These images show profiles of the pioneering scientists (Herbert Simon and Britton Chance) whose work serves as inspiration for modern AugCog efforts.

III. Natural Variation and Augmentation

Here [8] is a profile of one of my "lost" papers: "Range-based techniques for discovering optimality and analyzing scaling relationships in neuromechanical systems". This paper was published in 2009 at Nature Preceedings [9], but I'm not sure it ever got much exposure. The paper introduces something called the rescaled range technique, which I will describe below. The objective of this study was to take data collected from experiments [10] conducted in virtual environments, and then compare metrics of performance (movement behavior and muscle activity) to something called a "morphological scaling" (e.g. the proportion of forelimb length or volume to humeral length or volume). The application of mathematical modeling provides us with a tool called the "rescaled range".

What if we were to manipulate the length of these limbs well beyond their natural range? Could the tasks still be performed? Or would there be significant performance gains at certain scalings of size? The idea of the rescale range technique is to simulate this possibility based on empirical observations and mathematical modeling. The second picture from the top summarizes this idea in terms of physical implementation and the theoretical concept of hypo- and hyper-allometry [11]. 

By scaling each component of observed limb measurements by a certain factor, we effective also resample the performance data (as shown in the third and fourth figures from top). Indeed, this resampling shows that there are large-scale increases and decreases in performance for various manipulations. 

How are these findings useful? The first point is that these modeled manipulations of limb lengths serve as an analogue for what might be possible in the use and design of tools, devices used for teleoperation, and virtual representations of touch and the body. There are also a number of interesting relationships between physical perturbations and performance in such tasks [12]. This work has potentially great relevance to the design of immersive virtual environments, or touch- and movement-based virtual environments that have a significant real-world component.

IV. Towards a Cross-species Perspective

It is well-known in the field of human-robot interaction that humans recognize robots as having human-like qualities according to a non-linear function that results in the uncanny valley effect [13]. While this judgement of wheter or not a robot is human-like may be based on qualitative factors, recent neuroscience research suggests that this effect has roots in the perception-action system of the brain [14]. 

But do other species exhibit a similar effect? In [15], it was found that real fish can be attracted to the locomotion of a robotic fish. This attraction (or set of sensory cues) depends on hydrodynamic advantages [16] created by the swimming itself. In isolation, this finding could simply be a curious experimental artifact. However, the research in [17] demonstrates that fish sensorimotor learning can be manipulated using a virtual environment. So is the uncanny valley effect found in other species? A strange but fascinating world with more to come.....

NOTES:

[1] Photo courtesy Cyborg Foundation and Eveleth, R.   Ask a Cyborg. Nautil.us, Issue 001 (2013).

[2] For more information on the incorporation of cameras into the visual system, please see this Synthetic Daisies post on Steve Mann: Alicea, B.   Steve Mann, Misunderstood. Synthetic Daisies blog, July 18 (2012).

[3] Rock, I.   Orientation and Form. Academic Press, New York (1973).

[4] for more information on the science behind the supernumary hand illusion, please see: Guterstam, A., Petkova, V.I., and Ehrsson H.H.   The Illusion of Owning a Third Arm. PLoS One, 6(2), e17208 (2011).

Kunert, R.   Three fun ways to have three hands - for you at home. Brain's Idea blog, July 31 (2012). 

Parker-Pope, T.   Need an Extra Hand? NYT Well blog, February 24 (2011).

[5] For more information on the role of vision in touch (and the source of the diagram at lower right of the above image), please see: Ernst, M.O. and Bulthoff, H.H.   Merging the senses into a robust percept. Trends in Cognitive Science, 8(4), 162-169 (2004).

[6] For more information about the interplay between myoelectric control of a hand and the resulting neural responses, please see: Maruishi, M. et.al   Brain activation during manipulation of the myoelectric prosthetic hand: a functional magnetic resonance imaging study. NeuroImage, 21, 1604-1611 (2004).

[7] Kwok, R.   Once more, with feeling. Nature, 497, 176-178 (2013).

[8] More information on Figure 1: Picture at lower right from the Gritsenko Lab Website, University of West Virginia. This lab does research on the neural mechanisms behind the online correction of sensorimotor control using virtual environments. Somewhat similar to what I am getting at here -- the difference is that I am focusing more on the distortion capabilities/potential of the virtual environment itself.

Picture at lower left from the following article: Pappas, S.   Machine That Feels May Usher in 'Jedi' Prosthetics. LiveScience, October 5 (2011).

[9] Alicea, B.    Range-based techniques for discovering optimality and analyzing scaling relationships in neuromechanical systems. Nature Precedings, npre.2009.2845.2 (2009).

[10] The experiments involved reaching for and manipulating virtual objects (e.g. arm swinging and touching) with feedback. The feedback was distorted not only by the virtuality of the task, but also by distorting the tools (e.g. physics, shape) used to perform these tasks.

[11] Hypo- and hyper-allometry normally occur in the development of organisms, and usually describe evolutionary changes that unfold across related taxa. For example, in the case of the order Primates, the forearm:humerus relationship is highly variable across species, but not so much across individuals within a species.

[12] See the following papers for other work that provide more detail about these type of experiments:

Alicea, B.    Naturally Supervised Learning in Motion and Touch-driven Technologies. arXiv, arXiv: 1106:1105. [cs.HC, q-bio.NC] (2011).

Alicea, B.    Performance Augmentation in Hybrid Systems: techniques and experiment. arXiv, arXiv:0810.4629 [q-bio.NC, q-bio.QM] (2008). 

[13] For more information, please see: Moore, R.K.   A Bayesian explanation of the ‘Uncanny Valley’ effect and related psychological phenomena. Scientific Reports, 2, 864 (2012).

Guizzo, E.   Who's Afraid of the Uncanny Valley? IEEE Spectrum, April 2 (2010).

[14] Saygin, A.P., Chaminade, T., Ishiguro, H., Driver, J., and Frith, C.   The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions. Social, Cognitive, and Affective Neuroscience, 7(4), 413-422 (2012).

[15] Marras, S. and Porfiri, M.   Fish and robots swimming together: attraction towards the robot demands biomimetic locomotion. Journal of the Royal Society Interface, doi:10.1098 (2012). Video.

[16] Salumae, T. and Kruusmaa, M.   Flow-relative control of an underwater robot. Proceedings of the Royal Society A, 469, 20120671 (2013).

[17] Engert, F.   Fish in the matrix: motor learning in a virtual world. Frontiers in Neural Circuits, 6, 125 (2013).

See the following references for more information on Augmented Cognition:

[i] Schmorrow, D.   Foundations of Augmented Cognition. CRC Press (2005).

[ii] Costandi, M.   Augmented Cognition: science fact or fiction? Neurophilosophy blog, January 3 (2007).

[iii] Izzetoglu, K., Bunce, S., Onaral, B., Pourrezzaei, K., and Chance, B.   Functional Optical Brain Imaging Using NIR during Cognitive Tasks. International Journal of Human-Computer Interaction, 17(2), 211-227 (2004).

[iv] Alicea, B.   Behavioral Engineering and Brain Science in Virtual Reality. Figshare, (2012).

[v] Alicea, B.   The adaptability of physiological systems optimizes performance: new directions in augmentation. arXiv Repository, arXiv:0810.4884 [cs.HC, cs.NE] (2008).