Dispatches from the Emergent Interaction Workshop

 This content has been cross-posted at the Orthogonal Lab Medium.

Last month, the Orthogonal Lab was represented at the Emergent Interaction Workshop (part of SIGCHI 2021). We contributed a paper titled “Allostasis Machines: a model for understanding internal states and technological environments”, with a companion presentation now on YouTube. Thanks go to Bradly Alicea, Daniela Cialfi, Anson Lim, and Jesse Parent for their contribution. We are planning an expanded version of this work with Rishabh Chakraborty that will demonstrate Allostasis Machines as a Reinforcement Learning implementation.

The subtitle of this workshop was “Complexity, Dynamics, and Enaction in HCI”. Therefore, the focus was on advancing measurement and theory, in addition to better characterize complexity in the field of Human-Computer Interaction. The four-hour long session was summarized in our weekly meeting on May 22. I have also provided supplemental readings in four workshop-related categories at the bottom of this post.

Overview of the Emergent Interaction Miro board.

There were six other papers made available before the session. Two of the most interesting to the Orthogonal Lab group are “Fields of Affordances and Human Computer Interaction” by Jelle Bruineberg and “Simulating Social Acceptability With Agent-based Modeling” by Alarith Uhde and Marc Hassenzahl.

The Emergent Interaction utilized Zoom, Slack, and a Miro board to enable discussion during the session. Check out the overview paper titled “Emergent Interaction: Complexity, Dynamics, and Enaction in HCI” for more information.

Testing, 1, 2, Emergent T-shirt….

There were a number of interesting and innovative topics discussed in the workshop. Dynamical approaches came up several times, along with topics such as multifractality, attractor analysis, and co-evolutionary experimental design. For more information regarding the first two topics, check out Alan Dix’s blog on Making Sense of Quantitative Data, and Dan Bennett’s preprint “Multifractal Mice: Measuring Task Engagement and Readiness-to-hand via Hand Movement”.

Tom Froese presented on his Enactive Artificial Intelligence and HCI work. His Google Scholar profile features some really interesting work that cuts across the worlds of Artificial Life, Cybernetics, and Cognitive Science, but his workshop topic was how modern Machine Learning approaches are insufficiently embodied. I have posted references to two of his key works (workshop-wise) in the Further Readings section of this post.

Later, Parisa Eslambolchilar presented on first-order closed-loop feedback taking the form of sensor-based human interaction loops. She reviewed some of the things she developed in her Doctoral dissertation, then lead us into her more recent work. Learn more by reading “A Model-Based Approach to Analysis and Calibration of Sensor-based Human Interaction Loops”.

Then, Vassilis Kostakos discussed his work on modeling interactions between technology users (or users and interfaces) as a complex system. He utilized the “lynx-hare” predator-prey analogy, inspired by Lotka-Volterra co-evolutionary dynamics. Read more in this paper published last year in Human-Computer Interaction: “Modeling interaction as a complex system”.

Emergent Interaction is now on Twitter! Give them a follow to join the discussion.

Further Reading: Measurement techniques.

Rebout, N., Lone, J-C., De Marco, A., Cozzolino, R., Lemasson, A., and Thierry, B. (2021). Measuring complexity in organisms and organizations. Royal Society Open Science, 8, 200895.

Zhou, Q., Chua, C-C., Knibbe, J., Goncalves, J., and Velloso, E. (2021). Dance and Choreography in HCI: A Two-Decade Retrospective. Proceedings of CHI, 262, 1–14. Video

Further Reading: Enactive Approaches to Artificial Systems.

Froese, T. and Ziemke, T. (2009). Enactive artificial intelligence: Investigating the systemic organization of life and mind. Artificial Intelligence, 173, 466–500.

Froese, T., McGann, M., Bigge, W., Spiers, A., and Seth, A.K. (2012). The Enactive Torch: A New Tool for the Science of Perception. IEEE Transactions on Haptics, 5(4), 365–375.

Further Reading: Agent-based Modeling approaches.

Bonabeau, E. (2002). Agent-based modeling: Methods and techniques for
simulating human systems. PNAS, 99(3), 7280–7287.

Grimm, V., Revilla, E., Berger, U., Jeltsch, F., Mooij, W.M., Railsback, S.F., Thulke, H-H., Weiner, J., Wiegand, T., and DeAngelis, D.L. (2005).
Pattern-Oriented Modeling of Agent-Based Complex Systems: Lessons from Ecology. Science, 310, 987.

Further Reading: Criticalities and Characterizing Systems.

Dotov, D.G., Nie, L., and Chemero, A. (2010). A Demonstration of the Transition from Ready-to-Hand to Unready-to-Hand. PLoS One, 5(3), e9433.

Kelso, J.A.S. (2021). Unifying Large-and Small-Scale Theories of Coordination. Entropy, 23(5), 537.

Observer-dependent Models @ the Philosopher's Web Cafe

 

I gave a talk called "Observer-dependent Models" to the Philosopher's Web Cafe on December 11. I have made the slides available here, and the recording is here. Thanks to Jesse Parent (Orthogonal Lab Manager) and Charlotte Guo (series host) for hosting. It will be almost like being there (almost). 

The talk involved reviewing and redefining the role of observers in empirical and simulated systems. "Observer" refers mostly to computational agents (agent-based simulation and AI), although many of the ideas introduced here may apply to the analysis of empirical observations (experiments). Here is the abstract:

In many areas of science and philosophy, observers are seen as an integral part of understanding the natural world. Aside from a pedagogical role, observers are seen as less important in computational forms of inquiry. In this talk, I will reconsider a role for the observer in computational models as fully integrated with the agent. perhaps more fundamentally, causal outcomes and system dynamics are seen to be contingent on observers, while empirical observations themselves are dependent upon the actions of observers. As this is an article of faith in some interpretations of quantum mechanics, we extend this to algorithmic systems with a combinatorial solution space. The role for observers in computational and empirical investigations is established superficially using a number of concepts, including cybernetics, embodiment, and perceptual information processing. Then we will be introduced to more concrete examples of observer-oriented computational agents, such as observer-emitter systems and viewpoint networks. Finally, we will discuss how this approach goes beyond constructivism to consider multiple observers, multiple perspectives (relativism), and how they affect the interpretation of results.

There is a lot to follow up on from this talk, including a number of themes to explore within the topic of agent-based observers, with more to come in the new year. 

Presentations at Neuromatch (NM)3

Neuromatch 3 (NM3) happened a few weeks ago in virtual space, and it was great! There were hundreds of presentations over five days (October 26-30), many of them already archived on YouTube. This version of Neuromatch took the place of Society for Neuroscience (SfN), which was cancelled due to COVID. In this sense, Neuromatch is proving itself to be an improvement on the legacy conference. Between my two research groups (Representational Brains and Phenotypes and DevoWorm), we had five presentations submitted to NM3. Let's go through them one by one.

Contrast Between Biological and Artificial Neural Networks (BNNs vs ANNs). Download slides from Figshare. Video on YouTube. Click to enlarge.

This presentation was by Krishna Katyal and myself. Krishna is a regular contributor to the DevoWorm group. This presentation demonstrates several contrasts between Biological and Artificial Neural Networks, and how level of abstraction, network structure, and energetics all play a role in distinguishing the information-processing marvels known as biological brains.

Approaching Artificial Intelligence as an Embodied Developmental Process. Download slides from Figshare. Video on YouTube. Click to enlarge.

Thinking more along the lines of biological development, the next presentation features work done in the Representational Brains and Phenotypes Group. Taking a developmental approach to the classic Braitenberg Vehicle, we demonstrate how embodied developmental principles can be used to shape and guide networks for learning. Topics such as developmental contingency and the difference between morphogenesis and learning during the development of an artificial nervous system were also discussed.

The Psychophysics of Non-neuronal Cognition. Download slides from Figshare. Video on YouTube. Click to enlarge.

Are the advantages of cognitive information processing limited to organisms with a brain? This seems like a strange question, but can actually help us understand what a brain does and why it is important for coordinating the behavior of complex multicellular systems. In this presentation, which includes contributors to both the Representational Brains and Phenotypes and DevoWorm groups, we reconsider a model of Diatom movement as cognitive information processing. We also propose a series of potential scenarios in which this information processing occurs as well as Psychophysical-like measures to quantify these phenomena.

Trajectories in Cognition Studies - a focus on embodiment and future frontiers. Click to enlarge.

This abstract was submitted by Jesse Parent and Anson Lim, two regular contributors to the Orthogonal Research and Education Lab. Jesse is also a group leader and community manager in the lab. While they were not able to present during the scheduled time, they continue to work on this topic under the new and emerging Cognition Futures project.

Overcompensation of lens-accommodation in case of Convergence Insufficiency- A theoretical model. Click to enlarge.

The final talk was by Akshara Gopi (a regular contributor to the Orthogonal Research and Education Lab), along with myself and Ashwin Irungovel. This presentation was on convergence insufficiency in human vision, a topical specialty of Akshara and Ashwin. My contribution was to propose an agent-based model for this phenomenon. 

Check out all of the great talks at NM3, including one by Rishabh Chakrabarty (regular research contributor to the Orthogonal Lab) and his co-author called "Seeing through the Mind’s Eye: reconstruction of the visual stimuli using 3D Generative-Adversarial Modeling". This intriguingly titled talk features research that combines Neuroimaging data with Deep Learning.

If you are interested in these topics and want to be involved in this research, join us at our weekly research meetings (Saturday Morning NeuroSim and DevoWorm group), or watch them on YouTube! I also invite you to join the Orthogonal Research and Education Lab Slack or OpenWorm Slack (DevoWorm) for continued discussion. Hope to see everyone at Neuromatch 4!

ASAPBio Session on the "Past, Present, and Future of Preprints"

For Open Access Week 2020, Synthetic Daisies will feature an exciting panel discussion on preprints. On Monday (19th), I was part of a panel called "Past, Present, and Future of Preprints", hosted by ASAPBio. I live tweeted the event from the Orthogonal Research and Education Lab Twitter account. If you were not able to attend in person, the recording is on YouTube! The session started with a short introduction from each of our participants: Antonis Rokas, Soumya Swaminathan, Richard Sever, Ross Mounce, and Anjana Badrinarayan.

Yamini Ravichandran and Marco Fumasoni started us off with a short introductory presentation, followed by an introduction by each of our panelists. This part of the session culminated with Marco posing an initial question to the panel.

It turns out that there are many contributing factors to preprint adoption. Some of them involve legacy patterns from manuscript submissions and publications. But preprints also democratizes access to both the production and consumption of scientific literature. It turns out that cultural traditions (within fields and countries), researcher agency, and community incentives are also quite important.

The theme of research culture came up time and time again. But research culture is not only a motivating factor; pro-preprint behaviors can lead to other virtuous practices. For example, Ross Mounce suggests that preprints can encourage a culture of versioning, where different versions of a paper are viewed as important steps in the research process rather than simply being erratum.

There was also a discussion of the role traditional journals play in the research dissemination process. One future direction of preprint culture is to decouple papers from journals. Towards the end of our session, we heard a choice quote from Antonis Rokas and the Rokas Lab.

This combines nicely with observations earlier in the session regarding citation metrics: with the movement towards iteratively-developed preprints with multiple supporting components (open data sets, supplemental figures and notes), there will be a need to distinguish article quality from journal quality. Altmetrics are one path forward, but a more robust system is needed. 

Thanks to everyone for participating! Thanks also go to Sarah Stryeck, Jessica Polka, and of course Iratxe Puebla for being a great community manager! Happy Open Access Week! 

UPDATE (11/3): A recording of the session is now on YouTube!

Multidimensional Chess Convoluted to a Pretty Picture

At last Monday's DevoWorm Group meeting, I gave a short lecture on ways to interpret multidimensional data using PCA, tSNE, and UMAP.  Here are the slides (below) and the YouTube link. The focus here is on Developmental Molecular Biology, but are generally useful, particularly for comparing methods. Here are the slides with a bonus from Leland McInnes, one of the originators of the UMAP technique!

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Click to enlarge.

Hybrid Models and Representations, latest developments

Our research group Representational Brains and Phenotypes has recently been doing a lot of work in the area of embodied nervous systems simulation. Our go-to model over the past year and a half [1] has been the Braitenberg Vehicle (BV), which makes explicit the relationships between interactions with the environment, a neuronal network, and behavior. In the recently open-sourced Meta-Brain Models initiative, we are looking beyond the highly-limited representational framework of the BV by creating hybrid models that combine Vehicles with a representation-rich model (one that specifies symbolic and semantic properties).  

Representational Brains and Phenotypes website. Click to enlarge.

A few weeks ago, members of our research group presented on this topic at the DevoNN (virtual) workshop. Our paper and presentation "Developmental Embodied Agents as Meta-brain Models" introduced the Meta-brain concept to a wider audience, which in this case coupled a developmental Braitenberg Vehicles (dBV) and Contextual Geometric Sturctures (CGS) [2,3]. dBV and CGS are coupled as layers that interact in ways that resemble central nervous systems [4]. Other model combinations are possible, but must conform to the representation-free/representation-rich layering.

 

DevoNN logo and the layered representations approach. Click to enlarge.

This has the potential to leverage not only embodied aspects of intelligent behavior, but its symbolic aspects of as well. There are also opportunities to expand our layered representations in the direction of growth, form, and analogues of biological plasticity [5]. DevoNN was part of Artificial Life 2020, which was in itself a great virtual conference experience. Everything from digital evolution to robotics, and from computational social science to adaptive systems. 

Artificial Life 2020 (proceedings here). Click to enlarge.

While our extended abstract is not in the Proceedings, I was also able to given a lightning talk on the OpenWorm approach to virtual organisms. 

OpenWorm lightning talk. Click to enlarge.

NOTES:

[1] Check out our Saturday Morning NeuroSim research meetings on YouTube. Contact the Orthogonal Research and Education Lab if you would like to join in!

[2] Dvoretskii, S., Gong, Z., Gupta, A., Parent, J., and Alicea, B. Braitenberg Vehicles as Developmental Neurosimulation. arXiv, 2003.07689. doi:10.13140/RG.2.2.31149.23526.

[3] Alicea, B. Contextual Geometric Structures: modeling the fundamental components of cultural behavior. Proceedings of Artificial Life, 13, 147-154.

[4] examples include the relationship between the Thalamus and Neocortex of Mammals. For functional and evolutionary context, see: Karten H.J. (2015). Vertebrate brains and evolutionary connectomics: on the origins of the mammalian "neocortex". Philosophical Transactions of the Royal Society B, 370, 20150060

[5] this recent paper is the subject of a presentation at the Dynamics Days Digital conference, but is providing some inspiration for further development towards AI innateness: Alicea, B. (2020). Developmental Incongruity as a Dynamical Representation of Heterochrony. ResearchGate, doi:10. 13140/RG.2.2.17401.08807/1