OpenWorm Annual Meeting 2025 (DevoWorm update)

Here are the slides for the DevoWorm group's report to the OpenWorm Annual Meeting (2024). You can watch Bradly Alicea present the talk on YouTube.

Thanks again to all of our contributors over the past year, all of our Github contributors, and all of our Google Summer of Code applicants. If you are interested in participating, join one of our meetings or contribute to our Github repo and organization.

Orthogonal Lab Annual Report for 2024

Another year of the Orthogonal Research and Education Laboratory (2024). We have had a great year! I posted a summary video on YouTube that covers activities related to Saturday Morning NeuroSim, our Open Science/Open-source interest group, the Computational Developmental Systems interest group, the Representational Brains and Phenotypes interest group, and the Cybernetics interest group. I also discuss our educational initiatives and conference/publication activities.

The seminal activity of the lab is the Saturday Morning NeuroSim meeting, which happens on Saturdays at 10AM ET (North America). This is based on the Saturday Morning Physics educational events, which originated in Germany and are popular in Physics departments and US National Labs. We had 45 meetings in 2024, and covered all manner of intersections between computational, neuroscience, social science, molecular biology, and complexity theory. 

The Computational Developmental Systems interest group can best be described as Neuro-Devo-Psych. This combines our work on Computational Critical Periods and Computational Developmental Biology (DevoWorm-associated). This also intersects with work in the Representational Brains and Phenotypes interest group and the Developmental Neurosimulation approach. 

Our Open Science/Open-source interest group sponsors Google Summer of Code participation in addition to assorted activities in the research practice and open-source project management spheres. The rejuvenation of our Cybernetics interest group features a reading group and other academic activities. 

Charles Darwin meets Rube Goldberg: a tale of biological convolutedness

Charles Darwin studying a Rube Goldberg Machine (Freepik Generative AI, text-to-image)

For this Darwin Day post (2024), I will discuss the paper Machinery of Biocomplexity [1]. This paper introduces the notion of Rube Goldberg machines as a way to explore biological complexity and non-optimal function. This concept was first highlighted on Synthetic Daises in 2009 [2], while an earlier version of the paper was discussed on Synthetic Daisies in 2013 [3]. The paper was revised in 2014 to include a number of more advanced computational concepts, after a talk to the Network Frontiers Workshop at Northwestern University in 2013 [4]. 

Figure 1. Block and arrow model of a biological RGM (bRGM) that captures the non-optimal changes resulting from greater complexity. Mutation/Co-option removes the connection between A and B, then establishes a new set of connection with D. Inversion (bottom) flips the direction of connections between C-B and C-A, while also removing the output. This results in the addition of E and D which reestablishes the output in a circuitous manner.

Biological Rube Goldberg Machines (bRGNs) are defined as a computational abstraction of convoluted, non-optimal mechanisms. Non-optimal biological systems are represented using flexible Markovian box and arrow models that can be mutated and expanded given functional imperatives [5]. Non-optimality is captured through the principle of "maximum intermediate steps": biological systems such as neural pathways, metabolic reactions, and serial interactions do not evolve to the shortest route but is constrained (and perhaps even converge to) the largest number of steps. This results in a set of biological traits that functionally emerge as a biological process. Figure 1B shows an example where maximal steps represents a balance between the path of least resistance and exploration given constraints on possible interconnections [6]. The paths from A-E, E-B, and C-D are the paths of least resistance given the constraints of structure and function. In the sense that optimality is a practical outcome of physiological function, a great degree of intermediacy can preserve unconventional pathways that are utilized only spontaneously.

This can be seen in a wide variety of biological systems and is a consequence of evolution. Evolutionary exaptation, the evolution of alternative functions, and serial innovation all result in systems with a large number of steps from input to output. But sometimes convolution is the evolutionary imperative in and of itself. As fitness criteria change over evolutionary time, traces of these historical trajectories can be observed in redundant pathways and other results of subsequent evolutionary neutrality. One example from the paper involves a multiscale model (genotype-to-phenotype) that exploits both tree depth and lateral connectivity to maximize innovation in the production of a phenotype (Figure 2). While our models are based on connections between discrete states, bRGMs can also provide insight into the evolution of looser collections of single traits and even networks, where the sequence of function is bidirectional and hard to follow in stepwise fashion.

Figure 2.  A hypothetical biological RGM representing a multi-scale relationship. Each set of elements (A-F) represents the number of elements at each scale (actual and potential connections are shown with bold and thin lines, respectively). Examples of convolutedness incorporate both loops (as with E5,1 and E5,5) and the depth of the entire network.

The paper also features extensions of the basic bRGM, including massively convoluted architectures and microfluidic implementations. In the former, interconnected networks represent systems that are not only maximal in terms of size or length, but also massively topologically complex [7]. One example of this is cortical folding and the resulting neuronal connectivity in Mammalian brains. The latter example is based on fluid dynamics and combinatorial architectures that are more in line with discrete bRGMs (Figure 3). 

Figure 3. A microfluidic-inspired bRGM model that mimics the complexity of biological fluid dynamics (e.g. blood vessel networks). G1, G2, and G3 represent iterations of the system.

References:

[1] Alicea, B. (2014). The "Machinery" of Biocomplexity: understanding non-optimal architectures in biological systems. arXiv, 1104.3559.

[2] Non-razors, unite! January 30, 2009. https://syntheticdaisies.blogspot.com/2009/01/non-razors-unite.html

[3] Maps, Models, and Concepts: July Edition. Synthetic Daises blog. July 13, 2013. https://syntheticdaisies.blogspot.com/2013/07/maps-models-and-concepts-july-edition.html

[4] Inspired by a visit to the Network's Frontier....  Synthetic Daises blog. December 16, 2013. https://syntheticdaisies.blogspot.com/2013/12/fireside-science-inspired-by-visit-to.html

[5] when dealing with a large number of steps or in a polygenic context, these types of models can also resemble renormalization groups. For more on renormalization group, please see: Wilson, K.G. (1975). Renormalization group methods. Advances in Mathematics, 16(2), 170-186.

[6] this balance is as predicted by Constructive Neutral Evolution (CNE). For a relevant paper, please see: Gray et.al (2010). Irremediable Complexity? Science, 330(6006), 920-921.

[7] in the paper, this is referred to as Spaghettification, a term borrowed from the physics of gravitation. See this reference for an interesting implementation of this in soft materials:  Bonamassa et.al (2024). Bundling by volume exclusion in non-equilibrium spaghetti. arXiv, 2401.02579.

Saturday Morning NeuroSim Discussion Thread: Causality

Over the past three years, the Saturday Morning NeuroSim group has met weekly on Saturdays (mornings in North America). The Saturday Morning format continues in the tradition of Saturday Morning Physics and covers a wide variety of topics.

Our discussion thread on causality begins with Causality and Circles on May 13. From a Mastodon post by Yohan John, we considered how spatialized diagrams are confused with temporal sequences in a feedback loop. We also covered three papers in this session.

Vernon, D., Lowe, R., Thill, S., and Ziemke, T. (2015). Embodied cognition and circular causality: on the role of constitutive autonomy in the reciprocal coupling of perception and action. Frontiers in Psychology, 6, 1660.

Raginsky, M. (2023). Directed Information and Pearl’s Causal Calculus. arXiv, 1110.0718.

Laland, K.N., John Odling-Smee, J., Hoppitt, W., and Uller, T. (2013). More on how and why: cause and effect in biology revisited. Biological Philosophy, 28, 719–745.

Our conversation continued after the last week of Neuromatch Academy, when the NMA curriculum featured causal networks. Our July 29 meeting featured a collection of references on Bayesianism, Probabilistic Graphical Models, methods of integration, time-series applications, and more. Some core readings are given below.

Stanford Encyclopedia of Philosophy: causal models. This article takes an epistemological approach and provides us with a baseline for structural equation model, graphical probabilistic models, and other statistical formulations of causal relationships.

Daphne Koller’s Probabilistic Graphical Models course. Hosted on Stanford University’s Open Classroom platform, this course includes units on representation, inference, learning, and causation. The causation unit covers decision theory, utility functions, influence diagrams, and the notion of perfect information.

Pearl, J. (2000). Causality. Cambridge Press, Cambridge, UK. This classic book by Judea Pearl builds from a theory of inferred causation, starting at causal diagrams, and continuing through direct effects, indirect effects, confounds, counterfactuals, bounding effects, and probabilities. The book also covers structural models, decision analysis, and Simpson’s Paradox as the basis for methods for detecting causal relationships.

Scholkopf, B. (2019). Causality for Machine Learning. arXiv, 1911.10500.

Heckman, J.J. (2005). The Scientific Model of Causality. Sociological Methodology, 35, 1–97. Causality from an econometrics point-of-view. Counterfactuals are a set of possible outcomes generated by determinants. A causal effect is defined by the change in the manipulated factor where amongst a set of factors, in a situation where all but one is held constant.

Taskesen, E. (2021). A step-by-step guide in detecting causal relationships using Bayesian structure learning in Python. Towards Data Science, September 7.

Which variables have a direct causal effect on a target variable? Hint: association and correlation are not equivalent to causation.

Bayesian Models:

Neuberg, L.G. (2003). Causality: models, reasoning, and inference. Econometric Theory, 19, 675–685.

Pearl, J. (2001). Bayesianism and causality, or, why I am only a half-Bayesian. In “Foundations of Bayesianism”, pgs. 19–36. Kluwer Press.

Methods of Interaction: networks and non-directional graphs, as opposed to directed acyclic graphs (DAGs), require a different set of considerations. The methods below cover highly interacting systems like graphs and how change over time can be properly interpreted as causal.

Leng, S., Ma, H., Kurths, J., Lai, Y-C., Lin, W., Aihara, K., and Chen, L. (2020). Partial cross mapping eliminates indirect causal influences. Nature Communications, 11, 2632.

Park, S.H., Ha, S., and Kim, J.K. (2023). A general model-based causal inference method overcomes the curse of synchrony and indirect effects. Nature Communications, 14, 4287.

From the Granger Causality Wikipedia entry.

Time-series using Granger Causality: the first two references apply Granger Causality to time-series datasets. In such cases, the datapoints are dependent with respect to time. Given two time-series x and y, x is the cause of y if x predicts y (lagged with respect to x over a certain time interval) given x and prior values of y. This is in comparison with simply predicting the current value of y given previous values of y, which would be the counterfactual case.

The final paper in the group (Stokes and Purdon) critiques Granger Causality from a Neuroscience perspective.

Carlos‐Sandberg, L. and Clack, C.D. (2021). Incorporation of causality structures to complex network analysis of time‐varying behaviour of multivariate time series. Scientifc Reports, 11, 18880.

Runge, J., Nowack, P., Kretschmer, M., Flaxman, S., Sejdinovic, D. (2019). Detecting and quantifying causal associations in large nonlinear time series datasets. Science Advances, 5(11), aau4996.

Stokes, P.A. and Purdon, P.L. (2017). A study of problems encountered in Granger causality analysis from a neuroscience perspective. PNAS, 114(34), E7063-E7072.

The third session (August 5) was a focus on causality specifically as it is treated in Neuroscience. This session followed up on a Twitter debate by Kording Lab and Earl Miller about the role of causality in neuroscience. The consensus to the question “Why is Neuroscience so into causality?” was that it provides a means to identify mechanisms for function. Causality in neuroscience differs from philosophical discussions about causality in that Neuroscience must infer causality from data, while philosophers (and statisticians) do the work of proving causality.

One interesting point from Kording Lab is that there is a difference between proximate causes and ultimate causes. In some fields, causality is obvious and so causal methods are not always necessary. But Neuroscience is partially about the behavioral substrate, and so we can turn to Niko Tinbergen’s four questions. The four questions concern 1) how a trait arose in development (proximate, dynamic), 2) how a trait arose in evolution (ultimate, dynamic), 3) what is the mechanism or structure of a trait (proximate, static), and 4) what is the adaptive value or function of a trait (ultimate, static).

You can read more about Tinbergen’s four questions and their causal implications in the following papers.

Beer, C. (2020). Niko Tinbergen and questions of instinct. Animal Behaviour, 164, 261–265.

Nesse, R.M. (2019). Tinbergen’s four questions: two proximate, two evolutionary. Evolution, Medicine, and Public Health, 2, doi:10.1093/ emph/eoy035.

Mayr, E. (1961). Cause and effect in biology. Science, 134, 1501–1506.

The other papers from this session focused on mental representations and causal functional connectivity in the brain, respectively.

Sloman, S.A. and Lagnado, D. (2015). Causality in Thought. Annual Reviews in Psychology, 66, 223–247.

While Bayesian approaches are good for theory-building, they are an incomplete account of what goes on in the cognitive world.

Biswas, R. and Shlizerman, E. (2022). Statistical perspective on functional and causal neural connectomics: The Time-Aware PC algorithm. PLoS Computational Biology, 18(11), e1010653.

The fourth session (August 19) picks up on a point covered in the second session, namely how causality can be inferred from network data. This covers related ideas of transitivity, weak interactions, and anti-causal models. Papers for this session include networks in ecology, anticipative and non-anticipative control theory, and anti-causal systems.

Typology of causal models for past, present, and future events.

Naghshtabrizi, P. and Hespanha, J.P. (2006). Anticipative and non-anticipative controller design for network control systems. Lecture Notes in Control and Information Science, 331.

Sugihara, G., May, R., Ye, H., Hsieh, C-H., Deyle, E., Fogarty, M., Munch, S. (2012). Detecting Causality in Complex Ecosystems. Science, 338, 496–500.

Chattopadhyay, I. (2014). Causality Networks. arXiv, 1406.6651.

Anticausal System. Wikipedia.

McCurdy, T. (2007). Causal Systems: understanding the basics. Physics Forums. September 23.

From the Necessity and Sufficiency Wikipedia entry.

Finally, some fields (cell and molecular biology) have working models of causation that while useful, are not particularly illuminating. In the cell and molecular biology example, the traditional model of necessity and sufficiency (a mechanism being necessary but not sufficient) can be criticized for not being complete with respect to incorporating counterfactuals or multiple potential causes. See this paper for more information:

Bizzarri, M., Brash, D.E., Briscoe, J., Grieneisen, V.A., Stern, C.D., and Levin, M. (2019). A call for a better understanding of causation in cell biology. Nature Reviews Molecular Cell Biology, 20, 261–262.

MAIN and Neuromatch Conference Presentations

The Orthogonal Research and Education Lab is on the virtual move! We have been featured at two conferences this week. The first conference is MAIN (Montreal Artificial Intelligence-Neuroscience) conference, a hybrid conference that focused on cutting-edge research in Neuro-AI. Our submission (Developmental Embodied NeuroSimulation) is a group effort and summarizes our work in this area over the past few years. The graphical abstract can be found below.

Graphical Abstract for Developmental Embodied NeuroSimulation.

We also had a presence at Neuromatch 4, with four flash talk presentations on four different topics. Neuromatch 4 was a great time, with two days of keynote talks, short talks, flash talks, and debate panels. 

Each flash talk was 7.5 minutes long, which requires an efficiency of words and ideas not typical of a longer format. The first talk is "The Universal Theory of Switching", which focuses on transitory "switching" phenomena. Switching behavior is ubiquitous across biological, physical, and algorithmic systems, and is controlled by sudden, first-order phase transition-like behavior we characterize as zeroth-order cybernetic regulation. 

Another talk is on "Allostatic Kinds". Allostatic Kinds are a way to regulate the boundaries of meaning and regulation of internal emotional and conscious states. This talk is presented by Jesse Parent, and features a mix of complex systems regulation, philosophy of mind, and consciousness studies. This talk was in conjunction with CEEALAR (Center for Enabling EA Learning and Research), an academic hostel located in Blackpool, UK.

Daniela Cialfi has built upon the lab's work on Meta-brain Models to develop "Economic Meta-brains", which are bio-economic agents that behave according to the free energy principle. Meta-brains are layered computational models that enable different levels of representation in the same agent. These model layers can be configured in geometrically specific ways, which in turn affects their function. The free energy principle enriches the meta-brains approach by adding a mathematically rigorous energetic component to a meta-brain agent. 

Finally, our presentation on "Gibsonian Information" comes with a preprint. Gibsonian Information is the information content of direct perceptual processing (sensu J.J. Gibson). We draw parallels between Shannon and Gibsonian Information, in addition to the role of such information in the dynamic interactions between agents and their environments. See our graphical abstract below, which simplifies the mathematics in the preprint. The talk also features a number of naturalistic settings in which Gibsonian Information can be demonstrated.

Graphical abstract for the Gibsonian Information paper/presentation (direct perception as information content).

Hybrid Models and Representations, latest developments

Our research group Representational Brains and Phenotypes has recently been doing a lot of work in the area of embodied nervous systems simulation. Our go-to model over the past year and a half [1] has been the Braitenberg Vehicle (BV), which makes explicit the relationships between interactions with the environment, a neuronal network, and behavior. In the recently open-sourced Meta-Brain Models initiative, we are looking beyond the highly-limited representational framework of the BV by creating hybrid models that combine Vehicles with a representation-rich model (one that specifies symbolic and semantic properties).  

Representational Brains and Phenotypes website. Click to enlarge.

A few weeks ago, members of our research group presented on this topic at the DevoNN (virtual) workshop. Our paper and presentation "Developmental Embodied Agents as Meta-brain Models" introduced the Meta-brain concept to a wider audience, which in this case coupled a developmental Braitenberg Vehicles (dBV) and Contextual Geometric Sturctures (CGS) [2,3]. dBV and CGS are coupled as layers that interact in ways that resemble central nervous systems [4]. Other model combinations are possible, but must conform to the representation-free/representation-rich layering.

 

DevoNN logo and the layered representations approach. Click to enlarge.

This has the potential to leverage not only embodied aspects of intelligent behavior, but its symbolic aspects of as well. There are also opportunities to expand our layered representations in the direction of growth, form, and analogues of biological plasticity [5]. DevoNN was part of Artificial Life 2020, which was in itself a great virtual conference experience. Everything from digital evolution to robotics, and from computational social science to adaptive systems. 

Artificial Life 2020 (proceedings here). Click to enlarge.

While our extended abstract is not in the Proceedings, I was also able to given a lightning talk on the OpenWorm approach to virtual organisms. 

OpenWorm lightning talk. Click to enlarge.

NOTES:

[1] Check out our Saturday Morning NeuroSim research meetings on YouTube. Contact the Orthogonal Research and Education Lab if you would like to join in!

[2] Dvoretskii, S., Gong, Z., Gupta, A., Parent, J., and Alicea, B. Braitenberg Vehicles as Developmental Neurosimulation. arXiv, 2003.07689. doi:10.13140/RG.2.2.31149.23526.

[3] Alicea, B. Contextual Geometric Structures: modeling the fundamental components of cultural behavior. Proceedings of Artificial Life, 13, 147-154.

[4] examples include the relationship between the Thalamus and Neocortex of Mammals. For functional and evolutionary context, see: Karten H.J. (2015). Vertebrate brains and evolutionary connectomics: on the origins of the mammalian "neocortex". Philosophical Transactions of the Royal Society B, 370, 20150060

[5] this recent paper is the subject of a presentation at the Dynamics Days Digital conference, but is providing some inspiration for further development towards AI innateness: Alicea, B. (2020). Developmental Incongruity as a Dynamical Representation of Heterochrony. ResearchGate, doi:10. 13140/RG.2.2.17401.08807/1

Virtuality: a new view on virtual experience

The first images when you Google "virtuality". Products from the Virtuality Group. LEFT: Virtual Reality Society, RIGHT: Wikimedia (Dr.  Jonathan D. Waldern/Virtuality Group).

At Neuromatch 2.0, I gave a talk on the concept of virtuality titled "Computational Virtuality as a Form of Artificial Intelligence" (slides available on Figshare). Virtuality is a means to summarize all brain and body systems involved in a virtual experience. This talk features two big ideas, the second of which the Saturday Morning NeuroSim group will be building on in the coming months.  

1) "Being Virtual" as Human Performance (click to enlarge).

2) Differentiating Allostasis machine (click to enlarge). Allostatic load is induced collectively by stimulus [1], leads to dysregulation of system and ultimately to a new homeostatic setpoint.

Overall, virtuality can be characterized as a constructive sensory experience (h/t to Anson Lim). The first big idea stems from the observation that current theories are too limiting to properly characterize virtual experiences at the systems-level. Indeed, the very notion of objective reality is based in part on the synchronization of perception and action. When this synchronization is disrupted or perturbed in any way, we observe virtuality. This does not even require a virtual environment -- virtuality can be experienced in our interaction with the physical world [for an example from fly-human interaction, see 2]. But virtual environments are the most common way to elicit the allostatic load required to observe virtuality.

There are two aspects of embodied performance essential to understanding the virtuality effect. The first is the idea of cognitive gaps, or disruptions to spatiotemporal representations caused by the perturbation of reality. The second involves inducing allostatic load in the internal model the represents perception-action coupling. The latter has wide-ranging effects in the homeostat (nervous system), and can lead to many different effects of varying length scales.

But where does the artificial intelligence part come in? In the second part of the talk, I conclude that the effect sizes for human experiments are too small and formal experimental design is too limiting to demonstrate virtuality. This is particularly true for observing virtuality over long periods of time. Therefore, we can use computational agents! To propose a general approach, we want to identify two attentional and three sensorimotor features essential for any agent to exhibit virtuality.

Naturally, we want our agents to be embodied in some manner. We propose working with two types of agent: Morphogenetic Agents, a novel agent type that can exhibit both pattern recognition and morphogenesis (percepetion-action), and the well-known Braitenberg Vehicle, in this case experiencing incongruous environmental physics which leads to gaps in the perception-action loop. 

NOTES:

[1] this requires a ecological view of perception. For a short introduction, please see: Lobo, L., Heras-Escribano, M. and Travieso, D. (2018). The History and Philosophy of Ecological Psychology. Frontiers in Psychology, doi:10.3389/fpsyg.2018.02228.

[2] Alicea, B. (2013). Perceptual Time and the Evolution of Informational Investment. Synthetic Daisies blog, September 24.