Why AI Doom Scenarios are Misguided

I think AI doom scenarios are flawed. Here’s why.

The discussions I have seen assume the human brain is essentially a computer (or computational system). It is thought that AI systems will soon match and then exceed the brain’s computational power, leading to artificial general intelligence (AGI). Then one can speculate about why such an intelligence might cause a catastrophe.

However, the brain is not a computational system, and human cognition depends on much more than the brain.

Like most AI prognosticators, I am not an expert in cognitive science or neuroscience.  But when thinking about these fields I think there is a mistake that it easy to fall into. This is concluding that because computational models can help explain some cognitive processes/brain processes, the mind/brain is in fact a computational system.

Like in other research areas, investigations in these fields deploy models to explain phenomena. These models represent a subset of relevant processes in restricted contexts, and they employ abstractions and idealizations (and, of course, some models are not actually computational, e.g., those using a dynamical systems approach).  The actual processes being targeted are complex and inevitably entwined with other cognitive as well as non-cognitive physiological processes. Computational models capture facets of cognitive/brain functions that contribute to scientific understanding, but they provide partial and provisional pictures. It is always expected that, over time, many models will be found to be too simple or just plain wrong. But in addition, there appears to be a trend toward finding that neglected or poorly understood neurobiological details (including down to the molecular level) are important contributors to our distinctive mental life. Decades ago, philosophers of mind became confident that the mind was essentially a computational/functional system, analyzable at some coarse-grained level and insensitive to its physical realization. We’ve learned more, and I see no basis for confidence in such a view now. The capabilities of the human mind appear to be deeply rooted in our bio-physical make-up.

This is one dimension of the difficulty, but there is another, even more difficult, problem for the notion that there is a straightforward path to AGI.

The relevant unit for thinking about cognition is not the brain, it is the organism. And when we reflect on our status as autonomous, embodied animals, it becomes clear that the brain-as-computer notion lacks the resources to grapple with our incessant, directed causal engagement with our environment across scales in time and space. AI’s will struggle to capture what it is to be an independent embodied agent. Their proxies for causal interactions (inputs and outputs) will be inadequate placeholders for the complex and innovative challenges presented by the evolving real world. I side with the philosophers of mind and cognitive scientists who conclude that human cognition (as well as that of other living things) is not only shaped by the fact of embodiment but also constituted in part by causal interactions with the world.

I believe this means that AGI will require embodiment and autonomy. This may happen someday. But here I note that scientists have only recently begun to artificially replicate some of the capabilities of single-celled organisms along this dimension.

To conclude, an artificial system hoping to match the capabilities of humans must not only reckon with the actual complexity of the “inside-the-skin” processes that subserve human cognition, but also face up to the more difficult challenge of matching capabilities that have only arisen because of our autonomous embodied engagement with the world.

I see two main implications. First, obviously, is that the road to AGI is much longer and more difficult than many suppose. But also, reflecting on the organismic basis of cognition calls for a reallocation of catastrophic risk-mitigating resources: the greater near-term risks will come from attempts to artificially create or enhance simpler living/near-living things (e.g., viruses). This connects to existing concerns about biorisk.

Addendum: Here’s some great recent work related to these topics.

On the implications for functionalism of our complex neurobiology, see Rosa Cao’s “Multiple Realizability and the Spirit of Functionalism“

On embodied cognition’s implications for AI, see Lisa Miracchi’s talk at the Sante Fe Institute: “Why Intelligent Reasoning Must be Embodied.”