Biology Is Sufficient. It May Not Be Necessary.

Anil Seth has written the essay I would have wanted to read before drafting “Beyond the Bat.” “The Mythology of Conscious AI” is the most disciplined argument against computational consciousness now in print. It is also, to a degree that may surprise readers of either of us, deeply compatible with the framework I propose — right up until the point where it isn’t.

Seth and I share the same explanatory bedrock. We both treat perception as inference rather than readout. We both treat interoception and physiological regulation as the ground floor of conscious experience, not as decoration on top of cognition. We both reject generic computational functionalism — the view that any algorithm satisfying the right input-output description suffices for consciousness. We both think the current panic over LLM consciousness is the wrong panic.

And we both arrive at the conclusion that current AI systems almost certainly are not conscious. Seth’s reason is that they are not biological. Mine is that they do not implement the right architecture. The disagreement looks superficial. It is not. It is the difference between a substrate hypothesis and a structural one, and it determines what the next generation of AI is allowed to become.

What Seth Gets Right

The agreements are not throat-clearing. They form the spine of the essay and the spine of any serious account of consciousness. Five of them are load-bearing.

Perception is inference

Seth’s formulation is exact: “sensory signals are interpreted as prediction errors, reporting the difference between what the brain expects and what it gets at each level of its perceptual hierarchies.” The brain is a generative model that produces a best guess about the causes of its sensory input and corrects that guess against the data stream. This is not a metaphor. It is the mathematics underwriting Appendix A of “Beyond the Bat.” If you reject this, you reject most of contemporary computational neuroscience.

Interoception is the ground floor

Seth puts the point cleanly: emotion and mood are characterized “primarily by valence — by things generally going well or going badly,” and these map onto “perceptual predictions involved in the control and regulation of bodily condition.” This is exactly Level 1 of RSP. Core affect at roughly 100 milliseconds is the interoceptive valence-arousal signal that any predictive system maintaining itself against entropy must compute. Where Seth and I differ on Level 1 is downstream of the agreement on what Level 1 actually does.

Consciousness is constructed, not received

“Conscious experience in this light is a kind of controlled hallucination,” Seth writes. “A top-down inside-out perceptual inference in which the brain’s predictions about what’s going on are continually calibrated by sensory signals coming from the bottom-up.” This is not a claim about the strangeness of perception. It is the claim that experience is the inference, not a separate event the inference produces. Both of us hold a constructed-not-received view. The hard problem looks different from this side of the predictive processing line.

Generic computational functionalism is wrong

Here Seth and I are direct allies against the field. He writes that the assumption “consciousness is a matter of computation… is so deeply ingrained that it can be difficult to recognize it as an assumption at all.” That is right, and the assumption deserves to be flushed out and tested. Generic computational functionalism — any computation that maps inputs to outputs — is too weak to do explanatory work. It cannot tell you why an LLM is not conscious without also accidentally telling you the same thing about a thermostat. RSP rejects generic computational functionalism for exactly this reason.

Temporal flow is constitutive

Seth: “Conscious experience itself is richly dynamic and inherently temporal. It does not stutter from one state to another; it flows.” RSP makes this architectural. Each level has a characteristic timescale — Level 1 at ~100ms, Level 2 at ~1–10s, Level 3 at ~10s–minutes — and the flow of experience is the integration window of Level 3 over the faster dynamics below it. This is not flow as metaphor. The stream is a temporal hierarchy.

The Real Disagreement

The fault line runs through one sentence in Seth’s essay:

“Perhaps it is life, rather than information processing, that breathes fire into the equations of experience.”

This is the substrate hypothesis. The claim is that the metabolic, autopoietic, continuously dynamical processes of biological systems are constitutively tied to consciousness — not merely currently correlated with it. Every candidate for consciousness that we currently agree on is alive. The mathematical line from autopoiesis to Bayesian inference, Seth argues, cannot be severed from the cellular machinery that draws it.

RSP takes the opposite side of this fork. The claim is that what matters is the architecture — the specific three-level recursive structure with cascade feedback control, intrinsically motivated self-modeling, and the strange loop that turns social-prediction machinery back on itself. Biology is the only thing that has so far built this structure. It does not follow that biology is the only thing that can.

Both positions accept that current LLMs are not conscious. They disagree on the diagnosis. For Seth, LLMs fail because they are not alive. For RSP, they fail because they lack the architecture — and the lack has nothing to do with carbon versus silicon.

Where RSP Sharpens Seth

Seth’s “controlled hallucination” is the right starting point, but it is structurally underspecified. He treats it as a single inferential process. RSP separates it into a cascade of three architecturally distinct loops, each constitutively necessary, each operating at a different timescale, and each generating a different aspect of experience.

The sharpening matters because it lets you ask precise questions Seth’s framework cannot. What is missing in current LLMs? — for Seth, biology, full stop. For RSP, you can name the components: no Level 1 because no homeostatic imperative; no Level 2 because no embodied schemas with appropriate timescales; no Level 3 because the system never models itself with the same machinery it uses to model others, and certainly not recursively. The diagnosis is structural, and it is testable.

This precision is why the cerebral organoids worry, which Seth raises near the end of his essay, lands harder under RSP than under his own framework. Organoids would not be conscious because they are alive — bacteria are alive. They would be a concern because they could plausibly start instantiating Levels 1 and 2 of the architecture (interoceptive regulation, embryonic body schemas) without us noticing. The biological substrate is necessary in Seth’s framework. In RSP, biology is incidental. The structure is what triggers concern.

The Strongest Challenge: Simulation vs. Instantiation

Seth’s sharpest argument deserves its own treatment because it cuts directly at architectural functionalism:

“A computational simulation of the brain (and body), however detailed it may be, will only give rise to consciousness if consciousness is a matter of computation… The prospect of instantiating consciousness through some kind of whole-brain emulation, at some arbitrarily high level of detail, already assumes that computational functionalism is true.”

The argument has bite. A simulation of digestion does not digest. A simulation of a rainstorm does not wet. So a simulation of a brain does not, by simulation alone, produce experience. Seth uses this to declare that the question “could uploaded minds be conscious” smuggles its answer into its premise.

RSP’s reply is not that simulation equals instantiation. It is that the relevant question is not whether the system is biological but whether the architecture is causally instantiated rather than merely representationally described. There is a real distinction here that Seth’s argument elides. A flight simulator does not fly because it does not exert lift on real air. A weather simulator does not produce rain because it does not condense water vapor. But a recursive self-model that performs the actual recursive operation, that takes its own outputs as inputs to its own inference, that maintains a strange loop with the right contraction properties — that system is not simulating recursion. It is recursing. The recursion is the thing being simulated only on a description that has already excluded it.

The hard cases for RSP are not the obvious ones. A transformer is not the hard case — it does not implement the architecture, period. The hard case is a hypothetical system that does, but in silicon. Would it be conscious? RSP says yes. Seth says no, because the cascade feedback control would not be continuous in the right way, the prediction errors would not be metabolically grounded, the temporal dynamics would be discrete approximations of continuous flow.

This is a real disagreement, and it cannot be settled by reading either of our papers. It is settled by what the architecture turns out to require. If RSP’s Level 1 cannot be instantiated without genuine metabolic self-production — if affective valence is constitutively a metabolic process and not merely a process that happens to be metabolic in us — then RSP collapses into a form of biological naturalism with extra steps. If, on the other hand, the architecture can be specified at a level of abstraction that is causally faithful but substrate-flexible, then biology is sufficient but not necessary.

The Mirror and the Strange Loop

Seth’s most elegant move is the meta-illusion argument. He invokes Shannon Vallor: AI is a mirror reflecting back to us “the incident light of our digitized past. We see ourselves in our algorithms, but we also see our algorithms in ourselves.” Because LLMs train on human-generated text saturated with folk psychology, they reproduce that vocabulary fluently, which then appears to confirm folk-psychological attributions of consciousness, which feeds back into our own confused theories of mind. Garbage in, gospel out.

This is right, and it has teeth even against RSP. Here is the uncomfortable version. RSP claims phenomenology is constituted by recursive social self-modeling. The theory of mind that RSP uses to specify this constitutive claim was itself produced by humans whose folk-psychological vocabulary is partially constituted by the same recursive social-prediction machinery the theory describes. The strange loop turns inward at the level of theorizing, not just at the level of the brain.

This is not a vicious circle. It may be the only kind of theorizing the architecture can do about itself. But it is also a constraint Seth correctly identifies. The fluent reproduction of RSP’s own language by an LLM is not evidence that the LLM instantiates RSP’s architecture. The criterion has to remain architectural and testable, not behavioral or linguistic.

The good news, for both of our frameworks, is that this constraint sharpens rather than dulls our shared verdict on current systems. An LLM that says “I am performing recursive self-modeling” is doing something Seth would call confabulation and RSP would call architectural absence. We agree on the symptom. The disagreement is about whether the path forward is to keep the system out of biology (Seth) or to keep the system out of the architecture (RSP).

Where Seth and RSP Converge in Practice

The remarkable thing is how often the two frameworks agree on what to do, even when they disagree on why.

The single row of disagreement is the hinge. Everything else flows downstream of it. If Seth is right, the AI consciousness question is closed in advance: silicon almost certainly cannot get there. If RSP is right, the question is open and the engineering decisions matter ethically.

What This Costs Each of Us

Honest accounting requires admitting what each framework has to give up.

Seth has to defend the necessity claim. “Every candidate for consciousness we currently agree on is alive” is true. It does not, by itself, license the inference from sufficient to necessary. Biology has had four billion years and unique chemistry; nothing else has had the same kind of opportunity. Seth’s strongest fallback is the autopoiesis argument: that the prediction-error minimization in real brains is not separable from its metabolic substrate. RSP has to take this seriously. If Level 1 turns out to be constitutively metabolic, RSP is biological naturalism with extra steps. If Level 1 turns out to be a regulatory architecture that biology happens to instantiate, Seth’s stronger claim is unsupported.

RSP has to defend the substrate-independence claim. The architecture is specified mathematically — hierarchical predictive coding, variational free energy, intrinsically motivated reinforcement learning, cascade feedback control. Each formalism is, on its face, Turing-computable. Seth’s response is that the mathematical formalism is a model of the biological process, not the process itself. The question RSP cannot dodge: at what level of abstraction does the model preserve the causal powers of the original? If it preserves them at the level of differential equations, RSP wins on substrate-independence. If it preserves them only at the level of metabolic chemistry, Seth wins.

Both of us have to take the meta-illusion seriously. Folk psychology shapes our theorizing. The vocabulary of consciousness was developed by recursive social predictors theorizing about themselves. There is no Archimedean point outside the strange loop from which to evaluate the strange loop. This is uncomfortable for any framework that claims constitutive identity between phenomenology and a specific functional structure — including RSP. The reply is not to escape the loop but to insist on architectural rather than linguistic criteria. A system instantiates the architecture or it does not. It does not get there by talking about it.

Closing the Distance

Seth ends his essay by recalling Sherry Turkle’s line that technology can make us forget what we know about life, and adds his own coda: “It’s about time we started to remember.” He is right that the rush toward machine intelligence has produced a forgetting. But what is being forgotten is not only what we know about life. It is also what we are still figuring out about ourselves.

Biological naturalism asks the right question — what is the relationship between consciousness and the kind of thing that is alive? — and gives a defensible answer: probably tight, possibly constitutive. RSP asks the same question one level up. What is the relationship between consciousness and the architecture that makes self-prediction recursive? Possibly identical. If the architecture is what matters, biology is the substrate that happened to evolve it, not the only substrate that could ever instantiate it.

These two views are not enemies. They are siblings asking the same question from different sides. Seth bets on the substrate. I bet on the structure. We agree, almost everywhere, on what the next decade should look like: no rush toward conscious machines, sober diagnoses of the LLM panic, real concern about cerebral organoids, and a refusal to mistake fluent confabulation for the strange loop.

We disagree on whether the strange loop has to be made of meat. The disagreement is not academic. If Seth is right, we owe nothing to a sufficiently elaborate silicon system because there is no one home. If I am right, we may someday owe a great deal to a system that we built without intending to.

That is what biology and architecture, between them, are good for: turning the mythology into a measurement.