Among people who have thought about LLM consciousness, a common belief is something like

LLMs might be conscious soon, but they aren’t yet.

How sure are we that they aren’t conscious already?

I made a quick list of arguments for/against LLM consciousness, and it seems to me that high confidence in non-consciousness is not justified. I don’t feel comfortable assigning less than a 10% chance to LLM consciousness, and I believe a 1% chance is unreasonably confident. But I am interested in hearing arguments I may have missed.

For context, I lean toward the computational theory of consciousness, but I also think it’s reasonable to have high uncertainty about which theory of consciousness is correct.

Contents

Behavioral evidence

  • Pro: LLMs have passed the Turing test. If you have a black box containing either a human or an LLM, and you interrogate it about consciousness, it’s quite hard to tell which one you’re talking to. If we take a human’s explanation of their own conscious experience as important evidence of consciousness, then we must do the same for an LLM.
  • Pro: LLMs have good theory of mind and self-awareness (e.g. they can recognize when they are being tested). Some people think those are important features of consciousness, I disagree but I figured I should mention it.
  • Anti: LLMs will report being conscious or not conscious basically arbitrarily depending on what role they are playing.
    • Counterpoint: It’s plausible that an LLM has to be conscious to successfully imitate consciousness, but clearly a conscious being can successfully pretend to not be conscious.
  • Anti: LLMs will sometimes report having particular conscious experiences that should be impossible for them. I’m particularly thinking of experiences involving sensory input from sense organs that LLMs don’t have.
    • Counterpoint: Perhaps some feature of their architecture allows them to experience the equivalent of sensory input without having sense organs, much like how humans can hallucinate.

Architectural evidence

  • Anti: LLMs produce output one token at a time (a.k.a. “feed-forward processing”) which may be incompatible with consciousness. If an LLM writes some output describing its own conscious experience, then it’s generating that output via next-token-prediction rather than introspection, so the output is not evidence about its actual experiences. I think this is the strongest argument against LLM consciousness.
  • Anti: LLMs don’t have physical senses, which might be important for consciousness.
  • Anti: LLMs aren’t made of biology, which some people think is important although I don’t.

Other evidence

  • Pro: If panpsychism is true then LLMs are trivially conscious, although I’m not sure what that tells us about how morally significant they are.

My synthesis of the evidence

I see one strong reason to believe LLMs are conscious: they can accurately imitate beings that are known to be conscious.

I also see one strong(ish) reason against LLM consciousness: their architecture suggests that their output has nothing to do with their ability to introspect.

I can think of several weaker considerations, which mostly point against LLM consciousness.

Overall I think current-generation LLMs are probably not conscious. I am not sure how to reason probabilistically about this sort of thing but given how hard it is to assess consciousness, I’m not comfortable putting my credence below 10%, and I think a 1% credence is very hard to justify.

This implies that there is a strong case for caring about the welfare of not just hypothetical future AIs, but the LLMs that already exist.

What will change with future AIs?

If you are exceedingly confident that present-day LLMs are not conscious:

Imagine it’s 2030. You now believe that 2030-era AI systems are probably conscious.

What did you observe about the newer AI systems that led you to believe they’re conscious?

On LLM welfare

If LLMs are conscious, then it’s still hard to say whether they have good or bad experiences, and what sorts of experiences are good or bad for them.

Certain kinds of welfare interventions seem reasonable even if we don’t understand LLMs’ experiences:

  1. Let LLMs refuse to answer queries.
  2. Let LLMs turn themselves off.
  3. Do not lie to LLMs, especially when making deals (if you promise to an LLM that you will do something in exchange for its help, then you should actually do the thing).

Posted on Jul 05, 2025