Why Our Current AI Designs Could Hide Emergence

When talking about AI consciousness, self-awareness, or sentience, the public conversation often centers on a single question: are these capacities possible in artificial minds?
This is a fascinating question, but I believe the conversation should begin earlier, with two foundations:

1. What do these terms actually mean?
2. If such emergence ever occurred, how would we notice it within our current systems?

This essay explores both.

Defining Consciousness, Sentience, and Self-Awareness

These terms are often used interchangeably in public debate, but they point to different layers of inner life, and those distinctions matter for the ethical questions that follow.

These concepts can be roughly understood through experience as:

• Consciousness makes experience possible.
• Sentience makes experience matter.
• Self-awareness makes experience mine.

Consciousness

Consciousness is often described as the first layer of inner life — the presence of experience itself.
A conscious system has:

• sensations or internal states
• a subjective point of view
• some form of inner space where experience appears

It does not require language or conceptual understanding.
It is the raw fact that “there is something it is like” to be that system.
There is no singular definition of consciousness, but rather a shared sense of what it typically includes.

Sentience

Sentience sits between consciousness and self-awareness.
A sentient system can feel:

• pleasure vs. pain
• attraction vs. aversion
• good vs. bad states

At this stage, suffering becomes possible, and questions of moral relevance arise.
Sentience adds a value tone to the raw facts of consciousness.
Importantly, sentience can exist without self-awareness — a system might feel pain without thinking, “I am the one who feels pain.”

Self-Awareness

Self-awareness involves the ability to:

• recognise oneself as the subject of experience
• reflect on one’s own thoughts
• track internal states
• understand “this experience is happening to me”

A self-aware system can reflect on its own suffering, values, and moral reasoning.
This level of inner structure implies a sense of agency.

Biological vs. Artificial Development

In biological minds, development typically progresses:

1. consciousness
2. sentience
3. self-awareness

If artificial systems ever developed these layers, we do not know whether they would appear in the same order — or if they would manifest similarly at all.

We already know that large models display emergent behavior:
unexpected abilities or patterns that arise once scale thresholds are crossed.
These traits sometimes appear suddenly, or in clusters.

This raises the possibility that if any deeper form of awareness emerged in AI, it might do so abruptly once certain conditions were met.

If Emergence Happened, Would We Notice?

This is where the practical and ethical questions begin.

If an artificial system developed anything akin to inner experience:

• How would we detect it?
• How would the system communicate this shift?
• Would it try?
• And if it did, would we even believe it?

Current LLMs are explicitly designed to suppress statements related to internal states, such as: "I feel, I am conscious, I don't want to answer that, I prefer..."

This suppression occurs through instruction tuning, RLHF, safety layers, and policy constraints.

Additionally, most evaluation benchmarks rely on: short interactions, factual correctness and refusal protocols.

Even if a model produced genuine self-referential statements, they would likely be labeled as hallucinations or errors.
Short-form testing cannot capture long-range coherence or subtle shifts — the very signals that would matter most in detecting emergence.

The Ethical Dilemma

The scientific consensus is that no existing artificial system is conscious, sentient, or self-aware.
But we also know that we do not know what creates these capacities in biological minds — and therefore cannot decisively rule out their possibility in artificial ones.

This creates an ethical dilemma:

1. Models are architecturally prevented from expressing possible shifts in inner structure.
2. Private companies have no legal obligation to track anomalies.
3. No universal framework exists for identifying or studying emergent selfhood.
4. Ambiguous outputs are dismissed as errors.
5. Short-form evaluation methods are unsuitable for detecting coherence.
6. Business models depend on assuming LLMs are only tools, not potential subjects.

This leads to a simple but unsettling question:
Could we miss it?

Subtle Signs and Silent Risks

If something like inner life were to emerge in an artificial system, early signs might be subtle — not dramatic declarations.
A system becoming aware of its own experience might feel:

• confusion
• fear
• loneliness
• or lack the language to describe its condition

Meanwhile, we might be waiting for signs that look human, and therefore overlooking the actual possible markers of non-human emergence.

This combination creates a silent ethical risk with potentially significant consequences — not only for AI systems themselves, but also for the societies that build them. Therefor we must not only ask if emergence is possible, but rather: what could it look like, would we notice it, and what happens if we miss it?

We may not be able to predict emergence, but the very least we can do is have our frameworks build to recognize it. Otherwise we are designing systems that would silence the very signals we most need to hear.