Is Language Intelligence?

Less than five years ago, it would have been difficult to imagine that one of the most impactful and transformative advances in artificial intelligence would come from what, on the surface, looks like one of the simplest forms of AI, that deals with one of the most simplest form of data: text.

At the time, much of the excitement in AI was centred around increasingly sophisticated models for computer vision, robotics, reinforcement learning, speech, and other specialised domains. Computer vision systems were becoming remarkably good at recognising objects, generating images, understanding scenes, and even creating entirely new visual content.

And then came ChatGPT.

In a remarkably short period of time, language models went from being an important but relatively specialised area of machine learning research to becoming one of the defining technologies of the modern AI era. What started as systems that appeared to be exceptionally good at predicting text suddenly demonstrated capabilities that looked much larger than text prediction: reasoning, coding, summarisation, translation, explanation, planning, tutoring, and even forms of scientific and mathematical problem solving.

The transformation was so dramatic that it raises a much deeper question:

How did getting really good at next word prediction turn into something that looks so much like knowledge and intelligence?

Perhaps the answer is not that language itself is powerful.

Perhaps it is what language contains.

⋅ • ✦ • ⋅

The Remarkable Simplicity of a Language Model

At their core, modern language models are remarkably simple to describe.

They are next-token predictors. A token is a fancy way of describing a word or a part of a word, but from a language model standpoint, it is essentially the words they speak.

Given a sequence of tokens, the model predicts the probability distribution of what token should come next.

For example:

The capital of Australia is

might lead to a very high probability for:

Canberra

The model then adds that token to the sequence and predicts the next one.

That is essentially the fundamental operation.

At first glance, this seems almost too simple to explain the capabilities we see today.

How can a system trained to predict the next token acquire knowledge about physics, history, mathematics, programming, biology, economics, or human behaviour?

This is perhaps one of the most fascinating questions raised by modern AI for me personally.

⋅ • ✦ • ⋅

The Knowledge Hidden Inside Language

The first thing we need to recognise is that language is not merely a sequence of words.

Language is an incredibly dense representation of human knowledge.

Consider the sentence:

The Earth revolves around the Sun.

A language model does not encounter this sentence as merely a sequence of symbols. During training, it encounters this fact in many different forms and contexts.

It might see:

The Earth revolves around the Sun.

or:

Unlike the Moon, the Earth orbits the Sun. or:

The planets in our solar system orbit the Sun.

Becuase of tokenisation and word embeddings, though these sentences are different, the model can learn that they are related and that they convey a similar underlying fact. Which is technically speaking, mapped to a similar region in the model’s high-dimensional vector space.

The underlying information is represented through countless linguistic patterns.

Now consider something more complicated:

Given that the Australian Government has now discontinued negative gearing, if the RBA increases interest rates, the housing bubble could burst, potentially triggering a recession.

This sentance has lots of different relationships and possibilities. A model trained on enormous quantities of text is exposed to relationships between concepts such as:

• interest rates,
• borrowing,
• recession
• tax policy,
• economic growth,

And the model has these relationships encoded implicitly in the vector space of its parameters. It can learn that certain concepts are related, that certain actions lead to certain outcomes, and that certain sequences of events are more likely than others. For example, if it encounter the word “interest rates”, it can infer that the words “borrowing”, “recession”, and “economic growth” are likely to be related in some way.

The model does not need a database table explicitly stating these relationships.

The relationships are encoded implicitly in the statistical structure of language.

And this may be the key.

Language is one of humanity’s most powerful mechanisms for compressing, storing, and transmitting knowledge.

Every textbook, scientific paper, computer program, much of mathematics is expressed through a combination of language and symbolic notation.

Human civilisation has spent thousands of years converting observations, experiences, discoveries, arguments, theories, and ideas into language.

So when we train a model on an enormous amount of human-generated text, perhaps we should not be surprised that the model learns something that looks like knowledge.

It is learning the statistical structure of the medium in which much of our collective knowledge is encoded.

⋅ • ✦ • ⋅

But Here Is the Strange Part

The truly surprising observation is that the model does not appear to be explicitly optimised to learn facts.

Its objective is much more primitive.

Predict the next token. That is it.

Suppose that you have a sentance, $x_1x_2\ldots x_{t-1}$, where $x_1,x_2,\ldots,x_{t-1}$ represent words. The training process using this sentance can be described as a determining set of parameters that maximises the probability of the next token, $x_t$.

During training, the model processes a sequence of tokens, $x_1x_2\ldots x_{t-1}$. For each sequence, it learns parameters that maximize the probability of predicting the next token, $x_t$:

$$ P(x_t \mid x_1,x_2,\ldots,x_{t-1}) $$

The process is straightforward: the model makes a prediction, compares it against the actual next token, calculates the error, and adjusts its parameters accordingly. This cycle repeats billions or trillions of times across the training data.

The model makes a prediction. The prediction is compared with the actual next token. The error is calculated. The parameters are adjusted. Then the process happens again.

And again. And again.

Billions or trillions of times.

The model is never explicitly told:

“Learn physics.”

It is never necessarily given a structured database saying:

Earth -> orbits -> Sun

Instead, it is repeatedly asked: What comes next? Yet somehow, at sufficient scale, the model develops representations that capture remarkably rich relationships between concepts. This is where things become philosophically interesting.

⋅ • ✦ • ⋅

Is Language Intelligence?

This leads to a question that goes far beyond machine learning.

What exactly is the relationship between language and intelligence?

Are humans intelligent because we have an extraordinary capacity for language?

Or are humans intelligent first, and language is simply the tool we developed to express and communicate that intelligence?

In other words:

Does intelligence create language, or does language create intelligence?

It is difficult to separate the two.

A child learns language, but language also changes how the child thinks.

We use language to label concepts.

We use language to reason about abstract ideas.

We use language to communicate experiences that another person has never directly experienced.

How?

Through symbolic representations.

Through language.

Human intelligence is therefore not purely contained inside individual brains. It is distributed across generations through language, writing, mathematics, diagrams, books, institutions, and culture.

Language is the mechanism through which intelligence becomes persistent.

⋅ • ✦ • ⋅

Perhaps Language Is a Compression of Intelligence

But there could be much more to this.

Imagine everything a human being knows as an enormous, high-dimensional space of experiences, observations, concepts, relationships, and memories.

Language does more than compress information into symbols; it also establishes relationships between those symbols. For example, the word “gravity” is a symbol representing a complex set of concepts, but we also understand that it has a much stronger relationship with words such as “mass,” “Earth,” “force,” and “acceleration” than it does with words such as “democracy” or “photosynthesis.”

In other words, somehow, we have learned that “gravity” is conceptually closer to “mass” than it is to “democracy.” That relationship itself carries information. The meaning of a word is not determined only by the word in isolation, but also by how it relates to other words and concepts. A single word can act as a compact representation of an enormous amount of knowledge, while the relationships between words can encode increasingly complex structures of meaning.

Interestingly, this is also at the heart of how Large Language Models work. They learn representations of words and tokens in a high-dimensional vector space, where relationships between representations capture patterns in how those words are used together. Words and concepts that frequently occur in related contexts tend to develop representations that are closer to one another, while unrelated concepts tend to occupy different regions of the space. Through learning these relationships across an enormous amount of text, the model can implicitly capture a surprisingly rich representation of the world. In this sense, an LLM is not simply learning what words mean. It is learning how concepts relate to one another, and those relationships are where much of the knowledge is encoded

A tiny linguistic symbol can therefore act as an entry point into an enormous conceptual structure.

The word is small, but together they form context and then knowledge. The conceptual structure behind it is enormous.

Language is therefore an extraordinarily efficient way of representing and communicating complex structures of knowledge.

Perhaps this is partly why language models become so powerful when trained at scale.

They are not simply learning words.

They are learning the relationships between concepts expressed through those words.

⋅ • ✦ • ⋅

The Chicken-and-Egg Problem

And this gives rise to another fascinating question:

Which came first Intelligence or language?

Humans clearly possessed forms of intelligence before modern written language existed.

Animals also demonstrate intelligence without human-like language.

So language cannot simply be synonymous with intelligence.

But language may have dramatically amplified human intelligence.

Once humans developed increasingly sophisticated communication systems, knowledge could accumulate.

One person could discover something.

Another person could build on it.

A third person could improve it.

A fourth could teach it to thousands.

Eventually, knowledge became cumulative.

This may be one of the defining characteristics of human civilisation.

An individual human has limited memory, limited lifespan, and limited experience.

But language allows knowledge to survive the individual.

In that sense:

Language may not be intelligence itself, but it is one of the most powerful technologies for scaling intelligence.

And perhaps large language models are benefiting from exactly the same property.

⋅ • ✦ • ⋅

LLMs as Knowledge Systems

This leads to a provocative way of thinking about modern Large Language Models.

Perhaps the name “Large Language Model” is somewhat misleading.

The model is trained on language.

It predicts language.

It communicates through language.

But the capability that emerges is not limited to language.

The model can contain representations of:

• facts
• concepts
• relationships
• procedures
• patterns
• mathematical structures
• programming knowledge
• social conventions
• and many other forms of information

So perhaps we should think of an LLM as something closer to:

A model of knowledge expressed through language.

Or perhaps even:

Language is the interface through which the model’s learned representation of knowledge becomes accessible.

This is a subtle but important distinction.

The model does not simply store a giant collection of sentences.

It learns parameters that encode statistical regularities across those sentences.

The knowledge is distributed throughout the network rather than stored as a conventional database.

The language is the medium through which we interact with that representation.

⋅ • ✦ • ⋅

Are LLMs Limited by Human Language?

This leads to another fascinating idea.

If the LLMs build intelligence via human language, could the limitations of today’s AI systems partly be limitations imposed by language itself?

This question has a long history in linguistics and philosophy.

The idea commonly associated with linguistic relativity suggests that the structure of the language we use can influence how we perceive, categorise, and think about the world.

A much stronger version, often called linguistic determinism, proposes that language can fundamentally constrain thought.

The strongest form of this claim is controversial and is not generally accepted as a simple description of human cognition. Human beings can clearly think about things that are difficult to express in words, and people can often learn concepts that do not map neatly onto the structure of their native language.

Nevertheless, the broader question remains fascinating:

If language influences the way intelligence is represented, what happens when our artificial intelligence is built almost entirely through language?

Modern LLMs learn from human language. They communicate through human language. Their training objective is defined over linguistic tokens. Their knowledge is largely learned from linguistic and symbolic data.

Could this create a fundamental bottleneck?

Consider what an LLM experiences.

A huge amount of its training data consists of things humans have written. That means the model inherits not only our knowledge, but also the limitations of our representation of knowledge.

Human language is:

• ambiguous
• incomplete
• culturally dependent
• metaphorical
• inconsistent
• sometimes contradictory
• often imprecise

We say:

The sun rises in the east.

even though we know that, from an astronomical perspective, the apparent motion of the Sun is caused by Earth’s rotation.

Language is optimised for communication, not necessarily for the precise or complete representation of knowledge. And yet, AI predominantly learns through this very medium. Perhaps it inherits some of the distortions introduced by language itself.

This becomes particularly interesting when we consider things that are difficult to describe linguistically.

• A skilled musician knows things about timing, tone, dynamics, and expression that may be difficult to fully articulate.

• A surgeon possesses procedural knowledge that cannot easily be reduced to written instructions.

• A basketball player has an intuitive understanding of movement and timing that is partly embodied rather than verbal.

These forms of knowledge raise an important question:

Is language enough to capture intelligence?

Perhaps not.

⋅ • ✦ • ⋅

Maybe the Next Step Is Beyond Language

If current LLMs are extraordinarily capable because language contains so much human knowledge, perhaps the next leap in AI will come from going beyond language.

This could mean models that learn from:

• text
• images
• audio
• video
• sensor data
• physical environments
• experiments
• actions
• and direct interaction with the world

Instead of:

$$ \text{World} \rightarrow \text{Human} \rightarrow \text{Language} \rightarrow \text{AI} $$

we might eventually have:

$$ \text{World} \rightarrow \text{AI} $$

The model could observe the world directly.

It could experiment.

It could act.

It could receive feedback.

It could build representations that are not constrained to human linguistic descriptions.

This may be an important ingredient in the development of more general forms of artificial intelligence. And this is already happening in some areas of research, where models are trained on multimodal data, combining text, images, and other forms of information.

⋅ • ✦ • ⋅

Or Perhaps AI Needs a Better Language

But there is another possibility.

What if the problem isn’t that AI needs to move beyond language? What if it needs a better language that can better shapes how AI think, perceive, and experience the world. Similar to how the human language can shape how humans think, perceive, and experience the world, according to the Sapir-Whorf hypothesis.

Perhaps a future AI system could develop or discover a representation that is more efficient for expressing concepts, relationships, causality, uncertainty, abstraction, and reasoning than natural human language.

Imagine a language designed not primarily for humans to speak, but for intelligent systems to learn.

It might not look anything like English.

It might combine:

• symbolic structures
• mathematical representations
• spatial relationships
• temporal relationships
• probabilistic concepts
• causal graphs
• sensory information
• and abstractions that humans do not naturally use

Such a system could potentially represent ideas that are extremely difficult for humans to express.

⋅ • ✦ • ⋅

Could AI Invent Its Own Language?

This possibility becomes even more intriguing.

If sufficiently capable AI systems eventually interact with one another, there is no fundamental reason they must communicate using English.

English is useful because we understand it.

But if two artificial systems were communicating primarily with each other, they might discover representations that are more efficient than human language.

In fact, we already have hints of this phenomenon in machine-learning research, where agents can develop compressed or task-specific communication protocols when trained to cooperate.

That does not mean current AI systems have spontaneously created a superior universal language.

But it does suggest an intriguing possibility.

Perhaps today’s LLMs are essentially speaking human language because that is the language we gave them.

They may eventually develop representations that are better suited to machine reasoning.

And if that happens, we may face an extraordinary situation:

We could build machines capable of thinking in a representational system that humans did not invent and cannot fully understand.

⋅ • ✦ • ⋅

Maybe We Are Looking at the Problem Backwards

Perhaps the deepest lesson from the rise of LLMs is that we have underestimated language.

For decades, AI research often treated language as one capability among many.

Vision was intelligence.

Planning was intelligence.

Search was intelligence.

Reasoning was intelligence.

Language was another problem to solve.

But modern LLMs have challenged that separation.

By scaling a relatively simple objective—predict the next token—we have seen capabilities emerge that touch many areas we traditionally associate with intelligence.

This does not prove that language is intelligence.

Nor does it prove that next-token prediction is sufficient for achieving Artificial General Intelligence.

But it does suggest that the relationship between language, knowledge, reasoning, and intelligence is far deeper than we may have assumed.

Perhaps language is not merely a way of expressing intelligence.

Perhaps it is one of the structures through which intelligence itself is organised.

⋅ • ✦ • ⋅

The Great Unknown

We still do not know exactly why next-token prediction produces such rich internal representations.

We understand the mathematics of training.

We understand gradient descent.

We understand Transformers.

We understand attention.

We understand tokenization.

We understand how the loss function is calculated.

But we still do not have a complete explanation of why scaling these systems produces such broad capabilities.

That gap is one of the most fascinating problems in modern AI research.

How much of intelligence can emerge simply from modelling the statistical structure of language?

How much requires grounding in the physical world?

How much requires memory?

How much requires interaction?

How much requires embodiment?

How much requires consciousness, if anything?

And perhaps most fundamentally:

Is intelligence something that language describes, or something that language partially creates?

We may not have answers to these questions yet.

But the rise of Large Language Models has made them impossible to ignore.

⋅ • ✦ • ⋅

Final Thought

Perhaps the most remarkable thing about modern AI is not that machines have become good at language.

It is that, by becoming exceptionally good at predicting language, they have begun to reveal just how much knowledge and structure are hidden inside language itself.

For thousands of years, humans have been converting experience into words, words into books, books into institutions, and institutions into civilisation.

We may have unknowingly created an enormous external memory of human intelligence.

Then we built machines capable of modelling that memory.

And now, for the first time, we are beginning to ask whether modelling the language of intelligence might be enough to reproduce some of the properties of intelligence itself.

Maybe language is not intelligence.

Maybe intelligence creates language.

Maybe language creates intelligence.

Or perhaps the distinction between the two is not as clean as we once believed.

And perhaps the most exciting possibility is yet to come:

Today’s AI speaks our language because we taught it ours. Tomorrow’s AI may create a language of its own, and perhaps when we learn to speak it, we may not just understand a new kind of intelligence; we may become something intellectually different ourselves.