AI and the Illusion of Understanding

There’s something I’ve been noticing more and more in the way I use AI.

In the past, if you wanted to understand an unfamiliar industry, profession, or group of people, you had to go inside: find people, ask stupid questions, listen to stories, argue, make mistakes, and gradually build your own picture of the world.

Now you can just ask AI.

For example:

“Why do people buy $100,000 watches?”

A few seconds later, you’ll get a perfectly convincing model: status, scarcity, craftsmanship, brand history, investment value, belonging to a certain circle.

And the answer might actually be very good.

But there’s something strange about this.

You get a good explanation without going through the process by which your own understanding usually develops.

Someone who bought a $100,000 watch might have done it because their father used to wear that exact watch.

Or because they spent ten years looking for that particular piece.

Or because they saw it when they were 19, when they couldn’t afford it, and finally bought it after their first big success.

AI can explain very well why people, on average, buy expensive watches.

But that doesn’t mean you understand what that object means to one particular person.

I recognize something similar from my own experience.

Early in my career, it was sometimes more useful to spend several hours sitting with a debugger, trying to understand why my code didn’t work, than to get the correct solution immediately.

While you were looking for one bug, you would discover ten other things: how the system was structured, where the boundaries between components were, why one part depended on another.

If someone had simply given me the correct code, I would have solved the problem.

But by making my own mistakes, I started to understand how the whole thing worked.

And I think something similar is happening with AI.

You can ask it how entrepreneurs think, what matters to architects, why people collect art, or how a particular profession works — and within a minute, get a pretty good model of the world.

That’s an enormous strength of AI.

But it comes with a new danger:

AI can create the feeling that you’ve understood something too quickly.

And the problem isn’t only that it does part of the work for us.

It can take away the signals we normally use to realize that we still don’t understand something.

When you spend several hours debugging code and still can’t find the bug, you run directly into the boundary of your own knowledge:

I don’t understand this yet.

When AI gives you a smooth, convincing explanation in five seconds, that boundary may never appear.

You got an answer — and the question feels closed.

Even though the investigation has barely begun.

Because understanding often doesn’t come from being told the right answer.

It comes from forming a hypothesis yourself, getting it wrong, testing it, running into a contradiction, and changing your model of the world.

So perhaps the best way to use AI is not to ask for the answer immediately.

First, formulate your own hypothesis. Then use AI to test it, find its weak points, and argue against your own thinking.

AI has made getting a ready-made answer almost free.

But it can’t automatically turn that answer into your own experience.

And perhaps one of the things we’ll now have to learn is to distinguish between:

“I know what people say about this.”

and

“I understand what’s actually happening here.”

The first one AI has made almost free.

The second still requires effort.

And sometimes the path through mistakes isn’t just the price of knowledge.

It’s how knowledge is formed in the first place.