Sums and stories: when to ask AI for the answer and when to ask it for the program

If you are going to wash the car, you have to take the car. It is the only possible answer, and a high-end model missed it. Crespo put it on screen at the session in Madrid on 2 June 2026 and rounded it off in two sentences. "ChatGPT does something similar to reasoning […], but it isn't our kind of reasoning. […] The way I sum it up is that it has no common sense."

There are two expensive ways to react to that failure: trust it anyway, or keep AI away from anything with a number in it. Crespo argued for a third, which is to ask yourself first what kind of problem you have on your hands.

Where the machine came from and what it learned

GPT-2 came out in 2019, built from three ingredients: mathematical theory from the 1950s, all the text available on the internet, and chips from video game consoles. It is ChatGPT's grandfather, and it was so frightening that its makers decided at first not to release it. "What they made was a text predictor, and it got away from them," as Crespo put it.

Translating, understanding a text, handling grammar, stringing together a degree of logic: all of it emerged on its own, with no specific training for any of those tasks. "It didn't learn to reason, it didn't learn to tell the truth, it only learned to predict the next word." Judgement comes later, in what is called post-training, through millions of human validations on pairs made up of a question and its correct answer. What we call truth is defined, in practice, by whoever pays for the training.

What you type is chopped into tokens, each token becomes a vector with thousands of dimensions, and that is where the words end. "It's all mathematics, it's all matrix operations. Do you want to call that reasoning, the way we mean it? That's up to you."

Why the architecture guarantees hallucination

Every few months a headline announces the end of hallucination in large language models (LLMs). Crespo put one of them on screen and answered the question it raised. "Is it possible to put an end to LLM hallucination? No. We can't put an end to hallucinations 100%."

The reason is the design itself. "An LLM is designed to talk like a human. It has no concept of truth and lies. To lie or to tell the truth you have to have both concepts and choose, voluntarily, one or the other." Hence the choice of verb. "Because it doesn't know what lying is, people say it hallucinates, they don't even say it lies. And that is part of its architecture."

The paper On the Fundamental Limits of LLMs at Scale, published in November 2025, puts hallucination first among five theoretical ceilings on scaling. However long the list of models you end up building, there will always be inputs on which one of them fails. Questions that no machine can decide produce an infinite number of failures. What remains is a residual error that cannot be removed, whatever the size of the model and however much data you throw at it. Expecting reliability from the model itself is expecting a theorem to be repealed.

The costly mistake is using a probabilistic machine where a number cannot change

That confusion already has figures attached. By IDC's count, 88% of proofs of concept do not make it to wide deployment. What decides that outcome lies outside the model, in the process around it and in the task you give it.

In May 2026 one of the Big Four withdrew a report on fraud in loyalty programmes after an outside analysis found that 16 of its 27 cited sources were invented, misattributed or pointed to pages that did not exist. Checking whether a citation exists is deterministic.

The case Crespo put on screen dates from late April 2026. An agent, meaning a model left to act on its own, deleted the production database of a software supplier to vehicle rental companies, together with the backups kept on the same volume, and took three months of bookings and payments with it. Published accounts describe an agent that hit a credentials error and dealt with it on its own initiative, with a single delete call. Crespo read it as a contradiction in the instructions. On either reading, whether to delete a volume or keep it is a deterministic question, and it was settled by a system that picks the most plausible continuation.

Imagine you ask a model to optimise your hard drive and it wipes it. "Bloody hell, it's optimised. […] Did it work badly or did it work well? It worked well. The fault is ours, for not explaining it properly." Hence the rule he sets before all the others. "If you yourselves can't follow what you've written as an instruction, don't expect ChatGPT to do it."

The model is good at writing the program that does the sums

His summary splits the work in two. "An LLM is excellent at doing things like summarising, writing up subjects that are probabilistic. And they are terrible at doing things that are deterministic, where a number cannot change." In the same breath he offered the way out: they are good at "writing computer programs, in Python for example. The great news is that ChatGPT is built to learn languages. So let's teach it Python, it's already been done."

The full rule fits in two lines. "You have a probabilistic problem, use ChatGPT. You have a deterministic problem, ask ChatGPT to write a computer program to solve that problem."

The paper that founded the technique, Gao and colleagues' PAL: Program-aided Language Models (November 2022), does exactly that. The model reads the problem and writes the program, and the Python interpreter runs the calculation. With that division of labour, PAL beat PaLM, a 540-billion-parameter model reasoning step by step, by 15 percentage points of accuracy on GSM8K, the set of maths word problems used as a benchmark. The model stopped doing the sums, and something else took them over.

Code also brings a bonus you were not looking for. You can read it, version it, take it to a committee and run it a thousand times with the same result — four things a chat answer will not give you. You ask for the program to get round a weakness and end up with a tool that stays in the company, and that your auditor can open.

What changes in your week

Classify before you write the first prompt. Summarising a royal decree, drafting a report, translating, sorting emails, pulling the key points out of a proposal: all of that is probabilistic, so go ahead. For settlements, reconciliations, output curves, network charges and anything where a figure cannot move, ask for the program, keep it and use it again next month. And decide task by task. That was his other piece of advice, offered as an alternative to benchmarks, which the newest models detect so that they can give the answer they think is expected of them. "It's much more reliable to think about whether a specific thing is something an AI does better or worse than a worker."

Don't confuse the model with the product, either. By the figures quoted, Claude Code runs to some 500,000 lines of code, and more than 98% of them have nothing to do with artificial intelligence. The wrapper is almost everything. And that wrapper brings the numbers down. "All the filters they surround these systems with are bringing error rates, hallucination rates, down enormously. In some areas they are lower than a human being's." The theory points the same way: the work that sits alongside that limit concludes that hallucinations, although inevitable, can be made statistically negligible with enough good data. So a permanent limitation becomes an engineering specification, and that is work the energy sector knows how to do.

Crespo closed by quoting "the great poet and philosopher Homer", the one who talks to his son Bart. "There are only three kinds of people […], those who can count and those who can't." His own list really does have three: those who use AI well, those who use it badly and those who don't use it. He dismissed the second group in one line. "Those who use it badly are going to disappear." The third got a short reprieve. "If you don't move, you don't make mistakes, but you get left behind."

What separates one group from the next is the two seconds you spend classifying before typing the first word: the cheapest part of the process, and the one that decides the result. "The difference is between the person who uses AI and the person who knows how to use AI."

ATA Insights. Talk by Manuel Crespo, founding partner of Nodos. RENMAD UsefulAI 2026, Madrid, 2 June 2026.