For decades, AI's progress has been mostly measured by what it can do: how many questions does it answer? How many images can he interpret? How accurate is it in predicting or diagnosing? But a different question began to impose itself on researchers: Can the system know when it does not have a reliable answer?
The problem with artificial intelligence is not only that it makes mistakes, but that it sometimes gives the wrong answer with confidence that suggests it is correct. In sensitive fields such as medicine, the difference becomes fundamental. A system that can indicate its uncertainty may be safer than a system that gives a definitive answer when the evidence is insufficient.
Therefore, a growing trend of research is trying to develop models that not only produce an answer, but can also estimate confidence and express uncertainty when the question exceeds the limits of what you can reliably know. Here the question changes. Instead of just asking: Can artificial intelligence answer? The most interesting question becomes: Can he know the limits of what he knows... and when should he say “I don’t know”?
From intelligence to “metacognition”
• “Metacognition.” Cognitive scientists call a person's ability to think about his thinking and evaluate the extent of his confidence what is known as "metacognition", or "metacognition". It is what helps us distinguish between what we know with confidence, what we think is likely, and what we do not have sufficient information about.
The importance of this ability is clearly evident in medicine. The expert doctor not only knows the answer to every case, but also recognizes when the evidence is not enough to make a safe decision. He requests additional examination, reconsiders the diagnosis, or consults a colleague before making a decision that may affect the patient's life. Knowing the limits of certainty is an essential part of good clinical judgment.
Today, researchers are trying to build something that functionally resembles this ability into artificial intelligence systems; Not that the machine becomes aware of what it knows, but that it can estimate the reliability of its answer, and express uncertainty when it does not have a sufficient basis for answering with confidence.
And here lies the new idea: the safest generation of artificial intelligence may not be the one that always answers, but also the one that can confidently recognize situations in which it should not answer.
• Teaching the machine to say: “I don’t know.” In a research study published on June 30, 2026 on the scientific research platform “arXiv”, a team led by researcher Gabrielle Kylie May Liu developed a new framework called “Reinforcement Learning with Metacognitive Feedback - RLMF”, with the aim of addressing a fundamental problem in linguistic models: that they appear confident in their answer even when this answer is unreliable.
This approach does not teach the machine “awareness” of its ignorance in the human sense, but rather trains it to better estimate its performance and match the level of confidence it expresses with the degree of uncertainty. Experiments showed that this method improved confidence measures, and outperformed traditional reinforcement learning methods by up to 63 percent in some comparisons, without sacrificing the accuracy of the answers.
The question did not remain confined to one study; On July 13, 2026, a research review entitled Metacognition in LLMs: Foundations, Progress, and Opportunities was published on arXiv, which comprehensively covered what research has reached into the “metacognition” of linguistic models. The researchers concluded that the field has begun to develop methods to measure the ability of models to monitor their performance, estimate uncertainty, and modify their behavior, but a fundamental question is still open: when, how, and to what extent can these models demonstrate effective meta-cognitive behavior? On August 15, 2026, a more recent paper appeared on arXiv that provided an initial proof of concept for a metacognitive framework that causes a set of models to monitor indicators in their performance, such as the probability of an answer being correct, the presence of conflicting information, and the complexity of the problem, and then uses these signals to determine whether a task calls for more deliberate processing. This does not yet represent a solution ready for clinical use, but it reveals the direction of the research: the goal is not only to make the model better able to answer, but also better able to distinguish cases in which its answer is not sufficient or requires more thinking and information.
Here lies the importance of transformation. Instead of always rewarding the system for providing an answer, a measure of AI quality in the future may become its ability to stop when evidence is insufficient, request additional information, or express uncertainty rather than fill the void with a confident answer.
• Trust is as important as accuracy. In medicine, the correct answer alone is not enough; An essential part of the clinical decision is knowing the strength of the evidence on which it is based, and the degree of trust it deserves. An expert doctor may suggest a particular diagnosis, but he does not deal with all possibilities with the same degree of certainty. He may request additional imaging or laboratory analysis, or consult a colleague, when he realizes that the information available is not sufficient to make a safe decision. Here, hesitation is not a weakness in knowledge, but rather part of the soundness of the medical decision.
Artificial intelligence systems face a similar problem, but in a different way. The model may produce a correct answer, and at other times it may produce incorrect or unsupported information, in what is commonly known as a “hallucination.” The most sensitive problem is that the wording of the answer may seem confident and convincing even when it is wrong.
This is why researchers distinguish between answer accuracy and confidence calibration. If the system says it is highly confident, does this confidence actually correspond to the probability that its answer is correct? The wider the gap between confidence and accuracy, the greater the likelihood that the user will trust an answer that does not deserve this trust.
In medicine, this issue becomes more than a technical problem. A secure system should not only be good in the cases it knows about, but also be able to raise a warning signal when a situation falls outside its scope of reliability, or when the available information is insufficient to make a decision.
Human experience is deeper than data
• What data cannot teach. But knowing the limits of the answer opens a deeper question: Can everything a person knows at all be transformed into learnable data and rules?
For centuries, philosophers have been concerned with the nature and limits of knowledge. From Ibn Rushd, who gave reason and proof a central place in his view, to the philosopher and scientist Michael Polanyi, who in the twentieth century formulated the concept of “tacit knowledge,” and summarized his idea with his famous phrase: “We know more than we can say.” Professional experience does not consist of books and explicit information alone. It is also formed from thousands of accumulated situations, and from observing subtle patterns that the expert may be able to catch before he can fully explain the reasons that led him to them.
A child may enter the emergency department, and some of his initial measurements appear to be within reassuring limits, but the expert pediatrician notices something in his appearance, behavior, or way of breathing that makes him not reassured. What we sometimes call “clinical intuition” may not be a vague intuition, but rather the result of years of exposure to situations and patterns that have become part of quick professional judgment.
Here a different challenge arises for artificial intelligence. An algorithm can learn from huge amounts of digitally represented data, but human experience also contains contextual and implicit knowledge that is sometimes difficult to codify, measure, and transform into training data.
Therefore, the question may not just be: How do we teach the machine more of what we know? But also: How do we teach them to deal with what we ourselves could not transform into clear data?
• Does the machine really know the limits of its knowledge? The answer so far: Not in the sense in which a person knows the limits of his knowledge; Recent research indicates that linguistic models can be trained to better estimate the degree of uncertainty in their answers and calibrate the level of confidence in them. But this does not mean that the model has become aware of what it knows and what it does not know, nor that its estimate of uncertainty is similar to a person's awareness of the limits of his knowledge.
Here we must distinguish between two different types of “not knowing”: The first is uncertainty that can be estimated mathematically: such as if the evidence is conflicting, or the probabilities of several answers are close, or the algorithm encounters a situation different from what it was trained to do. This is the area that calibration and meta-synthetic techniques are trying to improve.
The other level is more complex: knowledge that was not included in the data in the first place, or a context that was not represented in it, or implicit experience that is difficult to convert into numbers and words. Here, it is not enough to increase or decrease model confidence; Because the system may not have anything to tell it that something important is missing from the picture.
This is the crucial difference: AI can become better at estimating uncertainty in what it can process, but that does not mean that it has become able to recognize all the limits of knowledge that it has not reached. In simpler terms: a machine may learn to say, “I don’t know,” when it doubts its answer, but the more difficult challenge is to know when it should doubt at all.
• Humans see what algorithms may not see. Future models are likely to become better at estimating and expressing uncertainty, and this development in sensitive fields, most notably medicine, may be as important as improving accuracy itself. A system that knows when to lower its confidence, request additional information, or refer the decision to a human may be safer than a system that seeks to provide an answer every time.
