AI companies say their models are the best. Here's how to tell if that's true
Benchmarks have become the industry's favorite way to compare AI systems, but public scores can be misleading. Experts explain what the tests actually measure, why they can fail and how users can evaluate models for themselves.
It has become a familiar routine: every few weeks, an AI company releases a new model and declares it the best in the world. Such announcements are usually accompanied by a table filled with obscure test names, showing how the latest model outperforms its competitors across a range of measures. These tests, known as benchmarks, have become one of the main ways to assess the capabilities of AI models. But how well do they reflect the way these systems actually perform in the real world?
To answer that question, it helps to understand what benchmarks are designed to measure. A benchmark is a collection of tasks intended to assess a particular capability. Some test knowledge in fields such as mathematics or biology through multiple-choice questions. Others require models to complete practical tasks, such as identifying and fixing a software bug. Giving different models the same test makes it possible to compare their performance and track improvements between versions.
“A benchmark has a purpose. It aims to measure something very specific; each benchmark tests something different,” explains Omer Nevo, CTO of Irregular, a company that develops tests for some of the world's leading AI firms. “Specifically, we create benchmarks for ‘dangerous capabilities.’ In other words, the better a model performs on our benchmarks, the greater the potential harm it could theoretically cause. In a sense, it is actually better not to succeed on these benchmarks.”
When a model performs particularly well on such tests, Nevo says, its release may need to be delayed or initially restricted to a limited group of trusted users, as OpenAI and Anthropic have done.
For companies developing AI models, benchmarks offer a way to measure progress. They can show which capabilities have improved in a new version and where the model continues to struggle. “A model with significantly stronger benchmark results tends to be an improved model,” Nevo says. But he cautions that “there is no one-to-one correlation between benchmark performance and real-world usage.”
One of the main challenges in evaluating models is ensuring that they have not already encountered the test questions and learned the answers. AI models are trained on vast amounts of data, including material collected from the internet. If benchmark questions, answers or explanations are publicly available, they may become part of a model's training data.
“In effect, the model learns the test rather than acquiring the actual knowledge,” Nevo explains. “Consequently, the benchmark might make the model appear exceptional, succeeding at everything, yet it would likely perform far less effectively on similar, alternative questions.”
One way to limit this problem is to keep tests confidential. Nevo says a growing number of organizations now conduct their own evaluations without publicly disclosing the full set of questions and tasks. This makes it less likely that the answers will find their way into training data, but introduces another problem: researchers and the public have fewer opportunities to independently verify what is being measured or how the scores are calculated. Those reviewing the results are left to rely on the organization conducting the evaluation.
Even when a model has not been exposed to the answers, a benchmark can lose its usefulness as models improve. Once they become capable of answering nearly every question in a test, the benchmark no longer offers much insight into the differences between them. This makes the development of new tests an ongoing necessity, Nevo explains. They must be difficult enough to distinguish between models, but not so difficult that all of them fail. If every model performs poorly, the results offer little insight into their relative capabilities.
How do you choose a model?
When an AI company publishes a scorecard for a new model, it is worth looking beyond the headline results. The important questions are which capabilities have improved and whether those improvements matter for the tasks you actually need the model to perform. A higher score on programming benchmarks, for example, may be significant for someone using AI to develop software, but says little about whether the model will produce better writing.
Nevo recommends looking for improvements in areas that correspond to your intended use. This can help you decide whether a new model is worth trying, rather than assuming that a higher overall ranking makes it the right choice.
The number of benchmarks a company chooses to publish is also worth considering. Nevo says some companies may avoid disclosing results for tests on which their models perform poorly. “When a company publishes very few benchmarks, it’s a pretty good sign that things aren't going exceptionally well for them,” he says. A long list of impressive scores may be useful, but so is knowing how much of the overall picture the company is willing to show.
Still, benchmark results alone cannot tell you which model will work best for your needs. “Seeing a model perform well on benchmarks is a good sign,” Nevo says, “but it is far less reliable than actually using and testing it yourself.” Benchmarks can indicate which capabilities are worth exploring. Only hands-on experience can show whether those capabilities translate into good results on your own tasks.
There are also practical considerations beyond performance, including ease of use, price and usage limits. Tsoof Bar-Or, an AI content creator and CTO at Super8, compares choosing a model to choosing a car. “If I need to travel long distances, I’m not going to buy a race car,” he says. “What performs best on a benchmark isn't necessarily what’s best for me.”
Bar-Or also recommends checking latency, or how long a model takes to respond, as well as the limits placed on its use. “I don’t just want the most powerful model in the world; I also need to be able to use it consistently within my budget,” he says.
Not every task requires the most powerful model available. Bar-Or suggests trying a simpler model first to see whether it can deliver equally good results. If it can, users can save their usage allowance on more powerful models for tasks that genuinely require them.
To get a sense of how a model performs in everyday use, Bar-Or also recommends following discussions in user communities, including on X and in Facebook groups. “Since these models evolve so rapidly, there’s no substitute for getting a sense of the community’s ‘vibe,’” he says. But online enthusiasm should not replace personal testing. “The reason for doing something should be, ‘I tested it and it works for me,’ rather than, ‘I saw an influencer online say this is the better way to go.’”
So how should users conduct their own tests? Nevo suggests starting with work they have already given to an AI model. Take several previous tasks and run them again using the model you are considering switching to. “I would choose something that aligns as closely as possible with my actual use case, ideally, the actual use case itself,” he says. It is also useful to include tasks on which your current model has previously struggled.
Before comparing the results, however, you need to decide what counts as success. “For example, if I asked it to write code, does the code actually work? If I asked it to answer a question, is the answer correct?” Nevo says.
The criteria will depend on the task. For writing, users can check whether the model follows their instructions and adheres to the requested style. When summarizing a document, they can assess whether important details have been preserved and whether the model has introduced claims that do not appear in the original. This creates a small, personalized benchmark based on the tasks and standards that matter most to the individual, one that can be reused whenever a new model is released.
Even then, it is important not to draw conclusions from a single response. Models can produce different answers to the same prompt, Nevo says, which is why professional evaluations typically run tasks multiple times. For individual users, that means comparing results across several tasks and repeating particularly surprising tests before deciding that one model is better than another.
