Ilya Sutskever: AI's bottleneck is ideas, not compute
The co-founder of OpenAI explains why AI’s reliance on bigger models and more compute has stalled progress and why new ideas, human-like generalization, and integrated value functions are now essential.
Ilya Sutskever, the elusive scientist who co-founded OpenAI and helped build ChatGPT, has spoken publicly for the first time in months. In a rare interview with Dwarkesh Patel’s podcast, he laid out his sharp critique of the AI industry, and hinted at the focus of his new startup, Safe Superintelligence (SSI), now valued at $32 billion.
Sutskever argues that the industry’s reliance on brute-force “scaling” has hit a wall. Today’s AI models may be brilliant on tests, but they are fragile in real-world applications. He says the pursuit of general intelligence must now shift from simply gathering more data to discovering a new, more efficient scientific principle.
In the current AI gold rush, success is measured by scale: bigger models, more data, and massive multi-billion-dollar compute budgets. Yet Sutskever, a foundational scientist in the field, says this focus has created a deep inconsistency that is slowing AI’s real-world impact.
“You know what’s crazy? That all of this is real,” he said, reflecting on the almost fantastical nature of the moment. But he quickly turned to what he sees as the biggest weakness in today’s most powerful systems.
Sutskever highlighted a puzzling gap between the excellent performance of large language models (LLMs) on tests and their relatively limited economic impact.
“This is one of the very confusing things about the models right now,” he said. “How to reconcile the fact that they are doing so well on evals? You look at the evals and you go, ‘Those are pretty hard evals.’ They are doing so well. But the economic impact seems to be dramatically behind.”
He calls this problem “jaggedness.” A highly competent model can inexplicably get stuck in basic error loops. Sutskever explained it with a vibe coding example:
“You go to some place and then you get a bug. Then you tell the model, ‘Can you please fix the bug?’ And the model says, ‘Oh my God, you’re so right. I have a bug. Let me go fix that.’ And it introduces a second bug. Then you tell it, ‘You have this new second bug,’ and it tells you, ‘Oh my God, how could I have done it? You’re so right again,’ and brings back the first bug, and you can alternate between those. How is that possible? I’m not sure, but it does suggest that something strange is going on.”
He attributes this fragility to two connected issues:
RL Tunnel Vision: Reinforcement learning (RL), now widely used, “makes the models a little too single-minded and narrowly focused, a little bit too unaware.”
Evaluation-Driven Training: With high-quality pre-training data running out, companies carefully define RL training environments. This can lead to models being optimized to ace tests rather than master general skills. Sutskever likened it to a student who “will practice 10,000 hours” for competitive programming, excelling narrowly but failing to generalize.
The Human Advantage: Emotion as the Value Function
At the core of the problem is generalization. “These models somehow just generalize dramatically worse than people. It’s super obvious,” Sutskever said.
Humans, he argues, demonstrate “better machine learning, period.” We are far more sample-efficient and robust, even in areas like coding and math, which we did not evolve for. The difference lies in the human value function, a system that evaluates whether an intermediate step is good or bad, making learning more efficient.
In humans, this value function is shaped by emotions. It is “modulated by emotions in some important way that’s hardcoded by evolution.”
Sutskever illustrated this with a neurological example:
“I read about this person who had some kind of brain damage… that took out his emotional processing… He still remained very articulate… but he felt no emotion… He became somehow extremely bad at making any decisions at all. It would take him hours to decide on which socks to wear.”
Even simple emotions, while imperfect, provide critical guidance. AI systems, currently lacking this integrated system, struggle to self-correct and learn efficiently.
Sutskever says the “age of scaling,” roughly 2020 to 2025, is ending. That period, when big data and compute almost guaranteed progress, is giving way to the “age of research,” where new fundamental ideas are needed.
“The big breakthrough of pre-training is the realization that this recipe is good... Companies love this because it gives you a very low-risk way of investing your resources,” he said.
But now pre-training data is finite, and reinforcement learning, while resource-intensive, does not provide the same low-risk path.
“Is the belief that if you just 100x the scale, everything would be transformed? I don’t think that’s true. So it’s back to the age of research again, just with big computers.”
Sutskever says the bigger problem is ideas, not compute. “One consequence of the age of scaling is that scaling sucked out all the air in the room. Because scaling sucked out all the air in the room, everyone started to do the same thing.”
He believes the solution lies in discovering a fundamental machine learning principle that makes models more productive. “The thing which I think is the most fundamental is that these models somehow just generalize dramatically worse than people.” This, he says, is the big question for the next cycle of AI, and it cannot be solved simply by adding more data or bigger computers.
In regards to SSI, Sutskever said the company has raised $3 billion and has “sufficient compute to prove, to convince ourselves and anyone else, that what we are doing is correct.”
When asked how SSI will make money, he answered: “Right now, we just focus on the research, and then the answer to that question will reveal itself. I think there will be lots of possible answers.”
