Mapping Israel’s AI infrastructure opportunity
Data centers, sovereign models, and “AI integrity” emerge as the next battleground. “The shift from training to inference to agents isn't just a change in bottlenecks. It's a change in the types of companies that will win,” says Yonatan Mandelbaum, partner at TLV Partners.
For nearly a decade, Israel’s artificial-intelligence scene was defined by algorithms trained to answer narrow questions: Is this a tumor or not? Is this transaction fraudulent? Does the construction site match the blueprint? Today that world is receding fast. The country’s AI ecosystem is being reorganized around a far more ambitious goal, building autonomous agents that can reason, act, and be trusted inside the core systems of governments and corporations.
“A lot has changed since we last mapped Israel's machine learning infrastructure ecosystem,” says Yonatan Mandelbaum, partner at TLV Partners. “It's been nearly eight years since our first landscape and seven years since the sequel, and if you told us then that we'd be living in a world in which Israelis colloquially refer to one of the most impressive consumer products ever built as ‘Ha-chet’, we wouldn’t have believed you.”
Mandelbaum describes the shift as a series of abrupt technological eras. “The AI infrastructure opportunity has shifted dramatically three times in less than a decade. First it was about training. Then it was about inference. Now it's about agents.”
During what he calls the Training Era, from 2015 to 2022, the central problem was how to build models that could predict binary outcomes. Companies struggled with data collection, labeling, and the mechanics of turning research into repeatable engineering.
The arrival of large language models overturned that logic. “The birth of LLMs saw the emergence of the AI hyperscalers, which took upon themselves the massive costs and burden of training ‘super’ models,” Mandelbaum notes. These systems were no longer predicting a single outcome but generating sequences of possibilities. The constraint shifted to inference, how to serve enormous models quickly and cheaply. Token costs pushed many firms toward open-source alternatives such as Llama, Kimi, and Qwen, which could be fine-tuned and run on private infrastructure.
By late 2024 another inflection arrived. Models were finally good enough; the new challenge was usefulness. “The bottleneck is how to get them to do useful work reliably in the real world. Which is to say: the bottleneck is now agents,” Mandelbaum explains. The questions confronting engineers are less about model size and more about behavior: retrieving the right context, planning actions, coordinating with other agents, and proving that the system did what it was asked.
“For the most part, these are engineering challenges. Exactly the type of problems Israeli founders are uniquely qualified to solve,” he says.
That confidence is backed by a wave of infrastructure investment. Nebius has deployed 4,000 Nvidia HGX B200 GPUs in Israel, one of the country’s first publicly available Blackwell installations, and signed agreements for 80 megawatts of data-center capacity with almost $900 million in investment. Crusoe’s acquisition of Atero signals a similar bet on local talent. Nvidia, meanwhile, has effectively made Israel its second global headquarters following the Run:AI and Deci acquisitions and plans for a massive campus in Kiryat Tivon.
These moves are beginning to anchor what Mandelbaum calls “agentic sovereignty.” “The dependence on American or Chinese model providers is a strategic vulnerability,” he warns. “When agents control infrastructure, manage sensitive data, and make decisions, you can't outsource that capability entirely.” Companies such as AI21, AAI, and Decart have started local model building, but he argues that “more attempts are needed.”
A New Stack
If models are only one component, the rest of the machinery is still missing. Production-grade agents require orchestration frameworks for complex workflows, observability tools to track actions, memory systems that preserve context across days, and integration layers that connect software to the physical world.
“Evaluation frameworks are perhaps the most critical missing piece,” Mandelbaum says. Enterprises need ways to test and measure agent behavior and to train systems through feedback loops. “The currently available tools simply aren’t robust enough. Agents need to remember conversations, learn from past mistakes, be evaluated and recall relevant context at the right time.”
The result is “an entire new infrastructure stack” that has yet to be built, an opening that Israeli startups are rushing to fill.
The most sensitive frontier is trust. Mandelbaum argues that traditional cybersecurity labels are inadequate. “While AI security may be the soup du jour, we believe that a more appropriate title for AI security in the agentic infra era is: AI integrity.” The issue is not only preventing breaches but guaranteeing that autonomous systems act exactly as intended when they can read emails, transfer money, or write code.
“Israel has a natural edge here, rooted in its culture of adversarial thinking, where security emerges from system architecture rather than add-on defenses,” he says. Integrity must be baked in through provable execution paths, sandboxing, and interpretable decision-making. The companies that master this “won’t sell traditional security tools, they’ll build the foundational infrastructure for trustworthy agents, on par with systems protecting financial, defense, and nuclear assets.”
For Mandelbaum, the strategic implication is clear. “The pool of local talent is incredibly well versed in making systems more efficient, more secure, and more reliable. In other words: helping make AI systems work in production.” The progression from training to inference to agents is not simply a technical evolution but a reshuffling of winners.
“The shift from training to inference to agents isn't just a change in bottlenecks. It's a change in the types of companies that will win. And everything about that change favors Israeli engineering talent.”
