LLM hijacking: The hottest phenomenon in AI cyber
According to Dima Tatur, VP of cyber defense at Commit, “Once attackers gain direct access to an organization's AI wallet, and by the time the organization realizes that anything is even happening, it can result in hundreds of thousands of dollars in damage within a week to 10 days.”
An economic iron rule states that once the price of a product increases, a market and phenomena will develop around it in response to the rising price, both legitimate and illegitimate. In recent months, the scope of use and organizational spending on the use of AI models has increased significantly, and the iron rule has made its presence felt. On the legitimate side, organizations are transferring tasks from expensive Western models to cheap Chinese models. On the illegitimate side, cybercriminals are dressing up as organizational access to large language models (LLMs), hijacking the use in favor of selling it to other parties.
This attack, known as LLMjacking, is quickly becoming one of the most significant cyberattacks in the field, one that can cause enormous direct damage to organizations in just a few days, and has already resulted in cumulative organizational losses estimated at a billion dollars this year.
"LLM hijacking is the hottest phenomenon in AI cyber right now," Dima Tatur, vice president of cyber defense at Commit, told Calcalist. "Once attackers gain direct access to an organization's AI wallet, and by the time the organization realizes that anything is even happening, it can result in hundreds of thousands of dollars in damage within a week to 10 days."
LLM hijacking is not a new phenomenon. It was identified by researchers about two years ago, but in recent months, against the backdrop of the expansion of the use of AI and the rising costs of using AI in organizations, its scope has grown significantly. “The losses are averaging $50,000 to $100,000 per day per company affected,” said Tatur. “In investigations of incidents I have been involved in, we have seen accounts reaching within 10 days for losses of half a million dollars.”
Hijacking of business AI accounts is possible because AI agents and enterprise interfaces do not work with a chatbot interface, but directly with the model's API (a direct connection interface to the AI model). The agent identifies itself with the model using an API key, a code unique to the organization that functions as an identification password.
Darknet token exchanges
“The fundamental problem starts with the fact that unless an organization explicitly defines it, that API key acts as a completely open account and allows for unrestricted use,” said Tatur. “In LLM hijacking, attackers steal organizations’ API keys and gain direct access to their AI wallet. How does this happen? Developers build apps and AI agents, but most organizations are not aware enough and developers leave the keys exposed in the application code, the attacker detects the vulnerability, penetrates the application, gets to its configuration files, and from there pulls the model’s API key. In other cases, developers simply upload the code, with the API key, to public code repositories like GitHub.”
The attackers often don’t use the stolen API keys themselves, but act as intermediaries for other organizations. “The attackers set up a kind of token exchange (a measure of the extent of AI usage) on the darknet. They present an endpoint that the user goes to, and behind it they have hundreds and thousands of stolen keys. They don’t tell the organization, ‘Buy a stolen key from us,’ but rather, ‘Instead of paying $5 for a million tokens directly to the model provider, go to our server and pay only $1.’ It’s a behind-the-scenes service trade that is entirely based on stolen keys,” said Tatur.
There are several types of organizations that purchase stolen API keys. One type is actors who want to use AI for illegal activities, such as planning and executing a cyberattack. Such an entity will want to obscure its identity as much as possible, so it will knowingly purchase a stolen API key and use it to work against the model to plan the attack.
However, there are cases where legitimate companies purchase stolen keys without being aware of it. “In the world of tokenomics (the token economy), there is an arbitrage market — companies that buy huge quantities of tokens and sell them at a discount of up to 40%.
“Many organizations turn to these companies innocently to save costs, but behind the scenes, those same suppliers chain services and buy from third parties, and along the way, stolen keys are incorporated. This way, the company operates in a completely legitimate transaction, without knowing that in the layers behind the scenes there are stolen keys,” said Tatur.
The direct financial impact on the organizations from which the keys were stolen can be significant. “A customer contacted us and received a monthly bill of almost $400,000 and did not understand where it came from,” Tatour said. “We discovered that the attacker had left himself a backdoor in the system. When the customer first realized that a key had been stolen, he just stuck a new key in the code without addressing the breach — and the attacker simply stole the new key as well. We entered the model with the key, analyzed the prompts that ran there, understood what was stolen, located the malware that was implanted in the application and neutralized it.”
The direct financial impact can be secondary to the indirect damage of an LLM hijack. One of the damages is the disabling of organizational access to AI capabilities. “If an organization with awareness sets a monthly consumption cap, say $300,000, and the attackers overload the account, the consumption reaches its maximum. Once the budget is locked, the model is blocked and the entire organizational application is disabled,” said Tatur.
AI companies are not responding
Access disruption can also occur when an organization detects an API key theft but responds incorrectly: “If a company immediately replaces the API key in the model, it could create a self-shutdown. The previous key was embedded in dozens of agents and applications, and if you replace it in the model before updating all the surrounding systems, everything crashes and stops functioning.”
Another danger is the exposure of organizational information. “Once the attacker has obtained the API key, they have direct access to the entire history and entire memory window of that organization,” explained Tatour. "Everything that was previously done with this key is exposed to the attacker — both directly and through new questions that it asks the model, which can be based on the information stored in previous sessions. For example, an application that serves customers and operates through an AI agent. The application can transmit corporate information or sensitive medical information, depending on the type of activity. Once the attacker has the API key that the application and the agent used, he can directly access the model with it, see the entire history and extract all the information that passed there."
To prevent API key theft, Tatur recommends that organizations use virtual keys, which allow for separation between the agent and the model and provide full visibility with real-time alerts on anomalies. In the event of an LLM hijacking attack, an organization should take three steps.
Tatur: “First, block the compromised key and immediately limit the consumption limit. Second, perform a hard swap — move the new key to a secure vault and do not plant it again in the code, to prevent repeated theft. And third, investigate the infiltration path, and contact the model developers to report the theft and try to receive a financial credit.”
However, organizations are having difficulty getting a response from companies like OpenAI and Anthropic. "The companies that develop models don't really provide information, there's not enough visibility. Therefore, the ability to manage the event with them is relatively limited. It's usually through emails or with some kind of bot."
