Your company is already using AI. If you think it isn’t, that’s actually an even bigger risk.
Let’s be clear. AI is already part of your operations, whether you approved it or not. People are already trying to exploit it. The real question isn’t if your organization faces AI threats, but what those threats look like. Now is the time to understand them.
Most people don’t understand these risks yet. Security researchers have shared detailed findings on LLM vulnerabilities for years, but this information rarely reaches executives. Awareness of these risks is still lacking. This series aims to close that gap.
Why This Matters Right Now
Generative AI adoption in enterprise environments grew faster in 2024 and 2025 than any technology in recent memory. You spent years debating cloud migration strategies but woke up one morning to find that half your workforce was already connected to third-party AI tools, forget policies. Shadow AI is the new shadow IT, and it carries all the same risks and new ones as well …. So, congratulations on that.
The threats against AI systems don’t stop at the vendor’s door. Many of the most serious attacks target the AI that your own team deploys, configures, and connects to your data. The risks we’re going to cover in this series exist on a spectrum, and understanding where your organization sits on that spectrum is the starting point for everything else.
Who Is Actually at Risk in Hacking AI
The answer varies depending on which attack we’re talking about. Some threats apply specifically to organizations with their own AI models. Some of the things we will cover apply to anyone using any AI system. So that pretty much means anyone who got this far in the paper.
So, since I am a glutton for punishment, I’m going to break this out into 8 articles covering both areas. Here is a rough breakdown of how the articles in this series map to your exposure throughout the lifecycle.
THREATS THAT APPLY PRIMARILY TO AI BUILDERS AND DEPLOYERS
- Training-Phase Attacks (Article 2): Poisoning a model during training or fine-tuning. Most relevant if you are training or fine-tuning custom or open-source models on your data.
- Privacy and Extraction Attacks (Article 3): Forcing a model to reveal what it was trained on, including your sensitive data and PII. Relevant when you fine-tune a model on internal data or when your employees have fed sensitive data into AI tools.
- Adversarial Input Attacks (Article 4): Attacks at the token or character level meant to evade safety filters. This one is most important for organizations with AI-powered security tools or automated AI decision systems.
THREATS THAT APPLY TO EVERY AI USER
- Prompt-Based Attacks (Article 5): Hijacking what an AI system does through deviously crafted inputs. This applies to every deployment everywhere.
- Cross-Lingual Attacks (Article 6): Exploiting safety gaps in non-English languages to extract content that would be blocked in English. Applies to any tool accessible in multiple languages, including binary.
- Agentic and System-Level Attacks (Article 7): Compromising AI agents that have been given tools and the ability to act on your behalf. This is the fastest-growing threat category as AI moves from answering questions to doing things, and it scares me the most.
- LLMjacking (Article 8): Stealing your cloud AI credentials to run up costs on your bill while selling access to criminals. Applies to any organization running cloud-hosted AI infrastructure… and if you have read anything else I’ve ever written, you know I kind of have a thing for IAM and credentials.
The article you’re reading now covers all seven threat categories at a level that should give you a working mental model and help you choose others you want to read more deeply. Think of it as a map for our wordy hike together.
The Seven Threats: A First Look
Okay, so you just hired a sophisticated new employee. This employee is incredibly capable, never sleeps, can handle any volume of work, and has been given access to your systems, your customer data, and the ability to take actions on your behalf. Now, around the globe, hundreds of people are actively studying every way that an employee can be manipulated, deceived, corrupted, or kidnapped. That is the situation with enterprise AI right now.
The seven categories of attack we cover in this series each represent a different way that an adversary can go after that employee. Some attacks happen before the employee ever starts work. Others happen when they receive your work instructions. A few take advantage of the fact that the employee happens to speak forty languages, but only thinks safely in a few of them.
We good on that? Okay, let’s check the destinations on the map.
-
Training-Phase Attacks: Poisoning the Well
Before an AI model ever touches a production environment, it is educated by data. Enormous amounts of it. Training-phase attacks target this process, attempting to embed malicious behavior into the model before it ships or later, when you fine-tune or customize a base model for your needs.
Here is something to worry you, for free. No need to thank me. If an adversary can influence a small portion of training data, they can implant a trigger, a word, phrase, or pattern that causes the model to behave in a specific way when it appears. Anthropic’s research demonstrated that it takes only around 250 malicious documents to backdoor a model with 600 million to 13 billion parameters. The brainwashed model will act normally until someone uses the trigger. Then it isn’t.
Does this apply to you? If your organization is using a commercial model from a major vendor, the security during the training phase is mostly their responsibility. But if you are fine-tuning a publicly available model on your own data, or are as ambitious as building custom models, the training pipeline is a very early attack surface.
Fine-tuning is being adopted to customize AI for specific business cases, this category is growing more relevant to more organizations every quarter. How relevant is it to you?
-
Privacy and Extraction Attacks: Stealing What the Model Knows
Models remember things they were trained on. Privacy and extraction attacks exploit this by crafting inputs that cause the model to regurgitate this content, including things you specially told it never to share. Kinda like keeping a secret with a younger sibling. Maybe they will keep it, maybe a crafty adult will ask the question in such a way that the secret will be spilled.
Way back (in history or GPT sense) in 2021, Nicholas Carlini (Et al.) demonstrated that training data could be extracted from GPT-2 at scale. That work has been repeated multiple times since. Research shows that extraction rates improve as the adversary gets more sophisticated, and that scaling up model size doesn’t reduce this risk. Bigger brains have more things to mistakenly tell you.
The practical risk is this: if you feed sensitive data into an AI system, by fine-tuning or employee use of AI tools, some portion of that data may be recoverable by a sophisticated attacker. This is a privacy risk, a competitive risk, and in regulated industries, potentially a compliance risk.
This risk applies to commercial tools and private deployments alike.
-
Adversarial Input Attacks: What the Model Can’t See
AI safety systems are built to recognize harmful requests. Skilled adversarial input attacks are designed to remain invisible to those systems while making the same requests. The techniques range from appending algorithmically optimized character sequences to prompts, to replacing letters with visually identical characters from other Unicode scripts, to encoding malicious instructions in formats the safety layer doesn’t inspect but the model can still interpret.
Research by Carnegie Mellon and others has shown that optimized suffixes, meaningless-looking strings of characters appended to a request, can cause almost any model to comply with requests it would normally refuse. Success rates against some models (in controlled settings) have reached 99 percent. In the wild, against commercial models, they’re lower, but not near zero.
For most organizations, the immediate threat here is to AI-powered security tools and automated decision systems. In these systems, consistent, reliable behavior matters.
Imagine an AI that flags threats or routes requests, manipulating them to create a terrifying attack surface rather than control. Or if you are using AI to control AI, like a content moderator, what would happen if it stopped doing its job?
-
Prompt-Based Attacks: Hijacking the Conversation
OpenAI has said that prompt injection in AI-assisted browsing may never be solved. That’s a weighty statement coming from them.
This is a big one, it affects everyone. Every user of every AI system, everywhere. Prompt injection attacks work by crafting inputs that override existing instructions, causing it to do something you didn’t intend and didn’t authorize.
There are two versions of this, one more insidious than the other. Direct versions involve a user simply asking the model to ignore its rules. During the indirect version, the truly insidious one, malicious instructions are embedded in a document, a webpage, or an email that the AI reads as part of its job.
OWASP has ranked prompt injection as the number one vulnerability in LLM applications for two years running. Success rates in real deployments range from 50 to 84 percent, depending on how the system is configured. That is a massive success rate and should worry you.
The count of real-world incidents is ticking up rapidly. Browsers that summarize web content have been tricked into leaking user credentials. AI-powered resume screening systems were found to be processing injected instructions from “applicants”. Copilot has been manipulated through poisoned emails in the EchoLeak incident and others. If you use AI tools that process external content, prompt injections are an active operational risk today, so get on that.
-
Cross-Lingual Attacks: The Safety Gap Nobody Talks About
AI safety systems are trained mostly with English content. Most safety research on AI is conducted in English. Most red-teams operate in English, and most safety-related data exists in English. In languages where training data is scarce, those safety guardrails are weaker.
Yet more research found that native Bengali speakers using their own language on publicly available AI tools were approximately three times more likely to encounter harmful content than English speakers making the same requests. This is a simple one: the model’s ability to understand and follow safety instructions degrades in languages it wasn’t well-trained on.
Obviously, this has practical implications for global organizations. AI tools deployed in international markets may carry safety gaps that your domestic testing never found. It is a consistent and underappreciated entry point. It is greatly appreciated by your adversaries, though.
-
Agentic and System-Level Attacks: When AI Can Actually Do Things
The hacking AI landscape changed astronomically as AI stops just answering questions and starts taking actions. AI agents given access to tools, APIs, file systems, calendars, email, and databases, and the ability to execute tasks on behalf of users, create a fundamentally different attack surface than a conversational chatbot.
I prefer the term “delegate” to “agent” given the amount of power they have and how it models the user, but that is the subject for another paper.
Indirect prompt injection is a whole different beast in an agentic context. What if your AI agent reads an email, and a hidden instruction in that email causes the agent to forward your files somewhere, reset your credentials, or… or… or…? Now the attack has moved from a content exposure problem to an operational one. Researchers have created self-replicating prompt-injection attacks that spread through multi-agent systems, much like a computer virus spreads through a network. A single poisoned document becomes a breach of epic and nightmarish proportions.
Criminals aren’t just targeting AI agents as victims. They are using them as footholds into enterprise systems.
-
LLMjacking: Stealing the Keys to the Kingdom
The last category is, in some ways, the most familiar to us security folks because it resembles the credential-theft attacks we’ve been fighting for years, just aimed at a new target. LLM jacking is the unauthorized use of your organization’s cloud AI infrastructure via stolen credentials. The attacker pays nothing. You pay the bill. With the rising price of tokens, this can become very expensive, often into the 7-figure range or more.
If they then use that infrastructure to attack you, you are funding your own enemy. Have fun explaining that to the board.
Sysdig first identified this attack pattern in April 2024, when they observed stolen cloud credentials being used to access ten different cloud-hosted AI services. The entry vector was a vulnerable web application. The cost to the victim was estimated at up to $46,000 per day in AI consumption charges. With newer, more capable and expensive models, that number will easily exceed $100,000 per day.
This kind of thing is being carried out by an organized criminal enterprise. Dedicated marketplaces exist for buying and selling stolen access to AI. They even have sales and discounts for volume. In some cases, the stolen credentials were being used not to run up your bill, but to give sanctioned entities, organizations in countries with US technology restrictions, access to AI systems they’re otherwise prohibited from using. Now you are into a potential regulatory and legal liability, not just a financial one. You don’t want the three-letter initial windbreaker-wearing orgs showing up at your offices.
The Inevitability Argument
AI is not optional. Even if you opt out of it, you aren’t actually opting out; your vendors are using it, your competitors are using it, your customers are using it, and your employees are using it on personal devices and bringing the outputs back to work. The question of whether to engage with AI risk has already been answered. The open question is whether you engage with it deliberately.
The organizations that do best are the ones where security leadership understands the threat landscape and has real conversations with the business about risk tolerance, deployment standards, and acceptable use. You can’t have those conversations without understanding what the threats actually are. That’s what this series is for.
A Note on Shared Responsibility
The AI providers bear meaningful responsibility for the security of the models and platforms they run. They have teams working on these problems, and they seem to take it seriously.
But responsibility doesn’t stop with them. If you are relying on them alone, you are failing your security responsibilities. When you put AI in front of your employees and customers, you have taken on security responsibility. Most of the attacks we will cover in this series are not against OpenAI or Google. They are against your organization. That distinction matters, it shows where the work needs to be done.
Security is a shared-responsibility model in AI, just as it is in the cloud. The vendor secures the model. You secure the deployment, data, access, and configuration.
Next: Article 2, Training-Phase Attacks. How a few hundred carefully chosen documents can corrupt an AI model before it ever reaches production, and what that means for organizations building or adopting custom AI.
NOTE ON THE USE OF AI IN THIS DOCUMENT
The document is primarily human-conceptualized, written and edited. AI was used in the research and editing of this document.
Gerard Onorato, CISO|2026
About the Author
Gerard Onorato is a CISO with extensive experience in enterprise security architecture, identity, and risk. This series reflects his personal analysis and does not constitute legal or compliance advice.
