Insights · AI Security
AI cybersecurity threats, explained: and why most lists are unusable
Three different problems get filed under one heading. They have different owners, different controls and very different likelihoods inside a CUI boundary — and only one of them is where most defense contractors are actually exposed.
Most AI threat lists are unusable because they mix three unrelated problems under one heading. Prompt injection sits next to deepfake phishing, which sits next to employees pasting documents into a chatbot. All real. None of them the same kind of problem.
They have different owners, different controls, and different likelihoods. Handing that list to a forty-person machine shop produces paralysis, because there is no way to tell which items apply.
This is the reference I give teams so everyone works from the same definitions. Each threat gets a definition, how it works, what it means inside a CUI boundary, and the controls that answer it. Where a threat maps to a published taxonomy I’ve noted it — citing OWASP or MITRE in your documentation is more useful than inventing your own naming.
Framing
The three categories, and why they behave differently
| Category | What it is | Primary owner | Likelihood for a 40-person contractor |
|---|---|---|---|
| One — AI used against you | Familiar attacks with the operator cost removed | Security and IT | High. Already happening. |
| Two — attacks on AI you deploy | Manipulation of models and agents in your environment | Whoever builds or deploys | Low unless you deploy. High if you do. |
| Three — risks you create | Exposure from your own adoption, no adversary involved | Governance, procurement, legal | Very high. Usually undetected. |
Category one needs no AI strategy. The controls that answer it are controls you already own and are already required to operate. Category two is an architecture problem and only exists if you build something. Category three is where nearly every contractor I assess is exposed, and it’s the category least likely to appear in a threat briefing, because nothing about it looks like an attack.
Category one
Attacks that use AI against you
Old attack patterns with the cost removed. Nothing here requires the attacker to understand AI beyond how to use a tool.
AI-generated spear phishing
Definition. Targeted phishing where message content is generated from information about the target, at a volume and quality that previously required manual effort per victim.
How it works. Your public material becomes source data. Capability statements, past performance write-ups, conference bios, award announcements. The generated message carries correct program names, correct acronyms, correct internal tone. The language errors most awareness training is built around are gone.
In a CUI environment. Contracts and finance staff are the targets, not engineers. The valuable outcomes are a redirected payment, a handed-over credential, or an attachment opened by someone who deals with external parties all day as part of the job.
Controls. Phishing-resistant MFA. Out-of-band verification for payment and contract changes, written as procedure naming who calls whom on which number. Email authentication at enforcement, not monitoring. Awareness training that stops teaching people to look for bad grammar.
Voice and video impersonation
Definition. Synthetic audio or video of a specific real person, used to authorize an action or extract information.
How it works. A short sample of recorded speech is enough — panels, podcasts, webinars, recorded all-hands, voicemail greetings. Deployed in a live call or voice message, with urgency and a plausible reason the normal channel isn’t available.
In a CUI environment. This breaks the verification habit most companies rely on. “Call them back and confirm” assumed a voice was hard to fake. Executives with public speaking histories are the easiest to clone, which means the people who can approve exceptions are the people most easily impersonated.
Controls. Verification that doesn’t depend on recognizing a voice: callback to a number from your own directory, a code word for financial authorization, dual approval above a threshold. Apply it to leadership without exception, because leadership is the target.
Automated reconnaissance and vulnerability discovery
Definition. Machine-assisted scanning, enumeration and analysis of a target’s external attack surface, run across many targets at once.
How it works. Enumerate subdomains, identify exposed services, fingerprint versions, correlate against known vulnerabilities, prioritize by exploitability. All possible before. What changed is that it runs against thousands of organizations in parallel rather than one at a time.
In a CUI environment. Your unpatched edge appliance was always a weakness. The probability that someone looks for it went up. Small subcontractors were protected by the attacker’s time budget, and that protection is gone.
Controls. Scanning on a defined cadence with remediation timelines you can evidence. Attack surface inventory including systems nobody remembers standing up. Edge device patching treated as a priority tier.
MITRE ATLAS — reconnaissance techniques
AI-assisted malware
Definition. Malicious code generated or modified with model assistance, typically to vary structure and evade signature-based detection.
How it works. Lowers the skill floor for producing functional malware and makes variants cheap to generate. Reported measurements put AI-generated malware at roughly a fifth of observed AI-driven attacks, well behind phishing and impersonation.
In a CUI environment. Less significant than the attention it receives. If detection depends on signatures alone it matters more, and most contractors in the 800-171 population run endpoint tooling that is at least partly behavioral.
Controls. Behavior-based endpoint detection. Application control on systems handling CUI. Keep this in proportion — the two entries above deserve more of your budget.
Credential attacks at scale
Definition. Password spraying, credential stuffing and account takeover made cheaper and better targeted through automated analysis of breach data and organizational patterns.
How it works. Naming conventions inferred from public directories, password patterns predicted from previously breached credentials belonging to the same people, attempts paced to stay under lockout thresholds.
In a CUI environment. The targets are your externally exposed authentication surfaces: VPN, remote desktop, webmail, any SaaS holding CUI. One reused credential on an account without MFA remains among the most common ways in.
Controls. MFA on every externally reachable authentication surface, with no exception for service accounts or legacy applications. Conditional access. Authentication anomaly monitoring. Credential checks against known breach corpora.
Category two
Attacks against the AI systems you deploy
This applies if you run AI inside your own environment — an assistant with access to internal documents, an agent that acts on your behalf, or a model you fine-tuned. If your people only use commercial tools through a browser, skim this and go to category three.
Prompt injection
Definition. Instructions embedded in content the model processes, which the model follows as though they came from the authorized user.
How it works. Direct injection is a user typing instructions that override the system prompt. Indirect injection is the serious one: instructions hidden in a document, email, web page or file the model reads while doing its assigned job. The model has no reliable way to separate data it should analyze from instructions it should obey.
In a CUI environment. An assistant with retrieval access to your document store is the risk case. A poisoned document tells it to summarize an unrelated folder and include the result. Nothing is exploited in the traditional sense. The system does what it was asked.
Controls. Treat everything the model reads as untrusted. Constrain retrieval to what the requesting user is authorized to see. Require confirmation before any action with external effect. Log inputs and outputs. Never architect a system where injection results in privileged action.
OWASP LLM01
Insecure output handling
Definition. Downstream systems consuming model output without validating it, allowing generated content to become executable or trusted.
How it works. Output flows into a browser, a shell, a database query, a code path or a document processed automatically. If output is trusted at that boundary, whoever influences the model influences the downstream system.
In a CUI environment. Relevant if model output has been built into a workflow — generated code committed with light review, output rendered in a web interface, generated queries executed against a database.
Controls. Validate and encode output at every boundary. Never execute generated code without review. Least privilege on whatever consumes the output.
OWASP LLM02
Training data poisoning
Definition. Manipulation of training or fine-tuning data to introduce a bias, backdoor or specific failure mode into the resulting model.
How it works. An attacker contributes data to a corpus or compromises the pipeline assembling it. The result behaves normally except under a trigger the attacker chose.
In a CUI environment. Applies only if you fine-tune, and most contractors don’t. If you do, and the training data includes anything controlled, that pipeline is in scope for your SSP and needs the same protection as any other CUI system.
Controls. Provenance for training data. Access control and integrity checks on the pipeline. Evaluation against held-out data. Version control over datasets and model artifacts.
OWASP LLM03
Model and data extraction
Definition. Recovering training data, system prompts or model parameters through crafted interaction with a deployed system.
How it works. Membership inference determines whether a record was in the training set. Extraction recovers memorized content verbatim. System prompt leakage exposes the instructions and often the internal context supplied alongside them.
In a CUI environment. Consider it wherever controlled data has been used to fine-tune, or placed in a system prompt or retrieval context. Anything a user-facing model can reach, a determined user may be able to surface.
Controls. Don’t fine-tune on controlled data unless the deployment sits inside the boundary. Keep sensitive content out of system prompts. Rate limiting and monitoring for extraction patterns. Treat the model as a system that can disclose whatever it was given.
OWASP LLM06 · LLM10
Supply chain compromise
Definition. Compromise introduced through a model, dataset, library, plugin or extension obtained from a third party.
How it works. A model downloaded from a public repository, a browser extension with AI features, a plugin connecting a tool to your data, a package in the dependency tree. Any of them can carry malicious behavior, and model artifacts are particularly difficult to inspect.
In a CUI environment. The browser extension case is the one I find. An extension installed by a user, granted permission to read page content, on a machine where CUI is displayed in the browser.
Controls. Source models and components from vetted origins with integrity verification. Extension allow-listing on managed devices. Dependency scanning. Vendor review covering AI components specifically.
OWASP LLM05
Excessive agency
Definition. An AI system granted more capability, permission or autonomy than its task requires, so that a manipulated or mistaken model takes consequential action.
How it works. An agent able to send email, modify files, call APIs or authorize transactions. When something upstream goes wrong — injection, a hallucinated instruction, an ambiguous request — the consequence is an action taken rather than a wrong answer displayed.
In a CUI environment. This is where the threat picture is heading, and where design review effort belongs. An agent with a connector into your document store, operating with the permissions of whoever authorized it, is a privileged non-human identity your access review process almost certainly doesn’t see.
Controls. Least privilege on what the agent can do, not only on what it can read. Human confirmation for actions with external effect. Scoped, expiring credentials rather than inherited administrative access. Full logging of agent actions. Non-human identities included in access reviews.
OWASP LLM08
Guardrail bypass
Definition. Techniques that cause a model to produce output its safety configuration was meant to prevent.
How it works. Role-play framing, instruction layering, encoding, language switching, incremental escalation. New methods appear faster than vendors patch them.
In a CUI environment. Mostly an acceptable-use and reputational concern rather than a CUI concern — unless the model has access to controlled data, in which case the real control was never the guardrail. It was the access.
Controls. Don’t treat model-level safety configuration as a security boundary. Enforce authorization outside the model, in the systems holding the data.
Category three
Risks you create yourself
No adversary. These are the ones I find in nearly every environment and the ones least likely to appear in a threat briefing.
Shadow AI
Definition. Use of AI tools inside the organization without approval, review or visibility.
How it works. An employee with a deadline finds a tool that helps. It’s free, browser-based, requires no installation, and triggers nothing in your procurement or software management process.
In a CUI environment. Controlled data entering an unapproved tool is an unauthorized disclosure, potentially reportable within 72 hours under DFARS 252.204-7012. If your SSP doesn’t mention AI tools, you have no approval workflow to cite, no policy to point at, and no record of what left.
Controls. Inventory first. Then a fast approval path, then boundary controls, then policy — in that order. Restriction without an approved alternative moves the activity to personal devices.
Vendor retention and training on inputs
Definition. Terms permitting a provider to retain submitted content or use it to improve their models.
How it works. Consumer tiers commonly retain inputs and may train on them. Enterprise tiers usually don’t. The difference is contractual, not technical — same interface, same brand, different terms.
In a CUI environment. Free-tier use of a tool your organization also holds an enterprise agreement for is common and real. The user sees a familiar product. The data goes somewhere else.
Controls. Verify tier and terms per tool, not per vendor. Enforce authentication through your tenant so personal accounts can’t be used on managed devices. Record retention and training terms in vendor documentation.
Unlabeled derived output
Definition. Content generated from controlled material — summaries, extracts, translations, rewrites — carrying the same sensitivity and none of the marking.
How it works. A user summarizes a controlled document. The summary appears in a chat window with no header, no marking, no classification. It gets pasted into an email.
In a CUI environment. Your marking procedures cover documents. They rarely mention generated derivatives, so people handle them as new material rather than derived.
Controls. State in your handling procedure that AI-generated derivatives inherit the classification of their source. Train on it. Where tooling allows, apply labels automatically to content produced inside the boundary.
Over-permissioned connectors
Definition. An AI tool authorized to access a data source with broader permissions than its function requires, usually inheriting the rights of whoever authorized it.
How it works. A user clicks through an OAuth consent screen. The grant persists. When an administrator authorizes it, the tool inherits administrative reach across a document store, and a standard access review shows only that the administrator’s account is appropriate.
In a CUI environment. A quiet, persistent, privileged pathway into controlled data that most access review processes never surface.
Controls. Inventory OAuth grants and connectors. Include non-human identities in access reviews. Restrict who can consent to third-party applications. Map every connector to an owner, a justification and the data it can reach.
AI-generated compliance documentation
Definition. Policies, SSP content or assessment responses produced by a model and submitted without verification against the actual environment.
How it works. The output is fluent, well-structured and describes controls that sound right. Some aren’t implemented. Some don’t apply. Fluency reads as accuracy when the reviewer is short on time.
In a CUI environment. An SSP describing controls you don’t operate is a false statement about your security posture, submitted under a contract. Contracting officers check SPRS before award under DFARS 252.204-7019, and a representation you can’t support carries False Claims Act exposure.
Controls. Verify every control statement against the environment before it enters the document. Use AI for structure and language, never for factual assertions about your systems. Name the human who verified each section.
Vendor-activated AI features
Definition. AI capability introduced into a product you already use, through an update you didn’t request.
How it works. A SaaS platform inside your boundary enables an assistant feature in a routine release. No purchase, no request, no change ticket. The capability is simply there on Tuesday.
In a CUI environment. Your approved tool inventory is now inaccurate, and nothing in your process told you.
Controls. Recurring review of release notes for in-scope products. Treat new AI features as changes requiring assessment. Negotiate advance notice into contracts where you have leverage.
Evidence
What the published data actually says
| Finding | Figure | Source |
|---|---|---|
| Malicious breaches that were AI-enabled | 1 in 4, up 56% year over year | IBM 2026 Cost of a Data Breach |
| Average cost of an AI-enabled breach | ~$6M against a $4.99M global average | IBM 2026 Cost of a Data Breach |
| Security incidents involving unapproved AI tool use | 43%, more than double the prior year | IBM 2026 Cost of a Data Breach |
| Breached organizations with no AI governance policy | Close to 7 in 10 | IBM 2026 Cost of a Data Breach |
| Defense contractors named as ecosystem entry points | Qualitative finding | Flashpoint 2026 Global Threat Intelligence Report |
One caveat that matters more than the numbers. The IBM study covers 602 organizations across all sectors, not the defense industrial base. Treat those figures as cross-sector signal about direction, not as a measurement of DIB risk. The Flashpoint finding is the only one in the table that names this audience directly.
Prioritization
Which category applies to you
Everyone gets category one. The controls that answer it are ones you’re already required to operate. If MFA, verification procedure and patch cadence are solid, most of it is handled without anything called an AI strategy.
Category two applies only if you deploy. Ask one question: does any AI system in your environment have retrieval access to internal data, or the ability to take an action rather than return an answer? If no, this category is reading material. If yes, prompt injection and excessive agency are the two to design against from the start, before the system reaches production.
Category three is where most contractors are exposed and don’t know it. No attacker, no exploit, no alert — just a gap between what people do and what the documentation says. It’s invisible until someone counts.
The exercise that settles it
Put IT, engineering, contracts and one program manager in a room. Ask each person which AI tools they have used for work in the last thirty days, including free ones, browser extensions, and features inside tools you already own. Make it explicitly non-disciplinary before you start, or you will get a clean list and a false one.
Compare the result to your approved software inventory. The gap between those two lists is your category three exposure, and it’s the only number in this brief that describes your company.
Controls
Controls that answer more than one threat
Five controls carry most of the weight across all three categories. If budget is the constraint, these come first.
| Control | Threats answered | 800-171 family |
|---|---|---|
| Phishing-resistant MFA | Spear phishing, credential attacks, impersonation follow-through | Identification & Authentication |
| Out-of-band verification procedure | Voice and video impersonation, BEC, contract fraud | Access Control · Awareness & Training |
| AI tool inventory and approval path | Shadow AI, vendor retention, derived output, vendor-activated features | Configuration Management |
| Non-human identity review | Over-permissioned connectors, excessive agency, supply chain | Access Control · Audit & Accountability |
| Egress control and logging | Shadow AI, data extraction, spill investigation | System & Communications Protection |
Nothing on that list is new security thinking. These are requirements that were always there. What changed is that AI created new places where they get tested, and most documentation written before 2025 doesn’t mention those places at all.
References
Primary sources for this brief
- OWASP Top 10 for Large Language Model Applications
- MITRE ATLAS — Adversarial Threat Landscape for AI Systems
- NIST AI Risk Management Framework (AI 100-1)
- IBM Cost of a Data Breach Report 2026 — Ponemon Institute, 602 organizations, all sectors
- Flashpoint 2026 Global Threat Intelligence Report
- DFARS 252.204-7012, -7019 · 32 CFR Part 170 · NIST SP 800-171
FAQ
Common questions
We don’t build AI systems. Can we skip category two entirely?
Mostly, with one check. If any tool in your environment has been granted retrieval access to internal document stores — an assistant connected to your file share or mailbox — you have deployed something, even if you didn’t build it. Prompt injection and excessive agency apply to that configuration.
Is banning AI tools a valid control?
Not on its own. A ban with no approved alternative reduces visibility rather than risk. The activity continues on personal devices where you have no logs and no recourse, and you find out during an assessment. Ban after you’ve built an approval path, not before.
Does using an AI tool automatically put it in CMMC scope?
It depends on whether the tool processes, stores or transmits CUI, or provides a security function to the environment. A tool used only for marketing copy is out. A tool an engineer uses on technical documentation is a CUI asset. The determination follows the same asset categorization every other system goes through — see the scoping brief.
If an employee pasted CUI into a chatbot, is that reportable?
Treat it as a potential unauthorized disclosure and run your incident process. Reportability under DFARS 252.204-7012 is a determination your legal or contracts function makes on the facts — what data, which tool, what terms, what retention. Decide who makes that call before you need it, not during.
Can we use AI to write our SSP?
For structure and language, yes. For factual assertions about your environment, no. Every control statement has to be verified against what your systems actually do before it goes in the document, and a named human should own that verification. An SSP describing controls you don’t operate is a representation problem, not a drafting problem.
Which of these should a fifty-person contractor address first?
The inventory in category three. Until you know which AI tools and AI-enabled features are already touching your environment, every other decision on this page is theoretical.
If you can’t produce that inventory, that’s the starting point
We run it as a fixed-scope engagement — AI in CUI Environments, two weeks, $4,500: tool inventory, data flow analysis, acceptable use policy for in-scope systems, and a guidance memo for leadership sign-off.
Book a scope call if you’d rather talk it through first. Thirty minutes, no charge.
AI SECURITY SERIES
- AI cybersecurity threats: the three categories You are here
- AI in a CUI environment: what changes in each governance process Published
- Running the AI tool inventory: a working method Next
- Acceptable use policy that survives an assessment Coming
- Non-human identity: connectors, agents and access review Coming
This brief reflects published guidance and reporting as of 28 August 2026. Threat taxonomies referenced here are maintained externally and change; verify against current OWASP and MITRE publications before citing them in your own documentation. Advisory content, not legal advice.
Cyber DSC · Insights · AI Security Series 01 · 28 Aug 2026