Insights · Infrastructure
Data center security: what AI changes, and what it doesn't
Most of a data center's control set is unchanged by AI. A specific and identifiable subset breaks — and it's the subset almost nobody covers in a vendor questionnaire.
A defense contractor asked me last month whether their cloud provider's AI service was "still FedRAMP." Reasonable question, wrong shape. The provider was authorized. The specific accelerator capacity they wanted to use sat outside that authorization boundary.
That gap is the theme of this brief. AI doesn't rewrite data center security. It stresses a narrow set of controls that most assessments treat as settled, while leaving the majority of the control set exactly as it was. Knowing which is which saves you from two failure modes: assuming nothing changed, and assuming everything did.
Baseline
What doesn't change
A data center is a building with controlled access, conditioned power, environmental monitoring and physical media in it. None of that is different because the racks contain accelerators.
Physical access control, visitor escort, badge and biometric logging, camera coverage and retention, cage and cabinet separation, personnel screening, media sanitization procedure, change management, environmental alerting, fire suppression, and the audit trail across all of it — the control objectives are identical. If a facility was well run for traditional workloads, most of that work carries over unchanged.
Say so plainly in your documentation. Assessors and clients are both being told that AI changes everything, and a clear statement of what didn't change is more credible than a rewrite.
Availability
Power, cooling, and availability as a security property
Training and inference racks draw an order of magnitude more power per rack than general compute, and dissipate it as heat. Air cooling stops being sufficient, so facilities move to direct-to-chip or immersion liquid cooling. That introduces plumbing, pumps, coolant distribution units and leak detection into a space that previously had fans.
The security consequence is not the plumbing. It's what controls the plumbing.
Building management systems, power distribution controllers and cooling control loops are operational technology. They frequently run on flat networks, use legacy protocols without authentication, are patched on a facilities schedule rather than an IT schedule, and are often reachable by a third-party maintenance vendor. When cooling was a comfort function, compromise meant discomfort. When cooling is the only thing preventing thermal shutdown of a cluster running a contracted workload, availability becomes a security property with a real adversary path.
What to change. Bring the BMS and power control network into your asset inventory and your segmentation design. Treat facilities vendors as third-party access requiring the same review as any other remote maintenance path. Test the failure mode: what happens to workloads, and to the data in flight, when cooling fails and nodes shut down hard.
Architecture
Network architecture: the east-west problem
Traditional enterprise architecture assumes most interesting traffic crosses a boundary you control and inspect. Segmentation, north-south inspection and host-based monitoring are built on that assumption.
A training cluster inverts it. The dominant traffic is node-to-node across a dedicated high-speed fabric, moving gradients and parameters between accelerators. These fabrics are engineered for latency, not for inspection. Remote direct memory access moves data between hosts while bypassing the operating system and CPU, which means host-based agents do not see it. Encryption and authentication on those fabrics are frequently optional and frequently off, because they cost performance.
So you have a high-volume, high-value, largely uninspected internal network carrying exactly the material you care about most.
What to change. Treat the cluster fabric as its own security domain with its own documented boundary, rather than as cabling inside a trusted zone. Physically or logically isolate it from management and general enterprise traffic. Where the fabric supports authentication and encryption, enable it and record the performance decision if you don't. Monitor at the fabric layer and at the job scheduler, since your endpoint tooling is not going to help you here.
Isolation
Tenancy and isolation on shared accelerators
CPU virtualization has three decades of hardening behind it. Accelerator partitioning does not. Vendor mechanisms for slicing a GPU between workloads are improving quickly, but the isolation guarantees, the residual-data behaviour of device memory between tenants, and the side-channel research base are all younger and thinner than the equivalent for CPUs.
For a defense contractor this is a scoping question rather than a research question. If controlled data is processed on shared accelerator capacity, the isolation boundary between you and the other tenant is part of your security boundary, and you need to be able to describe it.
What to change. For controlled workloads, prefer dedicated rather than shared accelerators, and get the tenancy model in writing. Where sharing is unavoidable, ask specifically how device memory is cleared between workloads and what the provider's answer is grounded in. Record the decision and its rationale — this is a POA&M item if the answer is unsatisfactory and the workload proceeds anyway.
Supply chain
Silicon, firmware, and weights
Three supply chain layers matter here, and only one of them is familiar.
Hardware and firmware. Accelerators, network interface cards and data processing units all carry their own firmware and, increasingly, their own processors. Baseboard management controllers sit below the operating system and outlive it. Firmware provenance, signed updates and BMC network isolation are the controls, and they are frequently nobody's job.
Export control. Advanced accelerators are subject to export controls, and where compute physically sits carries legal weight independent of any security consideration. For contractors with ITAR or EAR exposure, "which country is this capacity in" is a compliance question before it is an architecture question.
Model artifacts. Weights are a new class of asset. They are compact relative to the data used to produce them, portable, expensive to reproduce, and in some cases derived from controlled inputs. A checkpoint file on scratch storage may carry the sensitivity of everything that trained it, with none of the marking. Your media handling procedure almost certainly does not mention it.
Data
Data lifecycle inside the facility
| Stage | Traditional workload | AI workload |
|---|---|---|
| Ingest | Transactional, steady, small units | Bulk staging of large datasets, often copied to local scratch |
| Processing | Data stays in application boundary | Data replicated across many nodes and into device memory |
| Intermediate state | Logs, temp files, database journals | Checkpoints written continuously, often to fast local NVMe |
| Output | Records, reports, files | Model weights — a derived asset with inherited sensitivity |
| Disposal | Documented sanitization of known media | Scratch volumes and device memory frequently outside the procedure |
The disposal row is where I find gaps. A sanitization procedure written for servers and arrays rarely names ephemeral cluster scratch space, and almost never names accelerator memory. Neither is exotic — both just postdate the procedure.
Compliance
Where the frameworks land
| Framework | What it covers here | What it does not |
|---|---|---|
| NIST SP 800-53 | Facility and system control baseline, including physical and environmental families | Accelerator-specific isolation; cluster fabric assumptions |
| NIST SP 800-171 | Requirements for CUI in nonfederal systems, including any AI infrastructure processing it | Nothing AI-specific; the requirements are architecture-neutral by design |
| FedRAMP | Authorization of a cloud service offering, at a defined boundary | Newer services or accelerator capacity added outside that boundary |
| SOC 2 / ISO 27001 | Colocation provider control assurance | Whether the specific controls you rely on were in the audit scope |
| NIST AI RMF | Organizational AI risk governance | Facility, hardware, and infrastructure controls |
The recurring trap is the boundary question. A provider is authorized; a particular service within that provider may not be. Under DFARS 252.204-7012 a cloud service provider handling CUI must meet FedRAMP Moderate or equivalent security requirements, and that determination attaches to the service you are actually using, not to the vendor's name.
Procurement
If you're buying capacity rather than building it
Almost no company in the defense industrial base is constructing an AI data center. You are buying accelerator capacity from a hyperscaler, a specialist GPU cloud, or a colocation provider. Which makes this a diligence problem, not an engineering one.
Five questions before the first GPU contract
- Is this specific service inside the authorization boundary? Not the provider — the service, by name, on the authorization document.
- Where is the capacity physically located, and who can touch it? Country, facility, and the nationality and screening standard of personnel with physical or administrative access.
- Dedicated or shared accelerators, and how is device memory handled between workloads? A vague answer is an answer.
- What happens to our data at contract end? Training data, checkpoints, weights, logs, and scratch volumes — each named, with a sanitization method and a timeframe.
- What did the SOC 2 or equivalent actually cover? Ask for the scope section, not the certificate.
If you hold CUI and cannot get clean answers to one and four, that capacity is not usable for controlled work regardless of how good the pricing is.
References
Primary sources for this brief
- NIST SP 800-53 — Security and Privacy Controls, physical and environmental protection families
- NIST SP 800-171 — Protecting CUI in Nonfederal Systems and Organizations
- NIST SP 800-88 — Guidelines for Media Sanitization
- DFARS 252.204-7012 — cloud service provider requirements for covered defense information
- FedRAMP authorization boundary guidance
- NIST AI Risk Management Framework (AI 100-1)
FAQ
Common questions
Does using a FedRAMP-authorized provider mean our AI workload is covered?
Only if the specific service is inside the authorization boundary. Providers add services continuously, and newer offerings — accelerator capacity in particular — often sit outside the boundary for a period after launch. Check the service name against the authorization package rather than relying on the provider's overall status.
We use a colocation facility. Whose responsibility is the cooling network?
Theirs to operate, yours to understand. Get the segmentation model for their building management systems and the third-party access arrangements for facilities vendors. If your workload's availability depends on it, it belongs in your risk documentation whether or not you administer it.
Are model weights CUI?
They can be. A model trained or fine-tuned on controlled data may carry that sensitivity in derived form, and the determination follows the same analysis as any other derived output. Treat it as a determination to make deliberately with your contracts function rather than a question to leave open.
Is shared GPU capacity ever acceptable for controlled work?
It depends on the isolation model and whether the provider can describe it credibly. Dedicated capacity removes the question and is worth the cost premium for controlled workloads. If you go shared, document the tenancy model, the memory handling behaviour and who accepted the residual risk.
Our provider says the fabric is "internal" so encryption isn't needed. Is that right?
It's a defensible engineering position and a poor documentation position. Internal is not a control. Record what the fabric carries, what isolates it, why encryption was declined, and who made that call — then it's a risk decision rather than an assumption.
Where does this sit relative to CMMC scoping?
Exactly where every other system sits. If the infrastructure processes, stores or transmits CUI, or provides a security function to the environment, it is in scope and gets categorized like anything else. See the scoping brief for the categorization method.
Buying AI capacity for controlled work?
We review the architecture and the contract together — boundary, tenancy, data lifecycle and disposal — and give you a written determination on whether the capacity can hold CUI before you sign.
Book a scope call. Thirty minutes, no charge.
Related briefs
This brief reflects published guidance as of 29 August 2026. Hardware characteristics and vendor isolation mechanisms change quickly; verify specifics against current vendor documentation and your provider's authorization package before relying on them. Advisory content, not legal advice.
Cyber DSC · Insights · Infrastructure Brief 01 · 29 Aug 2026