Most on-prem AI projects assume a network, just a private one. You pull models from Hugging Face, you reach a license server once a week, you ship telemetry to a vendor dashboard, and you call the result "on-prem." For a lot of regulated work that is fine. For classified programs, operational technology networks, forward-deployed sites, and some healthcare and financial environments, it is not. The system has to run with no outbound path at all, and every assumption your platform team made about the open internet has to be rewritten.
Air-gapped deployments are not just on-prem deployments with the ethernet cable unplugged. The license model changes. The update cadence changes. The telemetry you rely on to catch drift and detect compromise is gone, and the hardware itself needs features most buyers never look at. Here is what actually changes, and what to build for.
What Air-Gapped Actually Means
The term gets stretched. A system behind a firewall with egress rules is not air-gapped. A system on a separate VLAN with a jump host is not air-gapped. NIST describes an air gap as an interface where any logical connection between two systems is not automated, which means a human has to carry data across the boundary. If a service account on the inside network can reach anything on the outside, you have a segmented network, not an air gap.
That distinction matters because it drives every other decision. Container images, model weights, OS patches, CVE feeds, time sync, DNS, and license checks all assume a reachable origin somewhere. Strip that away and each one becomes a scheduled sneakernet operation with chain of custody. The teams that get this right treat the boundary as a product, not an inconvenience, and they design the interior to tolerate long gaps between imports.
For buyers inside regulated frameworks, the vocabulary is specific. Defense primes work against DFARS and NIST SP 800-171 Rev 3, which organizes 97 security requirements into 17 families covering access, media, physical protection, supply chain and more. Federal buyers add FedRAMP High and the controls in SP 800-53. OT operators lean on IEC 62443 zone and conduit models. These frameworks do not require an air gap by name, but their physical and media-handling controls effectively force one for the highest-sensitivity enclaves.
| Family | Rev 2 count | Rev 3 count |
|---|---|---|
| Access Control (AC) | 22 | 20 |
| Awareness and Training (AT) | 3 | 3 |
| Audit and Accountability (AU) | 9 | 8 |
| Configuration Management (CM) | 9 | 10 |
| Identification and Authentication (IA) | 11 | 9 |
| Incident Response (IR) | 3 | 6 |
| Maintenance (MA) | 6 | 6 |
| Media Protection (MP) | 9 | 8 |
How Models and Updates Cross the Boundary
In a connected deployment, pulling a 70B open-weight model is a one-line command. Across an air gap, the same operation becomes a documented transfer with hashes, signatures, scan logs and an approver. Build the import workflow before you buy the hardware, because it shapes everything downstream.
A workable pattern looks like this:
- Staging enclave on the outside. A hardened workstation with internet access that pulls weights, container images, OS packages and CVE feeds from upstream mirrors. Nothing on this box is trusted; it is a quarantine.
- Scan and sign. Everything leaving the staging enclave is virus-scanned, SBOM-checked and signed with keys held on the inside. Model weights get SHA-256 and provenance notes recorded against a ticket.
- One-way transfer. Removable media under two-person control for low volume, or a hardware data diode for higher volume. Diodes use an optical path where the transmit fiber physically cannot receive and the receive side physically cannot transmit, so one-way flow is enforced in hardware rather than by firewall policy.
- Internal mirror. On the inside, a local registry, model store and package repository that the AI servers pull from. Nodes never reach out; they pull from the mirror.
Cadence matters. A monthly import cycle is reasonable for base models and OS patches. Fine-tuning datasets and adapters often move more frequently and need their own smaller lane. Published model weights are large: a 70B model in FP16 is roughly 140 GB on disk, and quantized variants plus adapters quickly turn a quarterly import into a terabyte of media. Size the diode or courier process accordingly.
What You Give Up in Telemetry and Licensing
Vendors have spent a decade building SaaS control planes that assume a callback. Pull the wire and large parts of the modern stack stop working the way the docs describe. Three categories need attention early.
Licensing. Any component that phones home for entitlement is a liability. That includes some GPU driver features, orchestration platforms, observability agents, and most commercial model APIs. Open-weight models solve most of this because the weights are the artifact; nothing calls out. For infrastructure, prefer tooling that supports offline license files with long validity windows, and get the renewal process on paper before you buy.
Telemetry. Vendor dashboards, hosted SIEMs, crash reporters, usage pings and update checks all disappear. You have to replace them inside the enclave. That means a local Prometheus and Grafana stack, a local log aggregator, local model evaluation harnesses, and a local vulnerability feed updated through the import lane. The gap you are closing is detection: without telemetry, you lose the signal that something has drifted or been tampered with.
Model updates and safety. You cannot subscribe to a safety filter that updates nightly. Red teaming, prompt-injection testing and jailbreak evaluation need to run inside the enclave against your actual model build, not against a hosted service. OWASP ranks prompt injection as the top LLM risk for 2025, including inputs imperceptible to humans, which means your evaluation suite has to catch things an outside service would otherwise flag.
The payoff for all this work is that attack surface shrinks hard. IBM's 2025 Cost of a Data Breach report found 13% of surveyed organizations had experienced a breach involving an AI model or application, and 97% of those lacked proper AI access controls. An air gap does not substitute for access control, but it removes the external exfiltration path that most of those incidents relied on.
Hardware and Physical Controls That Differ
An air-gapped build is not a stock server with the NIC disabled. The parts list changes.
- Tamper-evident chassis. Lockable bezels, intrusion-detection switches that log to the BMC, and physical seals on screws so field techs can tell at a glance whether a box has been opened.
- Disabled or removed radios. Wi-Fi and Bluetooth modules physically pulled, not just disabled in firmware. The same goes for cellular management cards in some ruggedized GPU platforms.
- Port control. USB ports epoxied, disabled in BIOS, or routed only to a controlled kiosk. Front-panel serial consoles with physical key locks.
- Verified boot and self-encrypting drives. UEFI Secure Boot with your keys enrolled, measured boot into a TPM 2.0, and SEDs with keys held in an on-prem KMS. Media sanitization procedures need to match the drive type; NVMe SEDs are easier to crypto-erase than classic HDDs.
- Validated cryptography. Where the contract calls for it, modules that have been through FIPS 140-3 validation, which the Cryptographic Module Validation Program made mandatory for all new validations as of September 2021.
- Confidential computing. For multi-tenant enclaves, GPU TEEs matter. Enabling confidential computing on NVIDIA H100 requires a host CPU TEE such as AMD SEV-SNP or Intel TDX alongside supported processors, so the CPU and GPU choices stop being independent.
Facility controls layer on top. Separate power feeds for the enclave, cameras that record to internal storage only, two-person physical access, and a documented media library with check-in and check-out logs. These are the controls that regulated buyers already recognize from classified or payment environments; the AI workload inherits them, it does not get to invent new ones.
Power and cooling do not forgive you for the air gap, either. A single 8-GPU H100-class server draws roughly 10.2 kW at full load, and per-GPU power has climbed from 400W on the A100 to 700W on the H100 to roughly 1,200W on the GB300. Enclaves tend to be smaller rooms with constrained airflow, which makes density planning harder, not easier.
Sizing an Air-Gapped Build
Workload sizing follows the same math as any on-prem LLM deployment, with two adjustments. First, you do not get elastic burst capacity from a cloud provider, so you size for peak concurrent use plus headroom, not average. Second, you cannot patch your way out of a capacity mistake with a vendor's hosted inference endpoint, so the margin on day one has to carry you to the next hardware refresh.
For a typical regulated enclave running a 70B-class model with retrieval and a few agent workflows, a rack-scale system with 4 to 8 current-generation GPUs covers 25 to 50 concurrent users with headroom for batch jobs. Larger programs with document AI pipelines, long-context summarization or multi-tenant agent fleets push into multi-node cluster territory. Smaller forward-deployed sites with a handful of analysts fit on a workstation or ruggedized edge node. The sizing estimator walks the math for a given model, user count and context length.
Model choice narrows in an air gap. Open weights win because the artifact is self-contained: Llama, Qwen, Mistral, DeepSeek, Gemma and gpt-oss can all be imported, hashed and run with no outbound call. Hosted frontier models cannot. If a workflow truly needs a specific hosted model, it belongs in a different enclave, not inside the air gap.
Where This Fits in a Program
Air-gapped systems are not a product line of their own; they are a configuration of the same vendor-neutral GPU platforms used for connected on-prem work, with the import lane, physical controls and offline software stack built in. A sensible program runs a connected development enclave for iteration, mirrors the hardened build into the air-gapped production enclave on a cadence, and tests the import process more often than the model itself changes. The teams that struggle are the ones that treat the air gap as a late-stage hardening step; the teams that succeed design for it from the first week.
If you are scoping a build like this, the practical starting points are the control catalog your auditor cares about, the data classification of what the model will see, and the physical envelope you have to work with. Hardware follows from there. For organizations that want help putting those pieces together, hardware.co runs a two-week discovery sprint and then builds the system; see the development process or request a quote to start the conversation.