Air-gapped AI: what zero egress actually costs, and what it does not buy

What does an air-gapped LLM deployment actually demand?

Three things, and only the first one is a network change. You have to enumerate and close every outbound path in a stack that assumes internet access at a dozen layers you did not write. You have to prove the closure with something that can fail, because a claim about egress is worth exactly as much as the test behind it. And you have to take ownership of an update clock — model, runtime, GPU driver, base image, CVE backlog — that a connected deployment gets for free and an isolated one does not.

Teams underestimate the third one. Air-gapping is usually presented as a security win, and at the confidentiality layer it is. What it really does is convert a confidentiality risk you cannot control into an availability and currency risk you can — and someone has to actually do the controlling, every month, for as long as the system runs.

This is engineering and practitioner guidance grounded in production delivery of retrieval and inference systems. It is not legal advice and it is not a compliance guarantee. Whether an air gap satisfies a specific regulatory or contractual requirement is a question for your counsel and your auditor, not for a vendor.

First: three different things are called "air-gapped"

Most disagreements about air-gapped AI are vocabulary problems that surface late, usually in a contract. There are three distinct topologies and they differ by an order of magnitude in cost:

Almost every organisation that opens a conversation asking for "air-gapped AI" is describing the first or second row, and a good number of them will be well served by the first. The word matters because it is the one that ends up in a security schedule, and the middle row is where most regulated deployments genuinely land: no internet route, an internal mirror, a signed import path.

If you write "air-gapped" into a contract and operate the first row, you have a finding waiting for you. Write down which row you mean.

The stack phones home more than you think

A default inference stack reaches outbound in more places than the architecture diagram shows, and the dangerous ones are those that fail silently or that succeed in staging because staging had a route. In rough order of how often we see them missed:

1. Serving-runtime usage statistics. vLLM collects usage stats by default for CLI and Docker usage and reports on a ten-minute timer. Opt out with VLLM_NO_USAGE_STATS=1, VLLM_DO_NOT_TRACK=1 or the cross-vendor DO_NOT_TRACK=1, or by creating ~/.config/vllm/do_not_track.
2. Model and tokenizer resolution at first call. Hugging Face libraries resolve a repo id against the Hub unless you tell them not to. HF_HUB_OFFLINE=1 makes every HTTP call raise instead of hanging, which is the behaviour you want in an enclave; HF_HUB_DISABLE_TELEMETRY=1 covers the usage pings that transformers, datasets and gradio route through huggingface_hub. Offline mode disables telemetry as a side effect. The failure this prevents is specific and common: weights are staged locally, but a tokenizer or a config file was never in the bundle, and the first request in production discovers it.
3. DNS. The most under-rated one. Blocking outbound 443 while leaving a resolver that forwards to the internet means query names still leave the enclave — and DNS is the classic covert exfiltration channel precisely because it survives naive egress rules. An enclave needs an authoritative internal resolver that does not forward.
4. NTP. Not an egress risk so much as a failure mode: clock drift breaks TLS validation and token expiry, and an isolated enclave with no internal time source drifts. Give it a stratum source inside the boundary.
5. Certificate revocation and CA bundle updates. OCSP and CRL fetches are outbound calls made by your TLS layer, not by your code, and depending on the client they either add latency or fail closed. Decide deliberately: internal PKI, stapling, or documented soft-fail.
6. Container base images and package installs at start-up. An entrypoint that runs pip install or pulls a base layer at deploy time is an internet dependency wearing a Dockerfile. Everything must be baked and staged.
7. Crash and error reporting SDKs. Sentry-class libraries are added once and forgotten, and they transmit stack traces — which in an inference service can contain prompt fragments.
8. Licence and entitlement checks in commercial components, including some observability and vector-database editions.
9. GPU driver and vendor tooling, which carry their own update checks and telemetry.
10. Dashboard update checkers — Grafana and friends ping for new versions by default.
11. Anything an agent can reach. If the system has tools, the tool layer is an egress surface by design. An HTTP tool with a permissive allowlist undoes the network policy above it, and prompt injection is the mechanism that will find it.

The opt-out environment variables in that list are a courtesy from good vendors, not a control. Treat them as defence in depth behind a default-deny egress policy — never as the policy itself. The one you did not know about is the one with no variable.

Prove it, don't declare it

An egress guarantee that no test can falsify is a belief. Three artefacts turn it into a property, and an auditor will ask for the third:

A default-deny egress policy at the boundary, expressed where it cannot be edited by the workload — a Kubernetes NetworkPolicy with an empty egress allowlist, a firewall rule set, a VPC with no route. Deny by default, allow by exception, and keep the exception list in review.

A build-time check that fails on new reach. A CI job that diffs the dependency and configuration surface and fails the build when a new outbound destination appears is the cheapest of the three and the one that catches regressions months later, when the person who set up the enclave has moved on.

A runtime observation, not a configuration file. The evidence that satisfies a reviewer is a record of what the workload attempted: flow logs or a capture from the boundary over a representative window, showing every attempted connection and its verdict. Configuration says what you intended. Flow logs say what happened. Denied-connection counts are also the best early-warning signal you get — a new denial after an upgrade means a dependency changed its mind about needing the internet.

This is the same discipline as the tenant-isolation test in our post on multi-tenant RAG isolation: assert the negative deliberately, and make the assertion something CI can run.

The clock you inherit

The day the enclave closes, you take ownership of everything that used to arrive on its own. This is the part that gets costed at zero and then dominates the running cost.

The model ages, and nothing tells you. Connected deployments get better silently; an isolated one is frozen at import. Model refresh becomes a scheduled project with an evaluation gate, because you cannot roll forward and observe — you have to know the new weights are better before they cross the boundary, on your own evaluation set. Budget for the eval set as an asset, not an afterthought.

The CVE clock does not stop. It moves to your desk. An isolated stack still accumulates advisories in the serving runtime, the Python dependency tree, the base image, the vector store and the driver. Two cadences are what regulated operators in practice run: a scheduled bundle (monthly or quarterly), and an out-of-band emergency path for anything the security team classifies as urgent. Define both up front, including who decides that a CVE qualifies — that decision right belongs to the customer's security team, and writing it down prevents the argument happening during an incident.

A bundle is more than the weights. A reproducible import is: model weights pinned by hash, tokenizer and config, the serving-runtime image, the full dependency closure, the vector-store binary and its index format version, the GPU driver and CUDA build, and the evaluation results that justified the change. Miss the index format and you discover on restore that the collections do not open.

The transfer medium becomes your largest attack surface. This follows directly, and it is the irony of the design: you removed the network, so the supply chain now runs through a channel with fewer controls than the network had. Sign bundles and verify signatures on the inside, not the outside. Pin weights by digest. Scan on import. And prefer safetensors over pickle-based checkpoints — loading a .bin PyTorch checkpoint deserialises a pickle, which is a code-execution path, and "we downloaded it from a reputable place" is not a control when the file has crossed on a USB stick through three hands.

What the air gap does not buy you

It is worth being blunt here, because air-gapping is often sold as a compliance conclusion rather than an architectural choice.

It does not remove regulatory obligations. The AI Act's Article 12 logging requirement for high-risk systems does not distinguish between connected and isolated deployments, and NIS2 incident-reporting duties apply to a system in a control room exactly as they do to one in a data centre. GDPR obligations follow the personal data, not the network topology. If anything, isolation makes the evidence harder to produce, because the logs are inside too — plan how they leave, under what redaction, before you need them. We wrote about which obligations are actually live now in what applies from 2 August 2026.

It does not stop prompt injection. It changes the payoff, not the vulnerability. An air gap frustrates exfiltration to the internet; it does nothing about exfiltration to another tenant, another user, or another document in the same enclave — which is why isolation inside the boundary stays a separate engineering problem with its own tests.

It does not address the insider, who is now the actor with the most access and the fewest network controls in their way.

It does not make the model weights trustworthy. You are still trusting whoever trained them. Provenance, licence terms and hash pinning are the same work they were before, and an air gap only guarantees that a bad artefact stays put once you have imported it.

And it does not, by itself, satisfy data-residency requirements. An enclave in the wrong jurisdiction is still in the wrong jurisdiction.

When it is the right answer

Air-gapping is proportionate when the requirement is external and specific: classified or defence workloads, OT and ICS environments where the network boundary already exists for safety reasons, and contractual terms that name the topology. It is also the right call when the honest answer to "what happens if this data reaches the internet once" is unacceptable at any probability.

For most regulated EU workloads, the proportionate answer is one row up the table: default-deny egress, an internal mirror, EU-resident inference under your control, with the isolation properties tested rather than asserted. That gets the large majority of the confidentiality benefit at a fraction of the operating cost, and it keeps the update clock manageable. The decision framework for that choice — self-host, EU-region API, or US-hosted, routed per workload rather than picked once — is in our post on private LLM inference and EU self-hosting.

The failure mode we see most often is not a team that chose wrong. It is a team that chose "air-gapped" as a word, implemented row one, wrote row three into the security schedule, and never built the artefact pipeline — so eighteen months later the enclave runs a frozen stack with an unreviewed CVE backlog, which is a worse security position than the connected deployment it replaced.

If you are working out which row you actually need, or costing the pipeline behind the one you have already promised, that is a short conversation to start. Our security and compliance pages set out how we approach the rest of it.