The hidden traps of air-gapped AI

Episode summary

A vendor says "air-gapped" and everyone in the room pictures the same thing: a severed cable and a machine in a concrete room. Three quite different deployments get sold under that word, they differ by roughly an order of magnitude in operating cost, and they protect against different things.

This episode walks a technical buyer through what they are actually agreeing to.

The three topologies people conflate

Default-deny egress with an internal mirror is not the same as a physically isolated enclave, and neither is the same as a network that merely has no inbound route. The middle option carries most of the confidentiality benefit of the strictest one at a fraction of the running cost — which is why it is the proportionate answer for most regulated EU workloads, and why writing the strictest option into a security schedule you have not costed is the most common way this goes wrong.

The eleven places a default stack still reaches out

Telemetry, model and tokenizer downloads, package installs at runtime, certificate and revocation checks, time synchronisation, crash reporting, container layer pulls. Each has an opt-out; the point is that they are defaults, so the enclave leaks until every one has been found and turned off, and a pip install inside a container build is enough to break the property you signed for.

The update clock you inherit

The day you unplug, you take ownership of the entire update lifecycle: model weights, base images, OS packages, CVE triage, and the evaluation set that tells you an update did not make the system worse. A signature that was valid when a file was downloaded says nothing about what happened to it on a USB stick.

And what the air gap does not buy

Article 12 logging and NIS2 reporting do not care about topology. Prompt injection still works inside the enclave. And the transfer medium becomes the largest attack surface you have — which is where the episode ends, on the observation that closing the door to the outside makes the person carrying the drive the most powerful actor in the architecture, operating with the fewest technical controls in the way.

The written version, with the topology table and the full outbound list, is in air-gapped is an operating commitment, not a network setting. For where the model runs when full isolation is not proportionate, see private LLM inference and EU self-hosting.

Full transcript

You're sitting in a meeting, right? And a vendor has just confidently promised you an air-gapped AI system. Sounds incredibly secure. Oh, it sounds amazing. You probably pictured like a server humming away in some concrete bunker. Right, yeah. The ethernet cables are physically severed and the whole operation is completely impenetrable to the outside world. And you sign the contract feeling pretty good about your security posture. But, you know, what did you actually just agree to?

Because when you take that buzzword out of the boardroom and hand it to a team of engineers, the reality of what it means to unplug a large language model from the world is, well, it's staggering. It really is. I mean, it's a massive architectural shock to the system because air-gapped is a word that feels absolute. You know, it sounds like a simple on-off switch. Just pull the plug.

Exactly. Yeah. But in practice, taking a modern AI stack offline uncovers this absolute mountain of operational debt. You're taking on three massive commissions. Okay, what are they? Well, first, you're committing to enumerating and closing every hidden outbound pass in software you didn't even write. Second, mathematically proving that closure. And third, taking total manual ownership of the system's entire update lifecycle. Okay, let's unpack this because before we even look at how the software, you know, fights back, we have to look at the contract itself.

It feels like a lot of these deployments crash and burn at the very beginning just because the technical buyer and the vendor are using the exact same word to describe entirely different physical realities. Yeah, the vocabulary mismatch is really where the trap is set. In the industry, we see three distinct network topologies that all get sort of lazily conflated under the term air-gapped. And the differences aren't minor, right?

Oh, not at all. The cost differences between them differ by orders of magnitude. Mm-hmm. So the most common setup, and by far the cheapest, is egress control. Egress control, okay. Right. This is just a standard network environment, but with a strict default deny firewall policy for outbound traffic. You have this very short, highly monitored allowance that lets the system talk to a specific internal mirror just to get its updates.

So it's not actually physically severed. It's just heavily restricted, like, I don't know, putting a really good lock on the front door of an office building. Yeah, exactly. The infrastructure is totally normal, but the policy is tight. Then the second topology is what we call disconnected, or no internet. Okay, how is that different? Here, the specific enclave where your AI lives has absolutely zero routing paths to the outside internet.

None. However, it still has internal network reachability to other parts of your corporate network. Oh, I see. Yeah, so updates arrive over a controlled internal channel from another secure server. Mm-hmm. But the cost jumps to medium here because you now have to fully build and own a network. Mm-hmm. So you don't have to own that internal artifact pipeline. Right, you're managing the delivery yourself. And then we get to the third one, the concrete bunker.

The true physical air gap. There is no network path whatsoever. Not to the internet, not to the corporate network, literally nowhere. Yeah, every single update, every patch, every new model wait, it's a massive logistical event. It requires moving data onto removable media, walking it through physical security checkpoints, scanning it, and manually mounting it. So the cost there. The cost there is just. Astronomically high. The friction is incredible.

It's like asking a contractor to build you a secure vault, but nobody actually specifies the threat model. Are we talking about like a heavy-duty locked filing cabinet or a bank vault in the city center? Or are we burying the data on a deserted island? That's a great way to look at it. Because you have to know which one is going into the security schedule. If you pay for the deserted island and they build you the filing cabinet, your compliance auditors are going to shut you down.

Which happens. It happens constantly, actually. Almost every organization that casually asks for air-gapped AI really just needs the first or second option for their regulatory framework. Just the egress control or the internal network. Right. But if a buyer forces the phrase physical air gap into a minding legal contract, well, the engineering team eventually realizes they can't afford the operational overhead of a disconnected basement server. So they just build the first option anyway.

Oh, wow. So they just ignore the contract? Basically. You end up legally bound. To a physical reality you aren't practicing. That is a massive compliance disaster from day one. But, okay, let's assume for the sake of the exercise that a team actually does it. They pick the strict disconnected enclave. They pull the literal plug on the internet route. The AI is completely isolated. We should be safe now, right?

But the software ecosystem wasn't built for this, was it? Not even a little bit. Yeah. The default AI inference stack operates on this deep, unspoken assumption. That it has infinite unhindered. Unhindered access to the internet. Always online. Always. When you put in a disconnected enclave, the software doesn't just sit there quietly. It panics. It tries to phone home constantly through all these hidden dependencies you might not even realize are running.

Yeah. Our sources for this deep dive actually highlight 11 different ways a standard AI stack tries to secretly reach out. And I don't want to just rattle these off. I really want to look at the mechanics of why this happens. Let's start with the stuff that feels intentional. Like the software actively trying to report on us. Yeah. Telemetry and licensing are huge culprits here. Take serving run times like VLLM.

They're incredibly popular. Yeah. But by default, if you use their CLI or their Docker images, it collects usage statistics and tries to report them on a 10-minute timer. Every 10 minutes. Every 10 minutes. And then you have your commercial components, like maybe a vector database you're using for a retrieval augmented generation or your GPU vendor tooling. Right. They carry their own telemetry and license entitlement checks. Yeah.

And you physically need to phone home to verify you actually paid for them. I mean, even dashboarding tools, right? Like Grafana. Exactly. Grafana is quietly pinging out just to check if a new version is available. That feels like the noisy neighbor category. It's annoying. Maybe a bit of a privacy concern. But if the network is completely unplugged, those pings just drop into the void. What about the stuff that actually breaks the system if it can't connect?

Ah. Well, that brings us to the silent crashers. The classic example is... Model and tokenizer resolution. Okay. How does that work? So, when you load a model using Hugging Face Libraries, which is basically the industry standard, the library's default behavior is to take the repository ID and resolve it against the web hub on the very first call. Wait, really? Even if you already downloaded the model weights to your local hard drive?

Even then. If you don't explicitly force the code into offline mode, it might look for, say, a missing configuration file or just try to check for a newer version. And because it has no interface? And it's not even on the internet. The request just hangs indefinitely. Your multi-million dollar isolated AI system just freezes on the very first user prompt, waiting for a web server it can never reach.

Oh man, that's brutal. And it gets deeper than just software libraries, right? There's a massive risk around basic infrastructure protocols. Like, what happens with timekeeping? If we isolate a server, it obviously can't reach an NTP, a network time protocol server on the internet. Which is a catastrophic failure-hode that most teams just do not see coming. If your enclave lacks an authoritative internal time source, the server's hardware clock slowly drifts.

And for anyone wondering why a few minutes of clock drift actually matters, it absolutely breaks modern cryptography, doesn't it? It breaks it entirely. TLS certificates, which are the standard for secure communication, have strict valid from and valid to dates. Right. If your isolated server's clock drifts by a few days, or if it resets, it looks at a perfectly valid security certificate and thinks, oh, this is expired.

Oh, wow. Yeah. Suddenly all internal secure connections, like between your database and your AI model, just fail. The logs will scream about cryptographic errors, but the real root cause is literally just a broken clock. That is wild. And it's a similar story with the certificates themselves. The TLS layer natively tries to make outbound calls to check if a certificate has been revoked. It uses OCSP or CRL fetches.

And if it can't reach the certificate authority? Either adds massive latency to every single request. Or it just completely refuses to connect. Okay. So the infrastructure itself is actively fighting the isolation. But let's talk about the really scary stuff. The covert exfiltration channels. Because blocking web traffic on port 443 isn't enough to stop data from leaking, especially when it comes to DNS. Yeah. DNS is the ultimate blind spot.

Most teams think, well, we blocked all web traffic, so the data's trapped. Yeah. Yes. The system that translates a web address into an IP address is often left open so internal services can find each other. Sure. That makes sense. The flaw is that a DNS resolver might be configured to forward unknown queries out to the internet. So I want to make this mechanism really clear for you listening.

It's like a maximum security prison, right? Where the guards block all the letters going out. But they still let prisoners ask the directory operator for phone numbers. Right. If a bad actor compromises the AI, they don't send the stolen data in the letter. They literally encode the stolen data into the name they are looking for. That is a perfect way to visualize it. The compromised system just asks the network, hey, what is the IP address for my secretpassword.evilserver.dat.com?

And the firewall just lets it through. Exactly. The firewall sees a standard DNS lookup and lets it through. The evil server gets the request, ignores the IP address part entirely, and just logs the stolen password. The data is gone, and your firewall didn't even blink. That is terrifying. And speaking of leaking data, what about crime? What about crash reporting tools? Our sources mention SDKs like Sentry. Oh, developers love adding automated error-tracking libraries like Sentry to their code bases.

You add it once and just forget about it. When the application crashes, these tools automatically bundle up a stack trace and transmit it to a third-party cloud server so the developers can debug it. Wait. A stack trace captures the state of the system's memory at the exact moment of the crash. So if an AI inference service crashes while processing a highly confidential user prompt… The stack trace likely contains fragments of that confidential prompt in plain text.

You're kidding. No. You built this incredible air-gapped vault to protect your most sensitive corporate data. The application has a minor memory error, and it just helpfully emails the confidential data to a third-party logging server. It's like the system is actively conspiring against its own security model. And we haven't even touched on things like container-based images. If a script runs a pip install or tries to pull a Docker layer dynamically, it's just going to fail in the dark.

Because that is an internet dependency wearing a Docker file. Oh. In a disconnected environment, everything has to be fully baked and staged beforehand. Right. And honestly, the absolute worst offender, the thing that intentionally opens the door, is an AI agent with an HTTP tool. If you are building an autonomous agent and you give it a tool that can make web requests and you leave that allow as permissive… Oh, no.

Yeah. It can completely bypass your network policies by design. But hold on. I need to push back on this a little bit. Looking at the documentation for a lot of these libraries and telemetry tools, they almost always have opt-out variables. I've seen donutrack equals one or offline mode equals true in half these repositories. Why are we treating this like a massive architectural hurdle? Can't we just flip those environment variables to one and be done with it?

This raises an important question because you are relying on vendor courtesy instead of a structural control. What do you mean? An environment variable that asks a software library nicely not to spy on you is not a security policy. Fair point. Its defense in depth, sure, should absolutely set those flags. But the one telemetry ping you don't know about is the one that doesn't have an opt-out variable documented.

The engineer who wrote it might have just hard-coded the ping. If you rely on software configurations to prevent egress, you are literally one typo away from an open connection. Okay, that makes total sense. We can't trust the software's intent. So how do we actually prove to a compliance auditor that the system is locked down? Because anyone can claim they air-gapped a system. But an egress guarantee that no test can falsify is just a corporate fairy tale.

You have to move from intent to proof. And that requires three specific engineering artifacts. Okay, let's hear them. First, you need a default deny egress policy placed at the network boundary. Hmm. This must be expressed somewhere. The AI workload itself absolutely cannot edit it. Like where? Like a Kubernetes network policy or a VPC configuration with no internet gateway. The workload cannot be in charge of its own cage.

The prisoner doesn't hold the keys to the cell. Precisely. Second, you need a build time check. Set up a continuous integration pipeline that actively scans your dependencies and your configurations. If an engineer adds a new library that suddenly tries to introduce an outbound destination, the CI job automatically fails the build. So it stops the regression from ever-reaching production. Exactly. And the third artifact. Runtime observation. Auditors do not care about your configuration files.

They want flow logs. The actual network traffic. Right. They want a historical record of what the workload actually attempted to do over, say, a 30-day window. They want to see every attempted connection, the destination IP, and the firewall explicitly dropping the packet. Here's where it gets really interesting. A configuration file just says what you intended to do, right? It's the blueprint of the prison. But the flow logs are the security camera footage.

The logs show the workload actually walking up and jiggling the door handle and the door refusing to open. That's spot on. And the denied connection counts in those logs are your best early warning signal. Because they show a change in behavior. Exactly. If you upgrade a Python library and suddenly see a massive spike in dropped outbound connections, it means that dependency changed its behavior and started searching for the Internet.

The camera footage proves the blueprint actually worked. So the vault is mathematically proven to be locked. We've got the blueprint, the camera footage, the CI checks. But what happens to everything inside the vault? Because the exact moment you achieve perfect isolation, you inherit what our sources call the UpGate clock. And this is the operational reality that bankrupts projects. Why is it so bad? Because when you are connected to the Internet, your system gets better and better.

Your system gets better silently. The models update, the security patches download, the vulnerabilities basically fix themselves in the background. The day you sever that connection, you take personal ownership of everything that used to arrive for free. You costed the project at zero maintenance, but the maintenance becomes the entire job. Totally. Let's start with the AI model itself. In an isolated system, the model ages silently. It is frozen in time at the exact moment of import.

So when a new model comes out. When a vastly superior model is released. Three months later. Getting it into the enclave is a massive scheduled project. You can't just roll it forward and observe how users react. You have to mathematically prove the new weights are better before they ever cross the physical boundary. Which means building your own evaluations. Yes. Building and maintaining your own custom evaluation dataset.

Which is a massive engineering effort on its own. And it's not just the models, is it? It's the CVE clock. The flow of common vulnerabilities and exposures. The hackers don't stop finding security flaws just because you unplugged your router. Right. The vulnerability clock just moves from the internet to your desk. Your Python dependencies. Your base image. The GPU drivers. The serving runtime. They all accumulate critical vulnerabilities.

So how do regulated industries handle that? Regulated operators are forced to run two distinct update cadences. They need a scheduled massive bundle. Maybe quarterly. And a high-stress out-of-band emergency path for severe threats. And building that bundle is incredibly complex, right? You aren't just dragging a simple text file across a border. Oh, the bundle complexity is staggering. You have to bring over the new model weights. But you also need the exact tokenizer version that matches those weights.

You need the serving runtime image. You need the vector store database binaries. What happens if you miss one? Well, if you update the vector database software, but you forget to update the index format version, you will successfully import everything, reboot the system, Yeah. so that literally none of your data collections will open. The entire RA pipeline just collapses because the formats mismatch. Wait, I want to zero in on how we actually get these complex bundles into the vault.

If we are doing a true physical air gap, the deserted island topology, that means the transfer medium is literally a physical USB stick. Or a removable hard drive. Yes. And for anyone who hasn't been burned by this, a lot of these AI models are distributed as PyTorch checkpoints. So? You can put them in files that deserialize using a Python module called Pickle. And, I mean, a Python Pickle isn't just a spreadsheet of numbers.

It's literally a set of instructions on how to rebuild data and memory. Right. If one of those instructions says, run this malicious script, the system just blindly obeys. It's a known arbitrary code execution path. You've hit on the core irony of the entire architecture right there. It's crazy. We cut off the internet to be secure. But now we're putting an executable set of instructions onto a USB stick, handing it to some junior engineers.

Having them walk it past three security desks. And plugging it directly into the core AI server. Our air gap just made our transfer medium the biggest attack surface in the company. Because you've perfectly removed the network controls. Which means the supply chain now runs through a physical channel that has far fewer inherent safeguards than a basic TLS encrypted web connection. So what's the fix? This is why you absolutely must demand safer formats like Safeton Torners instead of Pickle-based checkpoints.

Safeton Torners only store data, not executable code. That helps. You also have to cryptographically sign these bundles and verify those signatures on the inside of the enclave. You pin the weights by their cryptographic hash. Because the excuse of we downloaded it from a reputable website yesterday is completely meaningless once that file has traveled in a thumb drive in someone's pocket for 24 hours. It is wild how much risk this introduces.

With all this logistical overhead, the massive expense, the custom evaluation sets. You'd think an air gap is the ultimate impenetrable compliance shield. Like it just solves all your legal problems. But our sources are very clear that this is a myth. It is arguably the most dangerous myth in enterprise AI right now. We need to state this unequivocally. Air gapping is an architectural choice. It is not a compliance conclusion.

So what does the air gap absolutely not buy you? First, it does not magically waive your regulatory obligations. Look at the EU's AI Act. Article 12 requires extensive logging for high-risk systems. Right. The law does not care if your server is in the cloud or isolated in a basement. The logging requirement is identical. NIS2 incident reporting duties apply just as strictly. GDPR obligations follow the personal data, not the network topology.

If anything, being air gapped makes regulatory compliance harder. Because the audit logs the government actually wants to see are trapped inside the vault with the AI. You have to engineer a highly secure, redacted way to get those logs out long before an auditor ever comes into office. Exactly. Second, an air gap does not stop prompt injection attacks. Wait, really? But if the AI can't talk to the internet, a prompt injection attack can't trick it into leaking data to a hacker's server, right?

True. It frustrates exfiltration to the internet. Yeah. But it just changes the payoff for the attacker. It does absolutely nothing to prevent an attacker from using prompt injection to exfiltrate data to another tenant, another user, or another document existing within that same isolated environment. Oh, I see. If an attacker poisons an internal document, and the CEO queries the AI about it, the AI can still be manipulated into surfacing the CEO's private data to a lower-level internal database.

Huh. Isolation inside the boundary remains a totally separate, very difficult engineering problem. Wow. Okay, what else doesn't it fix? Well, it doesn't solve data residency. If you build an isolated enclave in a geographic jurisdiction where your data isn't legally allowed to be, it's still breaking the law. Unplugging the router doesn't change geography. Exactly. Okay. And finally, it does not make the AI models inherently trustworthy. You still have to trust the entity that trained those weights on the outside.

You have to verify the provenance in the license terms. The air gap only guarantees that if you accidentally import a maliciously trained model, it stays put in your basement. So what does this all actually mean? The ultimate failure mode here isn't necessarily picking the wrong piece of technology. It's choosing air gapped as a buzzword because it sounds safe to the board. Right. So a company implements the low-cost egress-controlled network, but they contractually commit to a physical air gap.

They don't budget for the evaluation sets, the artifact pipelines, or the manual CVE patching. And then what happens? Eighteen months later, they are stuck with a frozen, degrading stack full of unpatched vulnerabilities, failing their compliance audits. It is actually less secure than if they had just left it connected to a well-managed cloud environment. That is the trap. For the vast majority of regulated workloads, the truly proportionate answer is that first topology.

Default-denied egress, an internal mirror, and isolation properties that are actively proven with flow logs rather than just legally asserted. A smart middle ground. Right. It gives you the core confidentiality benefits at a fraction of the operating cost. And most importantly, it keeps the update clock from completely crushing your engineering team. This has been incredibly eye-opening. Thank you for walking us through the concrete realities of these deployments.

I think the takeaway for you listening is crystal clear. True security requires active, ongoing engineering and maintenance. It is never as simple as just unplugging a cable and walking away. I completely agree. And I want to leave you with one final thought to ponder. Looking at this strictly from a threat modeling perspective, we spent this whole time exploring how an air gap perfectly removes external network controls.

But consider the consequence of that isolation. By forcing all updates to happen manually, you have now elevated the insider, the person holding that physical USB stick, to the most powerful actor in your entire architecture. Wow. They are now operating with the fewest technical controls in their way. When you permanently close the door to the outside world to keep the monsters out, you have to ask yourself, who exactly are you locking inside?