The Air-Gapped GenAI Architecture: How Self-Hosted LLMs Secure Industrial IP and Compliance

In many industrial and manufacturing companies, there is growing pressure to use generative AI for automated error reports, maintenance logs, and root cause analysis. However, the reality in OT and IT practice is sobering: sending proprietary sensor data, machine telemetry, and process know-how through public hyperscaler APIs to US data centers risks uncontrolled leakage of sensitive intellectual property and blatant compliance violations.

The strategic response to this hurdle is not to abandon GenAI, but to completely decouple the platform from external interfaces. By operating self-hosted open-weights models declaratively on a hardened, Kubernetes -based infrastructure, ayedo enables the productive use of modern language models, fully isolated within its own European security perimeter.

1. The Problem: The Risks of External LLM APIs in Core Industry

Using public cloud APIs for industrial GenAI use cases creates significant operational, legal, and financial risks:

• 1. Loss of Data Sovereignty: Sensor data streams and error logs contain detailed insights into manufacturing tolerances, plant efficiency, and production volumes. When transmitted to external SaaS APIs, third-party usage terms apply, causing confidential OT telemetry to leave the protected corporate perimeter.
• 2. Unpredictable Cost Explosion: Token-based billing models are hardly budgetable for continuous industrial data streams. Unforeseen load spikes and rising egress costs for data transfers lead to massive budget overruns.
• 3. The Latent Vendor Lock-In Trap: Proprietary endpoints change their model versions, latencies, and pricing structures without notice. If an external API fails or the internet connection at the plant site breaks, automated reporting processes in production come to a halt.

2. The Solution: The Sovereign GenAI Stack Architecture

ayedo establishes a native, multi-layered LLM-serving architecture based on Kubernetes , providing highly optimized inference engines like vLLM for production and Ollama for controlled experiments—completely isolated through Authentik and network policies.

+——————————————————————————-+ | Kubernetes Cluster Perimeter (On-Premises / European Sovereign Cloud) | | | | +————————————————————————-+ | | | Authentik Identity Layer (OIDC / Role-Based Access Control) | | | +————————————+————————————+ | | | | | v | | +————————————————————————-+ | | | Network Policy & Isolation Layer (Calico / Cilium Zero-Egress) | | | +————————————+————————————+ | | | | | +—————————–+—————————–+ | | | (Production Workloads) | (Dev) | | v v | | +———————————-+ +————————+ | | | vLLM Serving Engine | | Ollama Dev Sandbox | | | | - Open-Weights (Llama 3 / Mistral| | - Rapid Prototyping | | | | - PagedAttention VRAM Mgmt | | - Isolated Ephemeral | | | | - Continuous Batching | | Workspaces | | | +——————+—————+ +————————+ | | | | | v | | +————————————————————————-+ | | | Storage Layer: S3-compatible Model Registry (Ceph / MinIO) | | | | - Signed Open-Weights & Quantized GGUF/AWQ Models | | | +————————————————————————-+ | +——————————————————————————-+

• 1. The Segregated Inference Structure: High-performance production queries (e.g., root cause analysis in seconds) run through vLLM with PagedAttention, maximizing graphics memory utilization. Developer and data science teams simultaneously access isolated Ollama environments to test new prompts without blocking production resources.
• 2. The Declarative Zero-Egress Hardening: With Kubernetes network policies, the LLM namespace is strictly isolated. Any outgoing traffic to the public internet is blocked at the kernel level. Data remains exclusively within the local cluster network.
• 3. Unified Governance via Authentik: Access to LLM endpoints is controlled via central OIDC tokens. Fine-tuned role-based access control (RBAC) ensures that developers, departments, and automated services can only access the models they are authorized for.

3. Strategic and Economic Value

Operating an autonomous, Kubernetes -based LLM infrastructure transforms incalculable AI experiments into a plannable, compliant corporate asset:

• 100% GDPR, NIS-2, and BSI C5 Compliance: Since no data leaves the European sovereign area or the company’s own data center, regulatory requirements for Critical Infrastructures (KRITIS) and sensitive manufacturing data are fully met.
• Radical Reduction of Total Cost of Ownership (TCO): Fixed hardware or bare-metal costs replace variable token bills. With high inference volume, the own platform typically amortizes within a few months.
• Complete Independence from Third Parties: Companies retain full control over model weights, system prompts, and versioning cycles—without the risk of sudden interface shutdowns.
• Protection of Intellectual Property: Internal process knowledge never flows into the retraining of external models but remains 100% owned by the organization.

Conclusion

True technological innovation in the industry requires no compromises on data protection. By combining modern open-weights models, vLLM, and a managed Kubernetes infrastructure, ayedo demonstrates that high-performance generative AI can be operated sovereignly, cost-effectively, and fully auditable within its own machine room.

FAQ: Practical Questions about Self-Hosted LLMs

Can modern open-weights models compete with proprietary hyperscaler models?

For specialized industrial and maintenance use cases such as log analysis, error classification, and report generation, high-performance open-weights models (like Llama-3 or Mistral variants) achieve comparable or superior precision through targeted fine-tuning and retrieval-augmented generation (RAG)—at a fraction of the operating costs and with deterministic latency.

What are the minimum hardware requirements for operating vLLM?

For quantized models (e.g., 8-bit or 4-bit AWQ with 8B to 14B parameters), a single enterprise GPU with 24 GB VRAM (such as an NVIDIA A10G or L4) is often sufficient. For larger 70B models, ayedo relies on node clusters with tensor parallelism across multiple A100/H100 accelerators, coupled via NVLink.

How is model loading solved without an internet connection (Air-Gapped)?

Model artifacts and checkpoints are pre-verified, cryptographically signed, and stored in a cluster-internal, S3-compatible object storage instance (e.g., MinIO or Ceph). The serving pods load the weights directly over the internal high-speed network, eliminating the need for any external connection to platforms like Hugging Face at runtime.