Busk: Created page with "One of the most popular and powerful use cases for KubeVirt, often referred to as Kubernetes-in-Kubernetes or Tenant Clusters. In this architecture, your underlying b..."

2026-08-02T15:11:02Z

Created page with "One of the most popular and powerful use cases for KubeVirt, often referred to as **Kubernetes-in-Kubernetes** or **Tenant Clusters**. In this architecture, your underlying b..."

New page

One of the most popular and powerful use cases for KubeVirt, often referred to as **Kubernetes-in-Kubernetes** or **Tenant Clusters**.

In this architecture, your underlying bare-metal Kubernetes cluster acts as the **infrastructure management layer** (the control plane for hardware), and KubeVirt provisions VMs that join together to form separate, isolated **tenant Kubernetes clusters**.

---

## Why Do This? (Key Benefits)

* **Multi-Tenancy & Hard Isolation:** Containers share the host kernel. If you need strict security boundaries between teams or environments (e.g., Prod vs. Dev), VMs provide hardware-level virtualization isolation.
* **Infrastructure-as-Code (GitOps):** You can manage the lifecycle of entire target Kubernetes clusters declaratively using standard `kubectl` commands or GitOps pipelines (ArgoCD/Flux).
* **Cluster API (CAPI) Integration:** Using the **Cluster API Provider KubeVirt (CAPK)**, you can declare a target Kubernetes cluster in YAML, and CAPI + KubeVirt will automatically spin up the VMs, run `cloud-init` to install Kubernetes on them, and join them into a fully functional tenant cluster.
* **Testing & Sandbox Environments:** It allows developers to spin up ephemeral, full-featured Kubernetes clusters (with custom kernel modules or CNI configurations) without needing real physical hardware.

---

## How the Architecture Looks

```
┌────────────────────────────────────────────────────────────────────────┐
│ BARE-METAL KUBERNETES CLUSTER │
│ (Management Cluster: Physical Servers + Storage + KubeVirt Operator) │
│ │
│ ┌──────────────────────────────────────────────────────────────────┐ │
│ │ TENANT KUBERNETES CLUSTER │ │
│ │ │ │
│ │ ┌────────────────────┐ ┌────────────────────┐ │ │
│ │ │ KubeVirt VM 1 │ │ KubeVirt VM 2 │ │ │
│ │ │ (Tenant Control) │ │ (Tenant Worker) │ │ │
│ │ │ │ │ │ │ │
│ │ │ ┌────────────────┐ │ Network │ ┌────────────────┐ │ │ │
│ │ │ │ Tenant K8s API │ ◄────────────────►│ Tenant Pods │ │ │ │
│ │ │ └────────────────┘ │ (Overlay/VLAN) │ └────────────────┘ │ │ │
│ │ └────────────────────┘ └────────────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────┘ │
└────────────────────────────────────────────────────────────────────────┘

```

---

## Typical Workflow

1. **Deploy Bare-Metal K8s:** You set up physical servers with a Kubernetes distribution (e.g., Talos, RKE2, OpenShift, or upstream K8s).
2. **Install KubeVirt:** Deploy the KubeVirt operator on the bare-metal cluster.
3. **Use Cluster API (CAPK):** Apply a manifest defining a tenant cluster (e.g., 1 Control Plane VM, 3 Worker VMs).
4. **Bootstrapping via cloud-init:** KubeVirt boots the VMs, and `cloud-init` runs scripts (like `kubeadm join` or `k3s`) to wire the nodes into the new nested cluster.

---

## Things to Consider

* **Nested Virtualization / Overhead:** Ensure hardware-assisted virtualization (`/dev/kvm`) is available on your bare-metal nodes so KubeVirt runs near native speed.
* **Networking:** You’ll need a CNI or plugin like **Multus** or **OVN-Kubernetes** on the bare-metal cluster to assign dedicated IPs or VLANs to the VM nodes so the tenant cluster can route traffic cleanly.

Kube-virt - k8s in k8s - Revision history

Busk: Created page with "One of the most popular and powerful use cases for KubeVirt, often referred to as **Kubernetes-in-Kubernetes** or **Tenant Clusters**. In this architecture, your underlying b..."

Busk: Created page with "One of the most popular and powerful use cases for KubeVirt, often referred to as Kubernetes-in-Kubernetes or Tenant Clusters. In this architecture, your underlying b..."