top of page

Persistent Volume for CNF deployed over multiple vSphere Clusters

Problem Description-

CNF could be deployed under multiple vSphere Clusters due to scaling or design implementation. If CNF is requiring Persistent Volume, the requirement from K8S is that all the nodes will see the same Datastore. TCA allow the CSI configuration assigning a single Datastore but In VF storage is based on FC and in a vSphere environment a LUN can be presented to 64 hosts maximum in ESX 7.0. u2.

Even with the documented workaround a set of Pods must be confined in a single cluster since in case that a pod for any reason will be moved from one cluster to another it will not possible to access the same datastore over multiple vSphere clusters.



Workaround-

To overcome the PV creation this is the documented workaround Prerequires:

  • Create TKG cluster

Steps:

  • In vCenter create a Datastore TAG

  • In vCenter assign a TAG to at least one Datastore for each cluster

  • In vCenter define a VM Storage Policy using the TAG

  • Create a new K8S Storage Class using the VM storage policy as follow: kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: psc annotations: storageclass.kubernetes.io/is-default-class: "false" # Optional provisioner: csi.vsphere.vmware.com allowVolumeExpansion: true # Optional: only applicable to vSphere 7.0U1 and above parameters: storagepolicyname: "k8s-persistent" csi.storage.k8s.io/fstype: "ext4" volumeBindingMode: WaitForFirstConsumer

  • Make the K8S Storage Class the default one and change the default as the non default kubectl patch k8s-persistent standard -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' kubectl patch vc-sphere standard -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'

After the previous steps the CNF could be onboarded.



8 views0 comments

Recent Posts

See All
bottom of page