Problem Description-
CNF could be deployed under multiple vSphere Clusters due to scaling or design implementation. If CNF is requiring Persistent Volume, the requirement from K8S is that all the nodes will see the same Datastore. TCA allow the CSI configuration assigning a single Datastore but In VF storage is based on FC and in a vSphere environment a LUN can be presented to 64 hosts maximum in ESX 7.0. u2.
Even with the documented workaround a set of Pods must be confined in a single cluster since in case that a pod for any reason will be moved from one cluster to another it will not possible to access the same datastore over multiple vSphere clusters.
Workaround-
To overcome the PV creation this is the documented workaround Prerequires:
Create TKG cluster
Steps:
In vCenter create a Datastore TAG
In vCenter assign a TAG to at least one Datastore for each cluster
In vCenter define a VM Storage Policy using the TAG
Create a new K8S Storage Class using the VM storage policy as follow: kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: psc annotations: storageclass.kubernetes.io/is-default-class: "false" # Optional provisioner: csi.vsphere.vmware.com allowVolumeExpansion: true # Optional: only applicable to vSphere 7.0U1 and above parameters: storagepolicyname: "k8s-persistent" csi.storage.k8s.io/fstype: "ext4" volumeBindingMode: WaitForFirstConsumer
Make the K8S Storage Class the default one and change the default as the non default kubectl patch k8s-persistent standard -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}' kubectl patch vc-sphere standard -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
After the previous steps the CNF could be onboarded.
Comments