Catch on How to disable TKG telemetry Service

Symptoms

When a TKG Management cluster is deployed, , participation in the VMware Customer Experience Improvement Program (CEIP) is enabled by default. The TKG Telemetry service is responsible for gathering the required CEIP related data and sending it to VMware. There is no option to directly disable this service from the UI.

Purpose

VMware’s Customer Experience Improvement Program (CEIP) is a voluntary program that collects information about how people use our products.

The data collected may include device identifiers and information that identifies your users. This data is collected to enable VMware to diagnose and improve its products and services, fix product issues, provide proactive technical support, and to advise you on how to best deploy and use our products.

When you deploy a management cluster by using either the installer interface or the CLI, participation in the VMware Customer Experience Improvement Program (CEIP) is enabled by default, unless you specify the option to opt out. If you remain opted in to the program, the management cluster sends data to VMware at regular intervals. This data is collected to enable VMware to diagnose and improve its products and services, fix product issues, provide proactive technical support, and to advise you on how to best deploy and use our products.

Cause

When customers opt in for the VMware CEIP program , management clusters send the following information to VMware:

• The number of workload clusters that you deploy.

• The infrastructure, network, and storage providers that you use.

• The time that it takes for the tanzu CLI to perform basic operations such as cluster create, cluster delete, cluster scale, and cluster upgrade.

• The Tanzu Kubernetes Grid packages that you implement.

• The plans that you use to deploy clusters, as well as the number and configuration of the control plane and worker nodes.

• The versions of Tanzu Kubernetes Grid and Kubernetes that you use.

• The type and size of the workloads that your clusters run, as well as their lifespan.

• Whether or not you integrate Tanzu Kubernetes Grid with vSphere with Tanzu, Tanzu Mission Control, or Tanzu Observability by Wavefront.

• The nature of any problems, errors, and failures that you encounter when using Tanzu Kubernetes Grid, so that we can identify which areas of Tanzu Kubernetes Grid need to be made more robust.

Resolution

See CEIP Status

To see the management cluster’s current CEIP status:

1. Export the kubeconfig targeting your management cluster to an environment variable, KUBECONFIG

export KUBECONFIG=~/.kube/config

1. Run the tanzu telemetry status command to see the CEIP status of the current management cluster, for example:

tanzu telemetry status
- ceip: |
    level: disabled
  shared_identifiers: ...

1. The output ceip.level listing indicates CEIP status:

• standard, or no status listed: CEIP participation is enabled on a management cluster and it sends telemetry data. This is the default behavior.

• disabled: CEIP participation is deactivated.

Opt Out of CEIP

To deactivate CEIP participation on a management cluster on which it is currently activated:

1. Run the tanzu telemetry update --CEIP-opt-out command.

tanzu telemetry update --CEIP-opt-out

1. To verify that the CEIP participation is deactivated, run tanzu telemetry status again.

2. The status should now be disabled.

- ceip: |
    level: disabled
  shared_identifiers: ...

1. CEIP being deactivated does not remove the cron job from the management cluster. The cron job is responsible for checking if CEIP is deactivated/activated. The job should log that whether it is not collecting telemetry. To verify if the cron job is sending telemetry or not, we need to check the logs of the pod created by the telemetry cron job.

Run the following command to view all pods in the telemetry namespace:

kubectl get pods -n tkg-system-telemetry

The output should look something like:

NAME                             READY   STATUS      RESTARTS   AGE
tkg-telemetry-1657519200-l8k9x   0/1     Completed   0          14h
tkg-telemetry-1657540800-j4dzw   0/1     Completed   0          8h
tkg-telemetry-1657562400-pj826   0/1     Completed   0          125m

Pick the most recent completed pod and check its logs using the following command:

kubectl logs pods/tkg-telemetry-1657562400-pj826 -n tkg-system-telemetry

The logs should contain the following lines, indicating that no data was collected

2022-07-11T16:01:52.686-0400  INFO  CEIP.config vmware/ceip_validator.go:77  loaded config  {"cluster[level]": "disabled"}
2022-07-11T16:01:52.686-0400  INFO  tkg-telemetry/main.go:57  cluster is opted out of telemetry, aborting collection

Workaround

Deploy Management Cluster using configuration file and deactivate participation in the VMware Customer Experience Improvement Program (CEIP) by setting ENABLE_CEIP_PARTICIPATION to false.

For example:

#! ---------------------------------------------------------------------
#! Basic cluster creation configuration
#! ---------------------------------------------------------------------

CLUSTER_NAME: aws-mgmt-cluster
CLUSTER_PLAN: dev
INFRASTRUCTURE_PROVIDER: aws
ENABLE_CEIP_PARTICIPATION: true
ENABLE_AUDIT_LOGGING: true
CLUSTER_CIDR: 100.96.0.0/11
SERVICE_CIDR: 100.64.0.0/13