Removing stuck K8 tasks “CREATE:IN_PROGRESS” – VCD CSE

When spinning up a K8 clusters in VMware Cloud Director and using the CSE plugin you might sometimes see that the cluster gets stuck in “CREATE:IN_PROGRESS” status, this can happen for a number of reasons but normally if ther cluster does not have the right network connectivity to complete it bootstrap process. when this does happen you are not able to delete the cluster from the UI and need to use the api to clear it up .

Connecting to VCD using Postman, I’m using VCD 10.3 and CSE 3.1.1 .

We need to generate an access token before we can do anything follow these instructions if you get stuck here

Postman settings:
Authentication Tab : Basic Auth – User = administrator@system with the password
Headers Tab: Accept, Value = application/*;version=36.0

Run this command
POST:https://<VCDIP/URL>/cloudapi/1.0.0/sessions/provider

this will give you a Access-Token, switch the authentication to Token bearer and copy and paste the token value from the reply above.

Grab the cluster ID for the K8 that you want to delete from the details section in the plugin UI

Run a Get to check that you can see the cluster details, this should come back as a status 200, if not go back and re do your token steps. make sure you change the cluster ID from the below command.

Get: https:// <VCDIP/URL> /cloudapi/1.0.0/entities/urn:vcloud:entity:cse:nativeCluster:c76606f2-0837-4e69-b30b-8cbf7b323a5c

Get: https:// <VCDIP/URL> /cloudapi/1.0.0/entities/<cluster-id>

We are now ready to delete the cluster entry.

Delete: https:// <VCDIP/URL> /cloudapi/1.0.0/entities/urn:vcloud:entity:cse:nativeCluster:c76606f2-0837-4e69-b30b-8cbf7b323a5c

now this should but sometimes it will fire off the task and give you a 202 reply, you will then see a task running in the VCD portal which will fail with a helpful debug message…..

The next step is to run the below to remove the webhook

Delete: https:// <VCDIP/URL> /cloudapi/1.0.0/entities/urn:vcloud:entity:cse:nativeCluster:c76606f2-0837-4e69-b30b-8cbf7b323a5c?invokeHooks=false

you should now see the entry removed from the UI, you can then do a manual clean up of the vApp.