Written by Jamie Tanna
on October 10, 2023
CC-BY-NC-SA-4.0 Apache-2.0
2 mins

Gotcha: Using vCluster on Elastic Kubernetes Service requires a Container Storage Interface driver

Featured image for sharing metadata for article

I've recently been playing around with vcluster on an Amazon Elastic Kubernetes Service (EKS) cluster, but about two commands into getting set up on my cluster, I had the vcluster setup fail.

After failing to create the vcluster, I investigated the pods in the vcluster, and saw that etcd was in a CrashLoopBackOff.

When I went to investigate the pod, I could see the following error in the events:

$ kubectl describe -n vcluster-test-vcluster pods/test-vcluster-etcd-0
...
Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  3m36s (x3 over 23m)  default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition

Searching around recommended that I dig into the PersistentVolumeClaims by running:

kubectl describe pvc -n vcluster-test-vcluster
Name:          data-test-vcluster-etcd-0
Namespace:     vcluster-test-vcluster
StorageClass:  gp2
Status:        Pending
Volume:
Labels:        app=vcluster-etcd
               release=test-vcluster
Annotations:   volume.beta.kubernetes.io/storage-provisioner: ebs.csi.aws.com
               volume.kubernetes.io/selected-node: ip-xxx-xxx-x-xxx.ec2.internal
               volume.kubernetes.io/storage-provisioner: ebs.csi.aws.com
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:
Access Modes:
VolumeMode:    Filesystem
Used By:       test-vcluster-etcd-0
Events:
  Type    Reason                Age                      From                         Message
  ----    ------                ----                     ----                         -------
  Normal  WaitForFirstConsumer  41s                      persistentvolume-controller  waiting for first consumer to be created before binding
  Normal  ExternalProvisioning  <invalid> (x8 over 41s)  persistentvolume-controller  waiting for a volume to be created, either by external provisioner "ebs.csi.aws.com" or manually created by system administrator

This error led me to this StackOverflow answer, which indicated that I needed to make sure that I needed to set up my cluster to be able to provision Amazon Elastic Block Store with the steps mentioned. Once completed, my etcd was able to start correctly, and I could continue with my setup.

Written by Jamie Tanna on Tue, 10 Oct 2023 13:38:13+01:00, and last updated on Tue, 10 Oct 2023 13:38:13+01:00.

Content for this article is shared under the terms of the Creative Commons Attribution Non Commercial Share Alike 4.0 International, and code is shared under the Apache License 2.0.

#blogumentation #aws #kubernetes.

This post was filed under articles.

Interactions with this post

Below you can find the interactions that this page has had using WebMention.

Have you written a response to this post? Let me know the URL:

Do you not have a website set up with WebMention capabilities? You can use Comment Parade.

← → Top