A CSI (Container Storage Interface) driver required for volume provisioning or mounting is not available on worker nodes. This occurs when the driver DaemonSet isn't deployed, pods failed to start, or node taints prevent scheduling. Fix by deploying the driver via Helm, checking node scheduling constraints, and verifying driver registration.
CSI drivers are plugins that run as DaemonSets on Kubernetes worker nodes. They enable dynamic volume provisioning and mounting for various storage backends (AWS EBS, Azure Disk, NFS, etc.). When a pod tries to use a volume with a CSI driver that isn't registered on its node, Kubernetes cannot complete the mount. This registration happens automatically via the kubelet plugin registration mechanism when CSI driver pods start successfully. If the driver pods aren't running or can't register (due to taints, resource constraints, or startup failures), volumes relying on that driver become unavailable.
List all CSI drivers and nodes:
kubectl get csinodes
kubectl get csidriver
kubectl describe csinode <node-name>If the CSINode object shows no drivers, or the driver you need is missing, the driver isn't registered on that node.
Check deployment status of the driver:
# Find CSI driver namespace (often kube-system, default, or custom)
kubectl get pods -A | grep -i csi
# Check specific driver pods
kubectl get pods -n <driver-namespace> -l app=<driver-name>
kubectl get pods -n <driver-namespace> -o wide | grep <driver-name>Expected: Node pods running on each worker node, Controller pod in at least one replica. If pods are CrashLoopBackOff or Pending, move to step 3.
If pods aren't running, check logs:
kubectl logs -n <driver-namespace> <csi-node-pod> -c <node-container>
kubectl describe pod -n <driver-namespace> <csi-node-pod>Common errors:
- ImagePullBackOff: Driver image not accessible; verify image registry and credentials
- CrashLoopBackOff: Driver startup failure; check logs for configuration/permission issues
- Pending: Pod scheduling blocked; check node taints and nodeSelector (next step)
For permission issues, ensure kubelet can access /var/lib/kubelet/plugins/ directory.
Node taints may prevent DaemonSet from scheduling:
kubectl describe node <node-name> | grep Taints
kubectl get daemonset -n <driver-namespace> <driver-name> -o yaml | grep -A5 tolerations
kubectl get daemonset -n <driver-namespace> <driver-name> -o yaml | grep -A5 nodeSelectorIf the DaemonSet lacks tolerations for node taints, add them:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: csi-node
spec:
template:
spec:
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
- key: node-role.kubernetes.io/control-plane
operator: Exists
- key: <custom-taint-key>
operator: Equal
value: <custom-taint-value>
nodeSelector:
kubernetes.io/os: linux # Ensure it matches your nodesInstall via Helm (most drivers provide official Helm charts):
AWS EBS CSI Driver:
helm repo add aws-ebs-csi-driver https://kubernetes-sigs.github.io/aws-ebs-csi-driver
helm install aws-ebs-csi-driver aws-ebs-csi-driver/aws-ebs-csi-driver \
-n kube-system \
--set serviceAccount.controller.annotations."eks.amazonaws.com/role-arn"=arn:aws:iam::ACCOUNT_ID:role/AmazonEKS_EBS_CSI_DriverRoleAzure Disk CSI Driver:
helm repo add azuredisk-csi-driver https://raw.githubusercontent.com/kubernetes-sigs/azuredisk-csi-driver/master/charts
helm install azuredisk-csi-driver azuredisk-csi-driver/azuredisk-csi-driver \
-n kube-systemGCP Persistent Disk CSI Driver:
helm repo add pd-csi-driver https://kubernetes-sigs.github.io/gcp-compute-persistent-disk-csi-driver
helm install gcp-compute-pd-csi-driver pd-csi-driver/gcp-compute-pd-csi-driver \
-n kube-systemNFS CSI Driver:
helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner
helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
-n kube-system \
--set nfs.server=<nfs-server-ip> \
--set nfs.path=/exportsAfter installation, verify pods are running and nodes register the driver (step 2).
Cloud-native drivers (EBS, Azure Disk, GCP PD) require proper credentials:
For AWS EBS:
Create IAM role with EBS permissions, then annotate the Kubernetes ServiceAccount:
apiVersion: v1
kind: ServiceAccount
metadata:
name: ebs-csi-controller-sa
namespace: kube-system
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT:role/EBS_CSI_DriverRoleFor Azure Disk:
Ensure cluster has Azure RBAC or service principal configured:
az aks show -n <cluster-name> -g <rg> | grep -i "servicePrincipal"For GCP PD:
Ensure Workload Identity is configured and ServiceAccount has proper IAM roles.
After driver deployment:
kubectl get csidriver
kubectl get csinodes -o wide
kubectl describe csinode <node-name>Then test provisioning:
kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: test-csi-pvc
spec:
storageClassName: <csi-storage-class> # e.g., ebs-gp2
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
EOF
kubectl get pvc test-csi-pvc
kubectl describe pvc test-csi-pvcPVC should bind within seconds. If stuck in Pending, describe again to see new errors.
CSI drivers require kubelet plugin registration mechanism to work correctly. Ensure kubelet is running with --allow-privileged=true and has access to /var/lib/kubelet/plugins/ directory. Each CSI driver implementation has specific requirements: EBS needs IAM permissions, Azure Disk needs Azure credentials, GCP PD needs Workload Identity, Secrets Store CSI needs SecretProviderClass definitions. For multi-cloud clusters, deploy multiple CSI drivers; they coexist by having unique driver names (e.g., ebs.csi.aws.com vs azure.csi.io). Kubernetes 1.13+ is required for CSI support; older clusters must upgrade. Some CSI drivers support topology-aware provisioning; configure allowedTopologies in StorageClass to ensure volumes provision in correct zones. For production, ensure CSI driver controller pods have sufficient resources and are spread across nodes for HA. Storage class selection matters: test different parameters (IOPS, volume type) and use fsGroupChangePolicy in Kubernetes 1.20+ for large volumes.
Failed to connect to server: connection refused (HTTP/2)
How to fix "HTTP/2 connection refused" error in Kubernetes
missing request for cpu in container
How to fix "missing request for cpu in container" in Kubernetes HPA
error: invalid configuration
How to fix "error: invalid configuration" in Kubernetes
etcdserver: cluster ID mismatch
How to fix "etcdserver: cluster ID mismatch" in Kubernetes
running with swap on is not supported
How to fix "running with swap on is not supported" in kubeadm