2.11.0
EditCreate An Issue

Troubleshooting OpenEBS - Dynamic LocalPV

OpenEBS Documentation is now migrated to https://openebs.io/docs. The page you are currently viewing is a static snapshot and will be removed in the upcoming releases.

General guidelines for troubleshooting



LocalPV PVC in Pending state

Application pod using LocalPV device not coming into running state

Stale BDC in pending state after PVC is deleted

BDC created by localPV in pending state

PVC in Pending state

Created a PVC using localpv-device / localpv-hostpath storage class. But the PV is not created and PVC in Pending state.

Troubleshooting: The default localpv storage classes from openebs have volumeBindingMode: WaitForFirstConsumer. This means that only when the application pod that uses the PVC is scheduled to a node, the provisioner will receive the volume provision request and will create the volume.

Resolution: Deploy an application that uses the PVC and the PV will be created and application will start using the PV


Application pod using LocalPV not coming into running state

Application pod that uses localpv device is stuck in Pending state with error

Warning  FailedScheduling  7m24s (x2 over 7m24s)  default-scheduler  persistentvolumeclaim "<pvc-name>" not found

Troubleshooting: Check if there is a blockdevice present on the node (to which the application pod was scheduled,) which matches the capacity requirements of the PVC.

kubectl get bd -n openebs -o wide

If matching blockdevices are not present, then the PVC will never get Bound.

Resolution: Schedule the application pod to a node which has a matching blockdevice available on it.


Stale BDC in pending state after PVC is deleted

kubectl get bdc -n openebs

shows stale Pending BDCs created by localpv provisioner, even after the corresponding PVC has been deleted.

Resolution: LocalPV provisioner currently does not delete BDCs in Pending state if the corresponding PVCs are deleted. To remove the stale BDC entries,

  1. Edit the BDC and remove the - local.openebs.io/finalizer finalizer
kubectl edit bdc <bdc-name> -n openebs
  1. Delete the BDC
kubectl delete bdc <bdc-name> -n openebs


BDC created by localPV in pending state

The BDC created by localpv provisioner (bdc-pvc-xxxx) remains in pending state and PVC does not get Bound

Troubleshooting: Describe the BDC to check the events recorded on the resource

kubectl describe bdc bdc-pvc-xxxx -n openebs

The following are different types of messages shown when the node on which localpv application pod is scheduled, does not have a blockdevice available.

  1. No blockdevices found
Warning  SelectionFailed  14m (x25 over 16m)    blockdeviceclaim-operator  no blockdevices found

It means that there were no matching blockdevices after listing based on the labels. Check if there is any block-device-tag on the storage class and corresponding tags are available on the blockdevices also

  1. No devices with matching criteria
Warning  SelectionFailed  6m25s (x18 over 11m)  blockdeviceclaim-operator  no devices found matching the criteria

It means that the there are no devices for claiming after filtering based on filesystem type and node name. Make sure the blockdevices on the node have the correct filesystem as mentioned in the storage class (default is ext4)

  1. No devices with matching resource requirements
Warning  SelectionFailed  85s (x74 over 11m)    blockdeviceclaim-operator  could not find a device with matching resource requirements

It means that there are no devices available on the node with a matching capacity requirement.

Resolution

To schedule the application pod to a node, which has the blockdevices available, a node selector can be used on the application pod. Here the node with hostname svc1 has blockdevices available, so a node selector is used to schedule the pod to that node.

Example:

apiVersion: v1
kind: Pod
metadata:
  name: pod1
spec:
  volumes:
  - name: local-storage
    persistentVolumeClaim:
      claimName: pvc1
  containers:
  - name: hello-container
    image: busybox
    command:
       - sh
       - -c
       - 'while true; do echo "`date` [`hostname`] Hello from OpenEBS Local PV." >> /mnt/store/greet.txt; sleep $(($RANDOM % 5 + 300)); done'
    volumeMounts:
    - mountPath: /mnt/store
      name: local-storage
  nodeSelector:
    kubernetes.io/hostname: svc1




See Also:

FAQs

Seek support or help

Latest release notes



Feedback

Was this page helpful?