Rook-Ceph OSD pods not getting created

Question

I have a fresh cluster with a few nodes. Each node is a Linux VM with a 6GB device mounted at dev/sdc1 which I would like to use. When I try to create the rook ceph cluster, rook-ceph-osd-prepare pod logs shows that it is skipping dev/sdc1 because of “Has BlueStore device label”

The sdc1 device is created and mounted using the following powershell commands.

New-VHD -Path $path -SizeBytes $size
$disk = Mount-VHD -Path $path -PassThru
Initialize-Disk $disk.DiskNumber -PartitionStyle MBR
$partition = New-Partition -AssignDriveLetter -UseMaximumSize -DiskNumber $disk.DiskNumber
Dismount-VHD -Path $path
Add-VMHardDiskDrive -VM $vm -ControllerType SCSI -ControllerLocation 3 -Path $path

What am I doing wrong and how do I fix this?

Answer 1

This is a very common issue on Ceph. Below I am citing a possible reason and possible solution from the official documentation for OSD pods are not created on my devices.

---

Possible Reason:

---

One common case for failure is that you have re-deployed a test cluster and some state may remain from a previous deployment. If your cluster is larger than a few nodes, you may get lucky enough that the monitors were able to start and form quorum.

---

Possible Solution:

---

If the error is from the file that already exists, this is a common problem reinitializing the Rook cluster when the local directory used for persistence has not been purged. This directory is the dataDirHostPath setting in the cluster CRD and is typically set to /var/lib/rook. To fix the issue you will need to delete all components of Rook and then delete the contents of /var/lib/rook (or the directory specified by dataDirHostPath) on each of the hosts in the cluster. Then when the cluster CRD is applied to start a new cluster, the rook-operator should start all the pods as expected.

Answer 2

Try to format disk with sgdisk --zap-all /dev/xxx and delete dataDirHostPath of specific osd.