Doing minor version update of Ceph cluster with ceph-ansible and rolling-update playbook

Question

I want to upgrade my ceph cluster to the latest minor version of the current installed major version (mimic). I have a ceph-ansible setup for deployment which is used to deploy new nodes with the site.yml playbook. I learned from the documentation that I need to execute the rolling-update playbook in order to perform an update be it minor or major.

For a minor update I just execute the playbook with keeping ceph_stable_release set to my current installed version.

To do a major update (after doing the minor update) I change ceph_stable_release in my case from mimic to nautilus and then execute the rolling-update playbook.

The documentation is short on this information.

Is this assumption correct?

Answer 1

To answer my own question:

I checked with the ceph community user mailing list and was assured that my assumption is correct.

To do a minor update of your ceph cluster keep the variable ceph_stable_release at your current installed version.

To check what versions your ceph components are currently running you can issue these commands, if you want to reassure yourself:

# ceph tell mon.* version 

and

# ceph tell osd.* version

The output should show same versions for monitors and osds.

Running the upgrade with

ansible-playbook -i inventory infrastructure-playbooks/rolling_update.yml

Will first update all monitors, then managers followed by osds. Depending on your ceph setup other types of services will be updated as well. Since I only run monitors, osds and managers I can't tell in which order these will occur.

For me updating mimic to its latests version, I had only one hiccup. ceph-ansible was waiting for an updated osd to recover, but one PG stayed in active+undersized+remapped+backfilling. It did not recover until I removed the norebalance flag with

# ceph osd unset norebalance

After this the osd recovered immediately and ceph-ansible could continue. Just as a side note, if this should happen to you as well.