Last incompatible policy: RELIABILITY_QOS_POLICY - ROS2 bag problem

Question

I'm using ROS2 Humble on Ubuntu 20.04 and I'm trying to export my ros bags to CSV files. In order to save the data, I created a python consumer that reads the messages and keep them in memory.

This is the bag info:

gorgo@razer:~/Downloads/rosbag2_2023_06_20-14_23_35$ ros2 bag info .

Files:             rosbag2_2023_06_20-14_23_35_0.db3
Bag size:          14.7 MiB
Storage id:        sqlite3
Duration:          158.450s
Start:             Jun 20 2023 18:23:36.22 (1687271016.22)
End:               Jun 20 2023 18:26:14.473 (1687271174.473)
Messages:          161929
Topic information: Topic: /fc/info/channels | Type: drone_racing_msgs/msg/RcChannels | Count: 15845 | Serialization Format: cdr
                   Topic: /fc/info/motors_thrust | Type: drone_racing_msgs/msg/SetpointMotorsThrust | Count: 15846 | Serialization Format: cdr
                   Topic: /perception/drone_state | Type: drone_racing_msgs/msg/DroneState | Count: 43087 | Serialization Format: cdr
                   Topic: /sensors/battery | Type: drone_racing_msgs/msg/SensorBattery | Count: 7923 | Serialization Format: cdr
                   Topic: /sensors/imu | Type: drone_racing_msgs/msg/SensorImu | Count: 79228 | Serialization Format: cdr

/sensors/imu is the topic I want to save.

gorgo@razer:~/drone-racing$ ros2 topic info /sensors/imu --verbose
Type: drone_racing_msgs/msg/SensorImu

Publisher count: 1

Node name: rosbag2_player
Node namespace: /
Topic type: drone_racing_msgs/msg/SensorImu
Endpoint type: PUBLISHER
GID: 01.0f.93.ea.6e.a4.7b.b8.01.00.00.00.00.00.12.03.00.00.00.00.00.00.00.00
QoS profile:
  Reliability: BEST_EFFORT
  History (Depth): UNKNOWN
  Durability: TRANSIENT_LOCAL
  Lifespan: Infinite
  Deadline: Infinite
  Liveliness: AUTOMATIC
  Liveliness lease duration: Infinite

Subscription count: 0

In the python script I tried different QoS.

self.qos_profile = QoSProfile(reliability=QoSReliabilityPolicy.BEST_EFFORT, history=QoSHistoryPolicy.KEEP_LAST,depth=1)

doesn't keep the pace of the producer and I lose ~4k messages out of 79228.

I then tried to set KEEP_ALL and depth 10k (is depth really necessary?) along with the YAML file to override the ros2 bag.

ros2 bag play . --qos-profile-overrides-path history_override.yaml --topics /sensors/imu

The YAML file contains:

sensors/imu:
reliability: best_effort
history: keep_all
depth: 10000

With rate 1 is still losing 4-5k messages. I can actually save all the messages lowering the rate to 0.8.

I thought the problem was best_effort but It seems I cannot set reliable and I don't know why. On python I set

self.qos_profile = QoSProfile(reliability=QoSReliabilityPolicy.RELIABLE, history=QoSHistoryPolicy.KEEP_LAST,depth=1)

and in the YAML:

sensors/imu:
reliability: realiable
history: keep_all
depth: 10000

I get the error:

gorgo@razer:~/drone-racing$ ros2 run ros2csv ros2csv
[INFO] [1687429831.257306779] [csv_subscriber]: CSV Subscriber initialized...
[WARN] [1687429835.403624258] [csv_subscriber]: New publisher discovered on topic 'sensors/imu', offering incompatible QoS. No messages will be received from it. Last incompatible policy: RELIABILITY

PC specs:

gorgo@razer:~/drone-racing$ neofetch 
            .-/+oossssoo+/-.               gorgo@razer 
        `:+ssssssssssssssssss+:`           ----------- 
      -+ssssssssssssssssssyyssss+-         OS: Ubuntu 20.04.6 LTS x86_64 
    .ossssssssssssssssssdMMMNysssso.       Host: Blade 15 Advanced Model (Early 2020) - RZ09-033 5.04 
   /ssssssssssshdmmNNmmyNMMMMhssssss/      Kernel: 5.15.0-75-generic 
  +ssssssssshmydMMMMMMMNddddyssssssss+     Uptime: 5 hours, 34 mins 
 /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/    Packages: 4336 (dpkg), 23 (snap) 
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Shell: bash 5.0.17 
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   Resolution: 3440x1440, 2560x1440 
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   DE: GNOME 
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   WM: Mutter 
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   WM Theme: Adwaita 
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Theme: Yaru [GTK2/3] 
 /sssssssshNMMMyhhyyyyhdNMMMNhssssssss/    Icons: Yaru [GTK2/3] 
  +sssssssssdmydMMMMMMMMddddyssssssss+     Terminal: gnome-terminal 
   /ssssssssssshdmNNNNmyNMMMMhssssss/      CPU: Intel i7-10875H (16) @ 5.100GHz 
    .ossssssssssssssssssdMMMNysssso.       GPU: NVIDIA 01:00.0 NVIDIA Corporation Device 1e93 
      -+sssssssssssssssssyyyssss+-         GPU: Intel UHD Graphics 
        `:+ssssssssssssssssss+:`           Memory: 8089MiB / 15845MiB 
            .-/+oossssoo+/-.
                                                               
                                                               

Thank you!