ROS - SerializationError when writing a bagfile with custom ROS message

Question

I have a created a custom ros message which has the following format

# Header information
std_msgs/Header header

# Model
string model

# Data
uint8[] data

The data is a list of 4D lists reading from csv file

def createData(csv_file): 
     x,y,z,w = np.loadtxt(dataFile, usecols=(0,1,2,3), \ skiprows=0, delimiter="\t", unpack=True) 
     p = [] 
     for p1, p2, p3, p4 in zip(x,y,z,w): 
        pt = [p1, p2, p3, p4] 
        p.append(pt)
     header = Header()
     header.stamp = rospy.rostime.Time.from_sec(time.time())
     header.frame_id = "main_frame"
     model = "sensor"
     msg = customMessage(header, model, p)
     return msg, p

with rosbag.Bag('test.bag', 'w') as bag:
    msg, points = createData(csv_file)
    bag.write(topic_name, msg)
bag.close()

where inside the customMessage(genpy.Message) class, I set the msg.data to p, msg.header to header, and msg.model to model I created a random data with this message type, and I am trying to write a bag file out of this

bag.write(topic_name, msg)

However I receive this error

Traceback (most recent call last):
  File "data2bag.py", line 306, in <module>
    bag.write(topic_name, msg)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbag/bag.py", line 391, in write
    msg.serialize(self._buffer)
  File "data2bag.py", line 123, in serialize
    except struct.error as se: self._check_types(struct.error("%s: '%s' when writing '%s'" % (type(se), str(se), str(locals().get('_x', self)))))
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/genpy/message.py", line 334, in _check_types
    check_type(n, t, getattr(self, n))
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/genpy/message.py", line 256, in check_type
    check_type(field_name+"[]", base_type, v)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/genpy/message.py", line 212, in check_type
    raise SerializationError('field %s must be unsigned integer type'%field_name)
genpy.message.SerializationError: field data[] must be unsigned integer type

The msg.data has a type of <type 'list'> which is the assumed type The writing of the bag file is successful if only the msg.data has a str type

Nevertheless, this does not make sense since it won't be working with a str data type

If I change the data to uint8 by: points = np.uint8(p)

I get this error instead

Traceback (most recent call last):
  File "data2bag.py", line 307, in <module>
    bag.write(topic_name, msg)
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/rosbag/bag.py", line 391, in write
    msg.serialize(self._buffer)
  File "data2bag.py", line 123, in serialize
    except struct.error as se: self._check_types(struct.error("%s: '%s' when writing '%s'" % (type(se), str(se), str(locals().get('_x', self)))))
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/genpy/message.py", line 334, in _check_types
    check_type(n, t, getattr(self, n))
  File "/opt/ros/kinetic/lib/python2.7/dist-packages/genpy/message.py", line 254, in check_type
    raise SerializationError('field %s must be a list or tuple type'%field_name)
genpy.message.SerializationError: field data must be a list or tuple type

Did anybody encountered such problem before?

I tried changing the data reading method

for p1, p2, p3, p4 in zip(x,y,z,w):
        pt = [int(p1), int(p2), int(p3), int(p4)]
        p.extend(pt)

However my p1, p2, p3 and p4 are float, I just did as int casting to try it. I got the same error

Answer 1

I suspect the elements you have in your list are just not of the right type. Impossible to say without seeing your code. Here is a minimal working example:

import rosbag
from std_msgs.msg import UInt8MultiArray
bag = rosbag.Bag('test.bag', 'w')

try:
    a = UInt8MultiArray()
    a.data = [1,2,3]
    bag.write('array', a)
finally:
    bag.close()

This works fine.

---

Update based on the code you posted.

I'm pretty sure your list contains arrays of strings, both of which is wrong. You want it to be flat and integers:

     p = [] 
     for p1, p2, p3, p4 in zip(x,y,z,w): 
        p.extend([int(p1), int(p2), int(p3), int(p4)])