Sending object metadata causes FPS drops in the stream

Question

I want to send some object metadata(class_id, confidence value, etc…) to another PC when the object is detected but it causes FPS drops and the stream is frozen. Which parallel programming technique I should use to solve it? Can you give me an example of it?

Checking if detected object in the class_dict:

if obj_meta.class_id in class_dict:
                send_one(obj_meta.class_id)

I am using this function to send class_id message.

from __future__ import print_function
import can

def send_one(class_id):
    bus = can.interface.Bus()

    bus = can.interface.Bus(bustype='socketcan', channel='vcan0', bitrate=250000)

    msg = can.Message(arbitration_id=0xc0ffee,
                      **data=[class_id]**,
                      is_extended_id=True)
    try:
        bus.send(msg)
        print("Message sent on {}".format(bus.channel_info))
    except can.CanError:
        print("Message NOT sent")

Answer 1

A little bit more code would help, but I'm assuming you are (were?) doing the check inside a gstreamer buffer probe. Buffer probe blocks buffer downstream so no new buffers keep coming until you've disposed of it.

A: using external service: use the msgbroker element to produce messages and inject into alternative service (eg rabbit, kafka). See reference implementation here. Then, use a service-specific consumer to process the data (and call your send_one).

B: from python: You should extract metadata as quickly as possible, and then process it from outside.

from queue import Queue, Empty
from threading import Thread
q = Queue()



...
#in buffer probe:
if obj_meta.class_id in class_dict:
    q.put(obj_meta.class_id)

...
def consume():
    while True:
        try:
            data = q.get(block=True, timeout=1)
        except Empty:
            pass
        
...

consumer = Thread(target=consume)

consumer.start()

you could improve from this eg by reading in batches, running multiple consumer threads, etc.

Answer 2

I am not sure what's your usecase but I would recommend to have a look at msgbroker (DS plugin) for msg passing between the applications