How to make stream from Tello SDK 2.0 captured by opencv with cv2.VideoCapture()

Question

I want to write a code, showing stream from tello with yolov5 torch model in real-time with low latency.

But I am stuck with how to use cv2.VideoCapture() with tello.

from threading import Thread
from djitellopy import Tello
import cv2, math, time
import os


tello = Tello()
tello.connect(False)
tello.streamon()
print(tello.get_udp_video_address())
print(type(tello.get_udp_video_address()))
cap = cv2.VideoCapture(tello.get_udp_video_address())
try:
    while True:
        img = cap.read()
        cv2.imshow('frame', img)
except KeyboardInterrupt:
    exit(1)
finally:
    print("fin")

output:

[INFO] tello.py - 107 - Tello instance was initialized. Host: '192.168.10.1'. Port: '8889'.
[INFO] tello.py - 422 - Send command: 'command'
[WARNING] tello.py - 432 - Aborting command 'command'. Did not receive a response after 7 seconds
[INFO] tello.py - 422 - Send command: 'command'
[INFO] tello.py - 446 - Response command: 'ok'
[INFO] tello.py - 422 - Send command: 'streamon'
[INFO] tello.py - 446 - Response streamon: 'ok'
udp://@0.0.0.0:11111
<class 'str'>
fin
Traceback (most recent call last):
  File "bruh.py", line 16, in <module>
    cv2.imshow('frame', img)
cv2.error: OpenCV(4.5.5) :-1: error: (-5:Bad argument) in function 'imshow'
> Overload resolution failed:
>  - mat is not a numerical tuple
>  - Expected Ptr<cv::cuda::GpuMat> for argument 'mat'
>  - Expected Ptr<cv::UMat> for argument 'mat'

[INFO] tello.py - 422 - Send command: 'streamoff'

I want to use cv2.VideoCapture(), since there're more sources with webcam & real-time objects detection. I have tried many ways to deal with it but they seems didn't work for me. Ways include give capture a random udp port and let it find out the correct one itself.

Appreciate anyone's response.

Answer 1

this is the basic code for getting video stream from the tello:

from djitellopy import tello
import cv2

me = tello.Tello()
#cap = cv2.VideoCapture(0)
me.connect()
print(me.get_battery())
me.streamon()


while True:
    img = me.get_frame_read().frame
    img = cv2.resize(img, (360, 240))
    cv2.imshow("results", img)
    cv2.waitKey(1)

when doing that, you have to call 'tello.streamon()' and then in the loop you make that image variable:

img = me.get_frame_read().frame

i merged it with your code and this is the result:

from threading import Thread
from djitellopy import Tello
import cv2, math, time
import os


tello = Tello()
tello.connect()#
tello.streamon()
#print(tello.get_udp_video_address())
#print(type(tello.get_udp_video_address()))
#cap = cv2.VideoCapture(tello.get_udp_video_address())

try:
    while True:
        img = tello.get_frame_read().frame#
        cv2.imshow('frame', img)
        cv2.waitKey(1)#
except KeyboardInterrupt:
    exit(1)
finally:
    print("fin")

I put a comment/# after everyting i added/changed and commented out everything that was uneccesary, you basically had to just call img = tello.get_frame_read.frame and ur good. then just show it and add a delay of 1 milisec. and i suggest you watch this video: https://www.youtube.com/watch?v=LmEcyQnfpDA , it has a bunch of cool stuff you might wanna try out. enjoy!