OpenCV and Tello Drone

[shahiro07]

I am new to this thread so I'm not sure if this is even allowed but, I have been following this Object detection tutorial on youtube (

<- the link for anyone that is curious) and I am trying to have the object detection program you build throughout the video stream from my tello drone rather than my webcam. I have tried different things that forums have said but nothing seems to work. This is the snippet of code that I believe would need adjustments:

from djitellopy import Tello
import socket
import time

tello = Tello()
tello.connect()
tello.streamon()
time.sleep(5)


cap = cv2.VideoCapture('udp://0.0.0.0.11111')

width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

while cap.isOpened:
ret, frame = cap.read()
time.sleep(10)
image_np = np.array(frame)

input_tensor = tf.convert_to_tensor(np.expand_dims(image_np, 0), dtype=tf.float32)
print(input_tensor)
detections = detect_fn(input_tensor)

num_detections = int(detections.pop('num_detections'))
detections = {key: value[0, :num_detections].numpy()
for key, value in detections.items()}
detections['num_detections'] = num_detections

# detection_classes should be ints.
detections['detection_classes'] = detections['detection_classes'].astype(np.int64)

label_id_offset = 1
image_np_with_detections = image_np.copy()

viz_utils.visualize_boxes_and_labels_on_image_array(
image_np_with_detections,
detections['detection_boxes'],
detections['detection_classes']+label_id_offset,
detections['detection_scores'],
category_index,
use_normalized_coordinates=True,
max_boxes_to_draw=5,
min_score_thresh=.8,
agnostic_mode=False)

cv2.imshow('object detection', cv2.resize(image_np_with_detections, (800, 600)))

if cv2.waitKey(10) & 0xFF == ord('q'):
cap.release()
cv2.destroyAllWindows()
break

 

[Hacky]

Welcome to this forum.
Some things that caught my eye:

time.sleep(10) means: sleep for 10 seconds!
I assume, this is not intended inside the loop. If in that step the program should sleep for 10 milliseconds, it would be: time.sleep(0.01)

You should read the frames in a seperate thread and put them into a queue. The detection should run in another thread and pick always only the latest frame from that queue (the others can be discarded if not needed or just used for the thread that displays the frames in order to maintain a smooth display. To see implementations for this concept, you can google for "opencv read thread queue".