Hello Tello Pilot!
Join our DJI Tello community & remove this banner.
Sign up

New Python library for Tello AI - jetson-tello

robagar

Active member
Joined
Apr 17, 2021
Messages
30
Reaction score
16
Announcing jetson-tello - a new Python library for connecting the Tello with the AI capability provided by NVIDIA's Jetson hardware

Makes it as simple as possible to feed drone video into a neural network for AI processing

Here's an example - detecting faces and recognising objects in view of a flying drone -

Python:
#!/usr/bin/env python3

import asyncio
import jetson.inference
from jetson_tello import h264_frame_to_cuda, FrameDecodeError
from tello_asyncio import Tello

face_detector = jetson.inference.detectNet("facenet", threshold=0.5)
object_detector = jetson.inference.detectNet("ssd-mobilenet-v2", threshold=0.5)


async def process_frame(frame):
    try:
        cuda, width, height = h264_frame_to_cuda(frame)

        face_detections = face_detector.Detect(cuda)
        object_detections = object_detector.Detect(cuda)

        print('faces:')
        for d in face_detections:
            print(d)

        print('objects:')
        for d in object_detections:
            print(d)

    except FrameDecodeError:
        pass   

async def main():
    global next_frame

    drone = Tello()

    await drone.wifi_wait_for_network()
    await drone.connect()
    await drone.start_video()

    async def fly():
        await drone.takeoff()

    async def process_video():
        async for frame in drone.video_stream:
            await process_frame(frame)

    try:
        await asyncio.wait([fly(), process_video()])
    finally:
        await drone.stop_video()
        await drone.disconnect()

loop = asyncio.get_event_loop()
loop.run_until_complete(main())
 
  • Like
Reactions: AIDrone and pgminin
Announcing jetson-tello - a new Python library for connecting the Tello with the AI capability provided by NVIDIA's Jetson hardware

Makes it as simple as possible to feed drone video into a neural network for AI processing

Here's an example - detecting faces and recognising objects in view of a flying drone -

Python:
#!/usr/bin/env python3

import asyncio
import jetson.inference
from jetson_tello import h264_frame_to_cuda, FrameDecodeError
from tello_asyncio import Tello

face_detector = jetson.inference.detectNet("facenet", threshold=0.5)
object_detector = jetson.inference.detectNet("ssd-mobilenet-v2", threshold=0.5)


async def process_frame(frame):
    try:
        cuda, width, height = h264_frame_to_cuda(frame)

        face_detections = face_detector.Detect(cuda)
        object_detections = object_detector.Detect(cuda)

        print('faces:')
        for d in face_detections:
            print(d)

        print('objects:')
        for d in object_detections:
            print(d)

    except FrameDecodeError:
        pass

async def main():
    global next_frame

    drone = Tello()

    await drone.wifi_wait_for_network()
    await drone.connect()
    await drone.start_video()

    async def fly():
        await drone.takeoff()

    async def process_video():
        async for frame in drone.video_stream:
            await process_frame(frame)

    try:
        await asyncio.wait([fly(), process_video()])
    finally:
        await drone.stop_video()
        await drone.disconnect()

loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Hi, please can you provide some numbers on inference time on the jetson, in particular for the mobilnet model?
 
sure, I'll need to add some timing code.. It should also skip frames to keep up with the video, but I haven't verified that it actually is yet
 
  • Like
Reactions: pgminin
ok, here's the detection times for frames captured from the drone, but loaded from file in the object detection example

Code:
detection times (seconds):
  1: 2.303
  2: 0.1095
  3: 0.05508
  4: 0.06146
  5: 0.05088
  6: 0.05064
  7: 0.05104

The first frame is slow, but after that its ~20fps :)

I'll do another experiment with live video from a flying drone & see if makes any difference..
 
  • Like
Reactions: pgminin
Some results from a flying drone:

Code:
  frame #93, detection time 0.04997s, dining table (0.95), chair (0.78), person (0.68), person (0.94)
  frame #95, detection time 0.04945s, dining table (0.96), chair (0.74), person (0.62), person (0.94)
  frame #97, detection time 0.04964s, dining table (0.96), chair (0.78), person (0.74), person (0.94)
  frame #99, detection time 0.04992s, dining table (0.94), chair (0.72), person (0.78), person (0.91)
  frame #101, detection time 0.04975s, dining table (0.97), chair (0.64), person (0.8), person (0.91)
  frame #103, detection time 0.05076s, dining table (0.98), chair (0.57), person (0.66), person (0.91)
  frame #105, detection time 0.05089s, dining table (0.98), chair (0.55), person (0.56), person (0.91)
  frame #107, detection time 0.0501s, dining table (0.97), chair (0.66), person (0.59), person (0.93)
  frame #109, detection time 0.05065s, dining table (0.97), chair (0.63), person (0.6), person (0.93)
  frame #111, detection time 0.05063s, dining table (0.97), chair (0.61), person (0.6), person (0.91)
  frame #114, detection time 0.05047s, dining table (0.88), chair (0.82), person (0.58), person (0.85)

Pretty good, apart from seeing a ghost ?
 
jetson-tello 1.1.8

Mostly a tidy up, but also added a utility function run_jetson_tello_app which makes it very easy to fly a drone and CUDA process video frames, without falling behind the live video stream.

Here's an example - take off, look around and find faces...

Python:
#!/usr/bin/env python3

import asyncio
import jetson.inference
from jetson_tello import run_jetson_tello_app


face_detector = jetson.inference.detectNet("facenet", threshold=0.5)


def detect_faces(drone, frame, cuda):
    face_detections = face_detector.Detect(cuda)

    for d in face_detections:
        print(d)


async def fly(drone):
    await drone.takeoff()
    for i in range(4):
        await drone.turn_clockwise(90)
        await asyncio.sleep(3)
    await drone.land()


run_jetson_tello_app(fly, process_frame=detect_faces)
 
  • Like
Reactions: Hacky and AIDrone
Hello,

It's not working for me. This message appeared:

>>> from jetson_tello import h264_frame_to_cuda, NoFrame
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/nvidia/.local/lib/python3.6/site-packages/jetson_tello/__init__.py", line 1, in <module>
from .video import h264_frame_to_cuda, h264_frame_to_numpy_array, decoded_frame_to_cuda, decoded_frame_to_numpy_array, NoFrameData, DecodedFrame, H264DecoderAsync
ImportError: cannot import name 'decoded_frame_to_cuda'

Can you help me ?

Thanks
 
Hi @Ivette sorry I've been away for a while - I'm in the middle of pulling the video decoding parts out into their own package (tello-asyncio-video) and I must have left jetson-tello in a rather poor state.. Hopefully I'll be able to get back onto it over the weekend
 

New Posts

Members online

No members online now.

Forum statistics

Threads
5,690
Messages
39,934
Members
17,023
Latest member
Repiv

New Posts