Hello Tello Pilot!
Join our DJI Tello community & remove this banner.
Sign up

New Python library for Tello AI - jetson-tello

robagar

Active member
Joined
Apr 17, 2021
Messages
30
Reaction score
16
Announcing jetson-tello - a new Python library for connecting the Tello with the AI capability provided by NVIDIA's Jetson hardware

Makes it as simple as possible to feed drone video into a neural network for AI processing

Here's an example - detecting faces and recognising objects in view of a flying drone -

Python:
#!/usr/bin/env python3

import asyncio
import jetson.inference
from jetson_tello import h264_frame_to_cuda, FrameDecodeError
from tello_asyncio import Tello

face_detector = jetson.inference.detectNet("facenet", threshold=0.5)
object_detector = jetson.inference.detectNet("ssd-mobilenet-v2", threshold=0.5)


async def process_frame(frame):
    try:
        cuda, width, height = h264_frame_to_cuda(frame)

        face_detections = face_detector.Detect(cuda)
        object_detections = object_detector.Detect(cuda)

        print('faces:')
        for d in face_detections:
            print(d)

        print('objects:')
        for d in object_detections:
            print(d)

    except FrameDecodeError:
        pass   

async def main():
    global next_frame

    drone = Tello()

    await drone.wifi_wait_for_network()
    await drone.connect()
    await drone.start_video()

    async def fly():
        await drone.takeoff()

    async def process_video():
        async for frame in drone.video_stream:
            await process_frame(frame)

    try:
        await asyncio.wait([fly(), process_video()])
    finally:
        await drone.stop_video()
        await drone.disconnect()

loop = asyncio.get_event_loop()
loop.run_until_complete(main())
 
  • Like
Reactions: AIDrone and pgminin

pgminin

Well-known member
Joined
Jan 7, 2019
Messages
101
Reaction score
97
Announcing jetson-tello - a new Python library for connecting the Tello with the AI capability provided by NVIDIA's Jetson hardware

Makes it as simple as possible to feed drone video into a neural network for AI processing

Here's an example - detecting faces and recognising objects in view of a flying drone -

Python:
#!/usr/bin/env python3

import asyncio
import jetson.inference
from jetson_tello import h264_frame_to_cuda, FrameDecodeError
from tello_asyncio import Tello

face_detector = jetson.inference.detectNet("facenet", threshold=0.5)
object_detector = jetson.inference.detectNet("ssd-mobilenet-v2", threshold=0.5)


async def process_frame(frame):
    try:
        cuda, width, height = h264_frame_to_cuda(frame)

        face_detections = face_detector.Detect(cuda)
        object_detections = object_detector.Detect(cuda)

        print('faces:')
        for d in face_detections:
            print(d)

        print('objects:')
        for d in object_detections:
            print(d)

    except FrameDecodeError:
        pass

async def main():
    global next_frame

    drone = Tello()

    await drone.wifi_wait_for_network()
    await drone.connect()
    await drone.start_video()

    async def fly():
        await drone.takeoff()

    async def process_video():
        async for frame in drone.video_stream:
            await process_frame(frame)

    try:
        await asyncio.wait([fly(), process_video()])
    finally:
        await drone.stop_video()
        await drone.disconnect()

loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Hi, please can you provide some numbers on inference time on the jetson, in particular for the mobilnet model?
 

robagar

Active member
Joined
Apr 17, 2021
Messages
30
Reaction score
16
sure, I'll need to add some timing code.. It should also skip frames to keep up with the video, but I haven't verified that it actually is yet
 
  • Like
Reactions: pgminin

robagar

Active member
Joined
Apr 17, 2021
Messages
30
Reaction score
16
ok, here's the detection times for frames captured from the drone, but loaded from file in the object detection example

Code:
detection times (seconds):
  1: 2.303
  2: 0.1095
  3: 0.05508
  4: 0.06146
  5: 0.05088
  6: 0.05064
  7: 0.05104

The first frame is slow, but after that its ~20fps :)

I'll do another experiment with live video from a flying drone & see if makes any difference..
 
  • Like
Reactions: pgminin

robagar

Active member
Joined
Apr 17, 2021
Messages
30
Reaction score
16
Some results from a flying drone:

Code:
  frame #93, detection time 0.04997s, dining table (0.95), chair (0.78), person (0.68), person (0.94)
  frame #95, detection time 0.04945s, dining table (0.96), chair (0.74), person (0.62), person (0.94)
  frame #97, detection time 0.04964s, dining table (0.96), chair (0.78), person (0.74), person (0.94)
  frame #99, detection time 0.04992s, dining table (0.94), chair (0.72), person (0.78), person (0.91)
  frame #101, detection time 0.04975s, dining table (0.97), chair (0.64), person (0.8), person (0.91)
  frame #103, detection time 0.05076s, dining table (0.98), chair (0.57), person (0.66), person (0.91)
  frame #105, detection time 0.05089s, dining table (0.98), chair (0.55), person (0.56), person (0.91)
  frame #107, detection time 0.0501s, dining table (0.97), chair (0.66), person (0.59), person (0.93)
  frame #109, detection time 0.05065s, dining table (0.97), chair (0.63), person (0.6), person (0.93)
  frame #111, detection time 0.05063s, dining table (0.97), chair (0.61), person (0.6), person (0.91)
  frame #114, detection time 0.05047s, dining table (0.88), chair (0.82), person (0.58), person (0.85)

Pretty good, apart from seeing a ghost ?
 

robagar

Active member
Joined
Apr 17, 2021
Messages
30
Reaction score
16
jetson-tello 1.1.8

Mostly a tidy up, but also added a utility function run_jetson_tello_app which makes it very easy to fly a drone and CUDA process video frames, without falling behind the live video stream.

Here's an example - take off, look around and find faces...

Python:
#!/usr/bin/env python3

import asyncio
import jetson.inference
from jetson_tello import run_jetson_tello_app


face_detector = jetson.inference.detectNet("facenet", threshold=0.5)


def detect_faces(drone, frame, cuda):
    face_detections = face_detector.Detect(cuda)

    for d in face_detections:
        print(d)


async def fly(drone):
    await drone.takeoff()
    for i in range(4):
        await drone.turn_clockwise(90)
        await asyncio.sleep(3)
    await drone.land()


run_jetson_tello_app(fly, process_frame=detect_faces)
 
  • Like
Reactions: Hacky and AIDrone

Ivette

New member
Joined
Aug 2, 2021
Messages
1
Reaction score
0
Hello,

It's not working for me. This message appeared:

>>> from jetson_tello import h264_frame_to_cuda, NoFrame
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/nvidia/.local/lib/python3.6/site-packages/jetson_tello/__init__.py", line 1, in <module>
from .video import h264_frame_to_cuda, h264_frame_to_numpy_array, decoded_frame_to_cuda, decoded_frame_to_numpy_array, NoFrameData, DecodedFrame, H264DecoderAsync
ImportError: cannot import name 'decoded_frame_to_cuda'

Can you help me ?

Thanks
 

robagar

Active member
Joined
Apr 17, 2021
Messages
30
Reaction score
16
Hi @Ivette sorry I've been away for a while - I'm in the middle of pulling the video decoding parts out into their own package (tello-asyncio-video) and I must have left jetson-tello in a rather poor state.. Hopefully I'll be able to get back onto it over the weekend
 

New Threads

Members online

No members online now.

Forum statistics

Threads
5,446
Messages
38,870
Members
15,462
Latest member
DavidH