Hello Tello Pilot!
Join our DJI Tello community & remove this banner.
Sign up

Tello: autonomous flight in JavaScript

baka

New member
Joined
Aug 26, 2018
Messages
3
Reaction score
1
Hello everyone!

I just finished a project, which enables Tello to autonomously follow a person.
All control is done within the web browser leveraging the TensorFlow.js MoveNet model.

You can also control the drone with the keyboard, and watch/record drone video feed in the browser.


 
  • Like
Reactions: Hacky

Hacky

Well-known member
Joined
Jul 19, 2021
Messages
194
Reaction score
85
Location
Germany
Thanks for sharing.

In the video description you mention MoveNet for pose detection but most of the scenes in the video seem to use a more simple person detection (showing a pretty inaccurate bounding box). At the end of the video, there are some examples marking just the nose with a yellow spot (pretty accruate). I guess these examples were using MoveNet?

Would it be possible to run all components also on a mobile device at a usable framerate?

In my signature you find a link to my pose tracking / gesture control example. When you also use pose tracking, this opens nice options to control Tello behaviour by particular body poses.
 

Dan

Well-known member
Joined
Feb 20, 2020
Messages
185
Reaction score
69
Great program and video for a great little drone
 

jeffjeff

Member
Joined
Jul 14, 2021
Messages
17
Reaction score
3
Age
76
Location
Härnösand Sweden
Website
tupomedia.se
Is this something that can be done with a standard Tello? (or perhaps only the EDU model?)

Can a clean video be saved at the same time? (Without icons for battery, person detection etc)

NICE presentation!
 

baka

New member
Joined
Aug 26, 2018
Messages
3
Reaction score
1
Hey @Hacky

You are doing a great job with drone gesture control! Very impressive. Reminds me of the Minority Report movie. Perhaps it may lead to combining gesture reading with a dancing drone swarm reacting to human movements :)

To answer your questions:

I am actually using MoveNet for body coordinates detection. But the bounding box is drawn using derivatives of actual points (multiples to the ears coordinates to be precise), which may cause box shaking you noticed. Probably I have to use a better formula for the box.

The fragment with a yellow dot is one of the earliest experiments using PoseNet. And the dot is at the exact point returned from the model. I switched to MoveNet later since it is newer and less resource-consuming.

I would be glad to know your suggestions on what other simpler person detection models you mentioned could we try?

Running on mobile: since this project has a server part for video feed processing, it can not be entirely run on a mobile device in its current state.

Possible option: run a server connected to Tello on a machine simultaneously connected to Tello and a second network (or connect your Tello to an arbitrary AP, I can't do that with mine).
Then open a browser page on mobile connected to that network. The pose estimation is executed in the browser, so the heavy load will be shifted to the mobile device. I guess modern high-end smartphones should handle it perfectly.

Here are the official demos of MoveNet and PoseNet to estimate how they behave on mobile:
MoveNet:
PoseNet:
 

Iann Ortnau

Member
Joined
Dec 30, 2021
Messages
13
Reaction score
0
Please help me, I'm making an app in electron to control the tello from the computer, I need help to run away from ffmpeg because it generates a lot of delay how can I capture and display the video in js efficiently?
 

Members online

Forum statistics

Threads
5,378
Messages
38,505
Members
15,008
Latest member
Matkeez