Tello: autonomous flight in JavaScript

[baka]

Hello everyone!

I just finished a project, which enables Tello to autonomously follow a person.
All control is done within the web browser leveraging the TensorFlow.js MoveNet model.

You can also control the drone with the keyboard, and watch/record drone video feed in the browser.

GitHub - bakoushin/tello-autonomous-drone: Tello drone can follow a person with zero human assistance

Tello drone can follow a person with zero human assistance - GitHub - bakoushin/tello-autonomous-drone: Tello drone can follow a person with zero human assistance

github.com

 

[Hacky]

Thanks for sharing.

In the video description you mention MoveNet for pose detection but most of the scenes in the video seem to use a more simple person detection (showing a pretty inaccurate bounding box). At the end of the video, there are some examples marking just the nose with a yellow spot (pretty accruate). I guess these examples were using MoveNet?

Would it be possible to run all components also on a mobile device at a usable framerate?

In my signature you find a link to my pose tracking / gesture control example. When you also use pose tracking, this opens nice options to control Tello behaviour by particular body poses.

 

[Dan]

Great program and video for a great little drone

 

[jeffjeff]

Is this something that can be done with a standard Tello? (or perhaps only the EDU model?)

Can a clean video be saved at the same time? (Without icons for battery, person detection etc)

NICE presentation!

 

[baka]

Hey @Hacky

You are doing a great job with drone gesture control! Very impressive. Reminds me of the Minority Report movie. Perhaps it may lead to combining gesture reading with a dancing drone swarm reacting to human movements

To answer your questions:

I am actually using MoveNet for body coordinates detection. But the bounding box is drawn using derivatives of actual points (multiples to the ears coordinates to be precise), which may cause box shaking you noticed. Probably I have to use a better formula for the box.

The fragment with a yellow dot is one of the earliest experiments using PoseNet. And the dot is at the exact point returned from the model. I switched to MoveNet later since it is newer and less resource-consuming.

I would be glad to know your suggestions on what other simpler person detection models you mentioned could we try?

Running on mobile: since this project has a server part for video feed processing, it can not be entirely run on a mobile device in its current state.

Possible option: run a server connected to Tello on a machine simultaneously connected to Tello and a second network (or connect your Tello to an arbitrary AP, I can't do that with mine).
Then open a browser page on mobile connected to that network. The pose estimation is executed in the browser, so the heavy load will be shifted to the mobile device. I guess modern high-end smartphones should handle it perfectly.

Here are the official demos of MoveNet and PoseNet to estimate how they behave on mobile:
MoveNet:

https://storage.googleapis.com/tfjs-models/demos/pose-detection/index.html?model=movenet

PoseNet:

PoseNet - Camera Feed Demo

 

[Iann Ortnau]

Please help me, I'm making an app in electron to control the tello from the computer, I need help to run away from ffmpeg because it generates a lot of delay how can I capture and display the video in js efficiently?

 

[littlepaw]

Hello everyone!

I just finished a project, which enables Tello to autonomously follow a person.
All control is done within the web browser leveraging the TensorFlow.js MoveNet model.

You can also control the drone with the keyboard, and watch/record drone video feed in the browser.

GitHub - bakoushin/tello-autonomous-drone: Tello drone can follow a person with zero human assistance

Tello drone can follow a person with zero human assistance - GitHub - bakoushin/tello-autonomous-drone: Tello drone can follow a person with zero human assistance

github.com

dear baka,I need some guidance regarding autonomous drone navigation to give the starting location and a goal,asking the drone to reach the goal autonomously.i need someone to guid me for the first steps.also the platform is ROS