Hello Tello Pilot!
Join our DJI Tello community & remove this banner.
Sign up

Unity Controller for Tello (Autopilot 2.0)

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
33
Reaction score
30
THIS IS ALPHA SOFTWARE. IT HAS NOT BEEN WELL TESTED, AND THE INPUTS HAVE NOT BEEN FINE TUNED. I TAKE NO RESPONSIBILITY FOR ANY DAMAGE THAT OCCURS TO YOUR TELLO WHILE USING THIS SOFTWARE. MAKE SURE TO VALIDATE INPUT VALUES IN FLIGHT CONTROLLER COMPONENT BEFORE TAKEOFF

This project would not be possible without the hard work put into these two libraries.
Krag's TelloLib : A c# library that allows the tello to be controlled via windows. It also exposes the underlying API which contains the positioning data.
comoc's TelloForUnity : Modifies TelloLib to work in Unity, including the video feed.


So I completely reworked this project from the ground up. I optimized and simplified everything, added tracking from takeoff, re-added rift support, and completely reworked the autopilot system. Instead of giving the tello a target point to fly to, the tello's only job is to try and match a target tello's position(Blue quad in the video). It's my theory that this will be easier for the PIDs to handle because there is a smaller range of inputs to deal with.

With the tello flying itself you are free to move the target with either the gizmos, my fly-to-target system, or via a custom script. The simulator has also been updated so you can test complex autopilot features without destroying your tello or waking the neighbors.

I have spent little time tuning the autopilot PIDs, so there is a lot of room for improvement there. The autopilot loads a PID profile which can be found in the hierarchy. You can update those values or create your own PIDProfile component. You can also swap PID profiles in flight for different situations. The tellos performance in the simulator will not match the real thing, so don't use the simulator to tune the pids.

What you see in the video above is the same project in the github repo. 'T' and 'L' to takeoff and land, 'P' to toggle autopilot on and off. (Any input to the flight controls will also deactivate the autopilot). With the autopilot active you can either move the target with the editor gizmo, or drag a point into the Autopilot component. Autopilot will also automatically deactivate when tracking is lost. YOU MUST BE FOCUSED ON A GAME WINDOW FOR INPUT TO WORK

3D TRACKING



VR SUPPORT
FPV supported for the Rift and VIve. Rift can use touch controllers but Vive will need joystick or keyboard inupt. To use VR, you need to enable it in the project settings.

INPUT MANAGER (TUNING & JOYSTICK SUPPORT)
All inputs go through the input manager. This is where you can fine tune your selected input. This provides a way to tune your keyboard input, or set up a flight stick. Sensitivity and Deadzone are self explanitory. Gravity is only for the keyboard, and determines how quickly the value returns to 0. Feel free to suggest input values you find. Input manager can be found at Edit - Project Settings - Input. Right now the only flightstick supported is the thrustmaster 1600m and throttle. If you have a joystick you would like to use, I can let you know how to set it up.

INSTALLATION
Download Unity Editor 2018.3
Download UnityControllerForTello HERE
Open Unity and select open project. Navigate to folder you just downloaded.
Allow Unity through firewall via inbound rules Image Example

TAKEOFF
Connect to tello wifi
Press play in editor
Perform pre-flight checks
Wait for video feed
Flight instructions located top left corner
 
Last edited:

trelo

Well-known member
Joined
Apr 2, 2018
Messages
68
Reaction score
12
Dude, wow.

This looks hyper complicated.
I am amazed that you are creating smth I cannot even comprehend.

:)

Keep up the good work.
 
  • Like
Reactions: VR_Dev

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
33
Reaction score
30
So I've been trying to make videos but this project is progressing pretty quickly. I'm already testing the autopilot algorithm on the real thing. I have the simulator up and running as a full test bed for development. The scene controller now has the option to switch between flying and simulating. I haven't tried the "Fly&Sim" mode lately so I would stay away from that.

First, I have craft rotation working which makes the third person view far more helpful. This video is the first attempt at flying through a replica of my office my coworker Dwight had on hand. The tracking doesn't start at the most accurate point, and it drops a lot, but you can see the 3rd person craft match the tello rotation.

Here you can see me testing headless flight via keyboard input, then Fly-To-Target Autopilot in the simulator, before giving it a try on the real thing. I have it working fully in the simulator, but the real thing needs some work. I'm pretty sure I am giving the PID controllers the wrong delta time between tello updates. I expect to have this fixed pretty quickly, and then I will be able to set a course for the tello to follow.

So not many people cloned the repo, but if you did; I deleted the whole thing and replaced it. There was something missing in the git ignore so I just started from scratch. If you want to update your repo it's probably just best to destroy it and pull again. Everyone else it's good to go. Just don't play with the autopilot yet with the real thing. Unity may seem intimidating but I'd be happy to talk anyone through it if they want to try it themselves.

Dude, wow.

This looks hyper complicated.
I am amazed that you are creating smth I cannot even comprehend.

:)

Keep up the good work.
Thanks a lot. I know you wont believe me but mapping the tracking wasn't very hard. The guys who deciphered all the low level commands for position deserve most of the credit.
 
Last edited:
  • Like
Reactions: Cliffs tello

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
33
Reaction score
30
Fly-To-Target autopilot testing. PID's seemed to be tuned ok, but they have a hard time fine tuning the tello's position at close range. I've even built a system to update the PID values after the tello has been in close proximity to the target for 3 seconds. When in range I crank up the I value but still not much luck. I will have to do more testing.

My tello seems to be far less stable than when I first bought it. It has taken a fair amount of crashes already, but hopefully I just need new props.


One thing that would help this project dramatically is being able to start the props and take off manually. Other libraries support this function, and the creator of aTelloPilot says this is the cmd that needs to be sent to the tello.

Code:
User Datagram Protocol, Src Port: 6525, Dst Port: 8889
TELLO_CMD
    SOP   : 0xcc
    SIZE  : 176
    CRC8  : 0x7f
    PACT  : 0x60
    DIR   :  -> TO DRONE
    CMD   : TELLO_CMD_STICK (80)
    SEQ   : 0
    DATASZ: 11
    DATA  : 6c 61 0b 5b 28 0d 06 13 21 45 02
        STICK - roll: 364, pitch: 364, thr: 364, yaw:1684, fastmode:0
    CRC16 : 0x7cec
I tried converting that Data packet to this which was also unsuccessful. I kind of just guessed at the conversion to tellolibs packet values.

C#:
public static void StartMotors()
{
    var packet = new byte[] { 0x6c, 0x61, 0x0b, 0x5b, 0x28, 0x0d, 0x06, 0x13, 0x21, 0x45, 0x02, };
    setPacketSequence(packet);
    setPacketCRCs(packet);
    client.Send(packet);
}
Update:

Took a crack at the circle flying. This is the tello following a moving target, instead of jumping from stationary target to stationary target. I think I am going to build a system where the target drone flies from point to point, and the actual tello's only job is to try and match the target drone's position.

This will give me much more control over its flightpath, and will make tuning the PIDs easier by eliminating the large range of inputs.

 
Last edited:
  • Like
Reactions: Cliffs tello

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
33
Reaction score
30
making a drone simulation in 10 mins

Thanks I'll take a look. Bought a mavik air so I've been pretty distracted with that. This weekend I'll make some progress. Before I got the air I already figured out how to start the props via tellolib, unfortunately it did not solve the takeoff offset problem, but I believe I have found a way to track accurately from takeoff which improves the overall tracking quality through an environment.
 

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
33
Reaction score
30
So like I said I figured out how to start the props manually, but there is still an offset when the tello finds tracking. The tello's tracking isn't accurate until it is about half a meter off the ground. Once it reaches a certain height tracking begins, but there is a random offset applied. I use to handle this by taking off, flying to a known point, then manually enabling tracking taking into account the offset. This worked, but was only as accurate as I could fly to an imaginary point in the air.

Now I track from takeoff, but monitor the tello's tracking for the offset. Once the offset is reported, I begin tracking. By taking the known offset into account, I can keep (0,0,0) as the origin point. I also use the tello's height data to adjust for the elevation the tello was at when tracking began. The offset and height data allow me to track the tello accurately from the position tracking was established.


This is the most impressive tracking I've gotten. Unfortunately this is not a consistent result, even with these short flights. I think there are some tricks I can use to squeeze out some more accuracy though.
 
  • Like
Reactions: sainmaoj and nahuel

nahuel

Well-known member
Joined
Dec 4, 2018
Messages
61
Reaction score
34
Wow! This is amazing... who knew you could do this with suck a little "toy" drone? Amazing job!
 
  • Like
Reactions: VR_Dev

bataurus

New member
Joined
Dec 27, 2018
Messages
2
Reaction score
3
Just got a Tello and I have developed a few mobile apps and working on a 2D platformer. I just pulled down the code. Very interesting.
 
  • Like
Reactions: VR_Dev and nahuel

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
33
Reaction score
30
Just got a Tello and I have developed a few mobile apps and working on a 2D platformer. I just pulled down the code. Very interesting.
Cool thanks. Just FYI for everyone this has fallen pretty far behind with my Mavic air and Christmas. Expect a major update in the next couple days with the tracking from takeoff and autopilot updates. Also VR support.
 

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
33
Reaction score
30
So I completely reworked this project from the ground up. I optimized and simplified everything, added tracking from takeoff, re-added rift support, and completely reworked the autopilot system. Instead of giving the tello a target point to fly to, the tello's only job is to try and match the a target tello's position(Blue quad in the video). It's my theory that this will be easier for the PIDs to handle because there is a smaller range of inputs to deal with.

With the tello flying itself you are free to move the target with either the gizmos, my fly-to-target system, or via a custom script. The simulator has also been updated so you can test complex autopilot features without destroying your tello or waking the neighbors.

I have spent little time tuning the autopilot PIDs, so there is a lot of room for improvement there. The autopilot loads a PID profile which can be found in the hierarchy. You can update those values or create your own PIDProfile component. You can also swap PID profiles in flight for different situations. The tellos performance in the simulator will not match the real thing, so don't use the simulator to tune the pids.

What you see in the video is the same project in the github repo. 'T' and 'L' to takeoff and land, 'P' to toggle autopilot on and off. (Any input to the flight controls will also deactivate the autopilot). With the autopilot active you can either drag transforms into the target position of the autopilot, or assign them via script. Autopilot will also automatically deactivate when tracking is lost. YOU MUST BE FOCUSED ON A GAME WINDOW FOR INPUT TO WORK

 
Last edited:

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
33
Reaction score
30
Wow...!!!!
So Please make this aviable for Android too!!!!!!!!!!!!!!!!
Sorry man I don't know what to tell you. I could compile this to an android app but I just don't see how that would work. The point of this is kind of be a platform for people to write complex autopilot code.

It's pretty in depth and I just can't see how it could translate to mobile. If there is some specific feature you would want let me know and I'll think about it.
 

New Threads

Members online

Forum statistics

Threads
2,613
Messages
20,413
Members
6,232
Latest member
Brplrp