Hello Tello Pilot!
Join our DJI Tello community & remove this banner.
Sign up

Unity Controller for Tello (3D Tracking from Takeoff)

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
29
Likes
22
#1
THIS IS ALPHA SOFTWARE. IT HAS NOT BEEN WELL TESTED, AND THE INPUTS HAVE NOT BEEN FINE TUNED. I TAKE NO RESPONSIBILITY FOR ANY DAMAGE THAT OCCURS TO YOUR TELLO WHILE USING THIS SOFTWARE. MAKE SURE TO VALIDATE INPUT VALUES IN FLIGHT CONTROLLER COMPONENT BEFORE TAKEOFF

This project would not be possible without the hard work put into these two libraries.
Krag's TelloLib : A c# library that allows the tello to be controlled via windows. It also exposes the underlying API which contains the positioning data.
comoc's TelloForUnity : Modifies TelloLib to work in Unity, including the video feed.

The goal of this project is to create the most accurate tracking possible, which will be the groundwork for fully autonomous flying. Right now you will need the Unity editor to use this project. Eventually maybe I will build an exe, but I much prefer using Unity inspectors instead of having to build UI's.

AUTONOMOUS FLYING (WIP)
Looking for testers/PID tuners.

3D TRACKING
I currently have tracking working in all 3 dimensions. After takeoff when the tello achieves a stable hover, press the space bar to begin tracking from that position. Unfortunately there is a large offset from the (0,0,0) takeoff position and when hover is achieved. The value after takeoff will read something crazy like (355,855,234) after launch. This is used as the offset once the space bar is hit, and the Unity tello will track from the global origin. The (0,0,0) position is set when you power the tello on, so make sure it is flat and pointed the direction you want to be forward when you power it on. I usually do this each time I want to do a tracking run, and have not experimented with tracking multiple flights without powering off and on at the launch point.

The quality of the tracking is extremely dependent on the amount of light, and even in good conditions the tracking will have random outliers. I try my best to limit this, but it may still happen. In the second video, there is a white plane which gets placed below the drone. This is representing Tello.state.height from TelloLib. This seems to be completely wrong though, as the values make no sense.

The native tello tracking is headless, meaning it does not take into account rotation. I have tried Tello.state.ToEuler() to try and get the craft rotation, but no luck yet. Once I have that working the 3rd person cam will be far more useful. If you want to replicate flying through a known environment, you need to measure and model the environment, and set the origin at the position you want to manually fly to.

TELLO SIMULATOR
This is the very beginning of a Tello simulator which will be used to test and tune autonomous flying before attempting it with the real thing. I have put very little tuning into this, and you can see the results are varied. The red block is the actual tello, and the yellow block is the simulated one. Anytime you hit Keypad0 the yellow will reset to the red position. There are two values on the DroneSimulator component, drag and inputDrag. Input Drag is the amount of drag being put on the drone when it is receiving any input, and drag is the amount of drag applied when no input is detected. These can be adjusted to try and get a more realistic performance from the simulator. There is a good chance this may never be perfect, and I may need to add more variables to tweak, but this could become a very powerful tool. To use you need to enable the "Drone Simulator" GameObject in the hierarchy.

VR SUPPORT
Not currently in the project but I will set it back up.
DJI Tello FPV with Oculus Rift / Touch input

INPUT MANAGER (TUNING & JOYSTICK SUPPORT)
All inputs go through the input manager. This is where you can fine tune your selected input. This provides a way to tune your keyboard input, or set up a flight stick. Sensitivity and Deadzone are self explanitory. Gravity is only for the keyboard, and determines how quickly the value returns to 0. Feel free to suggest input values you find. Input manager can be found at Edit - Project Settings - Input. Right now the only flightstick supported is the thrustmaster 1600m and throttle. If you have a joystick you would like to use, I can let you know how to set it up.

INSTALLATION
Download Unity Editor 2018 (shouldn't matter version, I try to stay on the latest)
Download UnityControllerForTello HERE
Open Unity and select open project. Navigate to folder you just downloaded.
Allow Unity through firewall via inbound rules Image Example

TAKEOFF
Connect to tello wifi
Press play in editor
Perform pre-flight checks
Wait for video feed
Flight instructions located top left corner
 
Last edited:

trelo

Well-known member
Joined
Apr 2, 2018
Messages
56
Likes
9
#2
Dude, wow.

This looks hyper complicated.
I am amazed that you are creating smth I cannot even comprehend.

:)

Keep up the good work.
 
Likes: VR_Dev

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
29
Likes
22
#3
So I've been trying to make videos but this project is progressing pretty quickly. I'm already testing the autopilot algorithm on the real thing. I have the simulator up and running as a full test bed for development. The scene controller now has the option to switch between flying and simulating. I haven't tried the "Fly&Sim" mode lately so I would stay away from that.

First, I have craft rotation working which makes the third person view far more helpful. This video is the first attempt at flying through a replica of my office my coworker Dwight had on hand. The tracking doesn't start at the most accurate point, and it drops a lot, but you can see the 3rd person craft match the tello rotation.

Here you can see me testing headless flight via keyboard input, then Fly-To-Target Autopilot in the simulator, before giving it a try on the real thing. I have it working fully in the simulator, but the real thing needs some work. I'm pretty sure I am giving the PID controllers the wrong delta time between tello updates. I expect to have this fixed pretty quickly, and then I will be able to set a course for the tello to follow.

So not many people cloned the repo, but if you did; I deleted the whole thing and replaced it. There was something missing in the git ignore so I just started from scratch. If you want to update your repo it's probably just best to destroy it and pull again. Everyone else it's good to go. Just don't play with the autopilot yet with the real thing. Unity may seem intimidating but I'd be happy to talk anyone through it if they want to try it themselves.

Dude, wow.

This looks hyper complicated.
I am amazed that you are creating smth I cannot even comprehend.

:)

Keep up the good work.
Thanks a lot. I know you wont believe me but mapping the tracking wasn't very hard. The guys who deciphered all the low level commands for position deserve most of the credit.
 
Last edited:
Likes: Cliffs tello

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
29
Likes
22
#4
Fly-To-Target autopilot testing. PID's seemed to be tuned ok, but they have a hard time fine tuning the tello's position at close range. I've even built a system to update the PID values after the tello has been in close proximity to the target for 3 seconds. When in range I crank up the I value but still not much luck. I will have to do more testing.

My tello seems to be far less stable than when I first bought it. It has taken a fair amount of crashes already, but hopefully I just need new props.


One thing that would help this project dramatically is being able to start the props and take off manually. Other libraries support this function, and the creator of aTelloPilot says this is the cmd that needs to be sent to the tello.

Code:
User Datagram Protocol, Src Port: 6525, Dst Port: 8889
TELLO_CMD
    SOP   : 0xcc
    SIZE  : 176
    CRC8  : 0x7f
    PACT  : 0x60
    DIR   :  -> TO DRONE
    CMD   : TELLO_CMD_STICK (80)
    SEQ   : 0
    DATASZ: 11
    DATA  : 6c 61 0b 5b 28 0d 06 13 21 45 02
        STICK - roll: 364, pitch: 364, thr: 364, yaw:1684, fastmode:0
    CRC16 : 0x7cec
I tried converting that Data packet to this which was also unsuccessful. I kind of just guessed at the conversion to tellolibs packet values.

C#:
public static void StartMotors()
{
    var packet = new byte[] { 0x6c, 0x61, 0x0b, 0x5b, 0x28, 0x0d, 0x06, 0x13, 0x21, 0x45, 0x02, };
    setPacketSequence(packet);
    setPacketCRCs(packet);
    client.Send(packet);
}
Update:

Took a crack at the circle flying. This is the tello following a moving target, instead of jumping from stationary target to stationary target. I think I am going to build a system where the target drone flies from point to point, and the actual tello's only job is to try and match the target drone's position.

This will give me much more control over its flightpath, and will make tuning the PIDs easier by eliminating the large range of inputs.

 
Last edited:
Likes: Cliffs tello

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
29
Likes
22
#6
making a drone simulation in 10 mins

Thanks I'll take a look. Bought a mavik air so I've been pretty distracted with that. This weekend I'll make some progress. Before I got the air I already figured out how to start the props via tellolib, unfortunately it did not solve the takeoff offset problem, but I believe I have found a way to track accurately from takeoff which improves the overall tracking quality through an environment.
 

VR_Dev

Active member
Joined
Aug 19, 2018
Messages
29
Likes
22
#7
So like I said I figured out how to start the props manually, but there is still an offset when the tello finds tracking. The tello's tracking isn't accurate until it is about half a meter off the ground. Once it reaches a certain height tracking begins, but there is a random offset applied. I use to handle this by taking off, flying to a known point, then manually enabling tracking taking into account the offset. This worked, but was only as accurate as I could fly to an imaginary point in the air.

Now I track from takeoff, but monitor the tello's tracking for the offset. Once the offset is reported, I begin tracking. By taking the known offset into account, I can keep (0,0,0) as the origin point. I also use the tello's height data to adjust for the elevation the tello was at when tracking began. The offset and height data allow me to track the tello accurately from the position tracking was established.


This is the most impressive tracking I've gotten. Unfortunately this is not a consistent result, even with these short flights. I think there are some tricks I can use to squeeze out some more accuracy though.
 
Joined
Dec 4, 2018
Messages
15
Likes
6
#8
Wow! This is amazing... who knew you could do this with suck a little "toy" drone? Amazing job!
 
Likes: VR_Dev

New Threads

Forum statistics

Threads
1,826
Messages
12,220
Members
4,252
Latest member
komninos