Tello. Whats possible?

Krag · Apr 7, 2018

Hmm. That is strange. I haven't actually parsed one yet myself. Maybe you have to be flying?

From tracing the code I see that is the way the HS and H speed are set in the app. Are those two at least working?

deadprogram · Apr 8, 2018

I'm not using the app, just Go code, so whatever commands those are, they are not being sent my my code as yet. Which possibly explains why so many of those numbers are empty.

The only API that is documented to perform movement is the "command" API, which is not the same API used by the mobile app. You cannot receive the normal UDP notificatiosn when in command mode, and I have yet to find a command which allows you to exit the command mode, and presumably get back to receiving the "normal" UDP notiifications.

Krag · Apr 8, 2018

I have confirmed that the structure is largely zeroed even when flying. That is too bad, it looked really cool. Maybe it is possible to turn those values on some how. I noticed there is a factoryMode.

I think the height and speed values are working though. You can test that by just moving the tello around by hand.

Sorry I haven't tested this myself. I am trying to get a C# program working and for some reason I am not able to receive any UDP messages either on 9000 or 6525. I must be doing something stupid because the python app works fine.

deadprogram · Apr 8, 2018

Krag said:
I have confirmed that the structure is largely zeroed even when flying. That is too bad, it looked really cool. Maybe it is possible to turn those values on some how. I noticed there is a factoryMode.

I think the height and speed values are working though. You can test that by just moving the tello around by hand.

Sorry I haven't tested this myself. I am trying to get a C# program working and for some reason I am not able to receive any UDP messages either on 9000 or 6525. I must be doing something stupid because the python app works fine.

I my case, I needed to make sure that the local port that I was expecting was bound to the response sent from the request sent to the remote port. In Go this is done by setting a local address as well as a remote address net - The Go Programming Language

Keep in mind that I have not done any C# in quite a while, so, YMMV.

But perhaps you need to use this constructor with port 6625: UdpClient Constructor (System.Net.Sockets)

And then use the drone's remote host for the .NET Send() method.

Anyhow, I am just guessing from my own needs in Go, which pretty much calls the same network stack "under the hood".

Krag · Apr 8, 2018

I found the problem. I was playing FarCry 5 and it was using the same port(s). Doh!

I see what you mean about being limited by not having "command" available. You can't do much without being able to take off. The Tello only sends 4 cmdid types when in this state (see below). I am going to have to dig into tcpdump captures to figure out the other side of the conversation.

Code:

                { 26, "Wifi" },//2 bytes. Strength, Disturb.
                { 53, "Light" },//1 byte or bit?
                { 86, "FlyData" },//Packed flydata struct.
                { 4176, "Data" },//large uncompressed data thats always the same. wtf?

Also. I was looking at the FlyData packet. A reason why it looks so empty is that some of the values in there like ground speed are only calculated when it has a good view of the floor. I have learned a lot by moving it around by hand even if the motors are off.

bluejune · Apr 9, 2018

Here are my findings.

•Message (UDP8889)
◦Connection Request
•conn_req + video port no.(u16) : 11 bytes to drone
•conn_ack + video port no.(u16) : 11 bytes from drone

◦General Packets
0 => 0xCC, (starting byte)
1, 2 => packet size(13bit) = ([2] << 8) | ([1] >> 3)
3 => CRC8 (from [0] to [2])
4 => packet type
5, 6 => cmd ID, ([6] << 8) | ([5])
7, 8 => seq No, ([8] << 8) | ([7])
data ....
N-1, N=> CRC16 (from [0] to [N-2])

•TimeStamp packet (to drone)
◦cmdID : 80 (0x0050)
◦Sent every 20ms
◦Packet size = (0x00b0 >> 3 = 22bytes)
◦Packet type = 0x60
◦Sequence No. = 0x0000 (is not increasing)
◦Data
•Unknown data (6bytes)
•TimeStamp
◦Hour (u8), Min (u8), Sec (u8), milliSec (u16)

•Video (UDP6038)
◦Video data is h.264 encoded (920x720 resolution)
◦But SPS (seq_parameter_set) and PPS(pic_parameter_set) do not exist
◦SPS and PPS seems to be delivered through the message packet

◦ 0 => SeqID
◦ 1 => Sub SeqID
◦ 2 ~ N => Video data
◦Seq ID
•Is increasing at every slice
◦Sub SeqID
•data frame number in a same slice (0x00 ~ 0x0A, 0x8B)
•Is increasing at every data frame in a same slice
•MSB7 is set when a slice data ends (0x8B)

Krag · Apr 9, 2018

Very nice! I will have to spend some time parsing that. One blank I can fill in. The 6 bytes of unknown data in cmd 80 is joystick data. Seems to be 5 axis with 11 bits each.

Code:

paramLong1 = axis1 & 0x7FF | (axis2 & 0x7FF) << 11 | (0x7FF & axis3) << 22 | (0x7FF & axis4) << 33 | axis5 << 44;
    this.a[9] = ((byte)(int)(0xFF & paramLong1));
    this.a[10] = ((byte)(int)(paramLong1 >>> 8 & 0xFF));
    this.a[11] = ((byte)(int)(paramLong1 >>> 16 & 0xFF));
    this.a[12] = ((byte)(int)(paramLong1 >>> 24 & 0xFF));
    this.a[13] = ((byte)(int)(paramLong1 >>> 32 & 0xFF));
    this.a[14] = ((byte)(int)(paramLong1 >>> 40 & 0xFF));

I dont know what a SPS or a PPS would look like. Could that be the large amount of data sent regularly to 6525?

In addition to the joystick info in cmd 80 I figured out that sending cmdId 84 and 85 should start take off/landing respectively. But so far all it does for me is start the motors briefly.

deadprogram · Apr 9, 2018

Wow, great work from everyone here.

Krag, just wondering if cmd 0x50 (80) needs to be sent repeatedly on takeoff to serve as a "heartbeat" or else the drone is not enabled for flight? The Parrot drone, and others, have a similar requirement.

RawrGuthlaf · Apr 9, 2018

bluejune said:
Here are my findings.

•Message (UDP8889)
◦Connection Request
•conn_req + video port no.(u16) : 11 bytes to drone
•conn_ack + video port no.(u16) : 11 bytes from drone

◦General Packets
0 => 0xCC, (starting byte)
1, 2 => packet size(13bit) = ([2] << 8) | ([1] >> 3)
3 => CRC8 (from [0] to [2])
4 => packet type
5, 6 => cmd ID, ([6] << 8) | ([5])
7, 8 => seq No, ([8] << 8) | ([7])
data ....
N-1, N=> CRC16 (from [0] to [N-2])

•TimeStamp packet (to drone)
◦cmdID : 80 (0x0050)
◦Sent every 20ms
◦Packet size = (0x00b0 >> 3 = 22bytes)
◦Packet type = 0x60
◦Sequence No. = 0x0000 (is not increasing)
◦Data
•Unknown data (6bytes)
•TimeStamp
◦Hour (u8), Min (u8), Sec (u8), milliSec (u16)

•Video (UDP6038)
◦Video data is h.264 encoded (920x720 resolution)
◦But SPS (seq_parameter_set) and PPS(pic_parameter_set) do not exist
◦SPS and PPS seems to be delivered through the message packet

◦ 0 => SeqID
◦ 1 => Sub SeqID
◦ 2 ~ N => Video data
◦Seq ID
•Is increasing at every slice
◦Sub SeqID
•data frame number in a same slice (0x00 ~ 0x0A, 0x8B)
•Is increasing at every data frame in a same slice
•MSB7 is set when a slice data ends (0x8B)

Have you successfully decoded the video to come to these conclusions? Regardless I'm grateful, I'm going to see if I can make a mock Android app or a java desktop app that uses ffmpeg libraries to connect since I should be able to pass in sps/pps values. It seems like that is precisely what Ryze is doing anyway.

And a shout out to all of you, lots of good info to start programming with, thank you. I have zero experience with drones so you have been very helpful with your deductions.

bluejune · Apr 9, 2018

RawrGuthlaf said:
Have you successfully decoded the video to come to these conclusions? Regardless I'm grateful, I'm going to see if I can make a mock Android app or a java desktop app that uses ffmpeg libraries to connect since I should be able to pass in sps/pps values. It seems like that is precisely what Ryze is doing anyway.

And a shout out to all of you, lots of good info to start programming with, thank you. I have zero experience with drones so you have been very helpful with your deductions.

No…
It was not successful to decode the video even with a created sps.
With decompiled apk codes, I can see some codes to classify video frames..
Need to check the decompiled codes more but it is hard to understand them because most of functions are decompiled as a, b, c... etcs.

RawrGuthlaf · Apr 9, 2018

bluejune said:
No…
It was not successful to decode the video even with a created sps.
With decompiled apk codes, I can see some codes to classify video frames..
Need to check the decompiled codes more but it is hard to understand them because most of functions are decompiled as a, b, c... etcs.

Ahh okay, I was hoping they weren't scrambling things that badly. Hopefully through trial and error I can discern a basic pattern.

Krag · Apr 9, 2018

deadprogram said:
Wow, great work from everyone here.

Krag, just wondering if cmd 0x50 (80) needs to be sent repeatedly on takeoff to serve as a "heartbeat" or else the drone is not enabled for flight? The Parrot drone, and others, have a similar requirement.

I think that is probably right. Either 80 or one of the other messages that is sent regularly. I think the reason the drone sends (my app anyway) the same large chunk of data over and over is because it is expecting a ack before sending the next chunk.

I didn't have time to work on it today. I got some new batteries in the mail and went flying.

npn · Apr 10, 2018

There is one more command (to be sent on 8889) needed to get a playable video stream:
0xcc, 0x58, 0x00, 0x7c, 0x60, 0x25, 0x00, 0x00, 0x00, 0x6c, 0x95
It will request an I-frame generation, together with the essential SPS/PPS info - Tello will send them on the video link, port 6038.

Sending it once at the beginning is enough, but could be better to send it periodically (let's say every second) to "repair" the stream in case of lost packets. I was able to play the stream with ffplay, by removing the two-byte header from every datagram and feeding them to the player.

bluejune · Apr 10, 2018

npn said:
There is one more command (to be sent on 8889) needed to get a playable video stream:
0xcc, 0x58, 0x00, 0x7c, 0x60, 0x25, 0x00, 0x00, 0x00, 0x6c, 0x95
It will request an I-frame generation, together with the essential SPS/PPS info - Tello will send them on the video link, port 6038.

Sending it once at the beginning is enough, but could be better to send it periodically (let's say every second) to "repair" the stream in case of lost packets. I was able to play the stream with ffplay, by removing the two-byte header from every datagram and feeding them to the player.

I also just found..
YES.. IT WORKS !!

Krag · Apr 10, 2018

Oh that is so Awesome!!! Great Job!

Edit: Now I am looking at that message (0x25) in my captures. It was one of my candidates for a heartbeat but it was too erratic. The app sometimes sends that message 3 or 4 times per second. Other times it will go up to 3 seconds between them. Maybe it is based on signal strength. But 4 iframes per second seems excessive. I wonder if that is why the video stutters so bad sometimes.

deadprogram · Apr 10, 2018

Does anyone have any code online that shows this?

bluejune when you say "feed to the player" do you mean piping output via stdout/stdin to the ffplayer command line application?

This is all very exciting, I want to do it too!

deadprogram · Apr 11, 2018

Krag said:
Very nice! I will have to spend some time parsing that. One blank I can fill in. The 6 bytes of unknown data in cmd 80 is joystick data. Seems to be 5 axis with 11 bits each.

Code:

paramLong1 = axis1 & 0x7FF | (axis2 & 0x7FF) << 11 | (0x7FF & axis3) << 22 | (0x7FF & axis4) << 33 | axis5 << 44; this.a[9] = ((byte)(int)(0xFF & paramLong1)); this.a[10] = ((byte)(int)(paramLong1 >>> 8 & 0xFF)); this.a[11] = ((byte)(int)(paramLong1 >>> 16 & 0xFF)); this.a[12] = ((byte)(int)(paramLong1 >>> 24 & 0xFF)); this.a[13] = ((byte)(int)(paramLong1 >>> 32 & 0xFF)); this.a[14] = ((byte)(int)(paramLong1 >>> 40 & 0xFF));

I dont know what a SPS or a PPS would look like. Could that be the large amount of data sent regularly to 6525?

In addition to the joystick info in cmd 80 I figured out that sending cmdId 84 and 85 should start take off/landing respectively. But so far all it does for me is start the motors briefly.

Any indication of what each axis controls, and also the range of permitted values for each axis? This would be very important info to know in advance, since incorrectly values are of course likely to cause a crash, and might take a while using trial and error to determine.

Krag · Apr 11, 2018

Makes sense. I'll try to figure it out tonight.

deadprogram · Apr 11, 2018

Krag said:
Makes sense. I'll try to figure it out tonight.

Thanks!

Krag · Apr 11, 2018

Here is a packet dump for each joystick direction. I don't have the time tonight to figure out the bits.

Code:

cc b0 00 7f 60 50 00 00 00 00 04 20 00 01 08 12 16 01 0e 00 25 54 //Both centered

cc b0 00 7f 60 50 00 00 00 00 04 20 a5 01 08 12 16 00 29 03 2a b8 //Left up
cc b0 00 7f 60 50 00 00 00 00 04 20 5b 00 08 12 16 04 9b 02 5f 2f   //Left down
cc b0 00 7f 60 50 00 00 00 00 04 20 00 d9 02 12 16 14 91 02 ba 52 //Left left
cc b0 00 7f 60 50 00 00 00 00 04 20 00 29 0d 12 16 1b 6b 01 46 01 //Left right

cc b0 00 7f 60 50 00 00 00 00 a4 34 00 01 08 12 22 04 ab 01 7a 5d// Right up
cc b0 00 7f 60 50 00 00 00 00 64 0b 00 01 08 12 22 10 7b 02 26 bb//Right down
cc b0 00 7f 60 50 00 00 00 6c 01 20 00 01 08 12 22 1b e8 02 44 53//Right left
cc b0 00 7f 60 50 00 00 00 94 06 20 00 01 08 12 22 22 e7 01 29 4c//Right right