Tello. Whats possible?

Krag · Apr 12, 2018

I tried to figure out how the joystick bits are packed into a message based on the packet captures I did. I got it to work but I am tired and I must be doing something wrong. The sticks seem to be centered at 1024 and range from 364 to -364. Which doesn't make any sense.

Code:

        short axis1=1024;//RightX center=1024 left =364 right =-364
        short axis2=1024;//RightY down =364 up =-364
        short axis3=1024;//LeftY down =364 up =-364
        short axis4=1024;//LeftX left =364 right =-364
        short axis5=1024;//Unknown.
      
        long paramLong1 = ((long)axis1 & 0x7FF) | (((long)axis2 & 0x7FF) << 11) | ((0x7FF & (long)axis3) << 22) | ((0x7FF & (long)axis4) << 33) | ((long)axis5 << 44);
        a[9] = ((byte)(int)(0xFF & paramLong1));
        a[10] = ((byte)(int)(paramLong1 >> 8 & 0xFF));
        a[11] = ((byte)(int)(paramLong1 >> 16 & 0xFF));
        a[12] = ((byte)(int)(paramLong1 >> 24 & 0xFF));
        a[13] = ((byte)(int)(paramLong1 >> 32 & 0xFF));
        a[14] = ((byte)(int)(paramLong1 >> 40 & 0xFF));
        Console.WriteLine(BitConverter.ToString(a).Replace("-",","));

Krag · Apr 12, 2018

Krag said:
. The sticks seem to be centered at 1024 and range from 364 to -364. Which doesn't make any sense.

That's what it is alright. Weird. Here is how you build a joystick packet from floating point values -1 to +1.

Creating Tello packets. |.NET Fiddle

Code:

    //Create joystick packet from floating point axis.
    //Center = 0.0.
    //Up/Right =1.0.
    //Down/Left=-1.0.
    public static byte[] createJoyPacket(float fRx,float fRy,float fLx,float fLy,float unk)
    {
        //template joy packet.
        var packet= new byte[]{0xcc ,0xb0 ,0x00 ,0x7f ,0x60 ,0x50 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x00 ,0x12 ,0x16 ,0x01 ,0x0e ,0x00 ,0x25 ,0x54};
       
        short axis1=(short)(660.0F * fRx + 1024.0F);//RightX center=1024 left =364 right =-364
        short axis2=(short)(660.0F * fRy + 1024.0F);//RightY down =364 up =-364
        short axis3=(short)(660.0F * fLy + 1024.0F);//LeftY down =364 up =-364
        short axis4=(short)(660.0F * fLx + 1024.0F);//LeftX left =364 right =-364
        short axis5=(short)(660.0F * unk + 1024.0F);//Unknown. Possibly camera controls.

        long packedAxis = ((long)axis1 & 0x7FF) | (((long)axis2 & 0x7FF) << 11) | ((0x7FF & (long)axis3) << 22) | ((0x7FF & (long)axis4) << 33) | ((long)axis5 << 44);
        packet[9] = ((byte)(int)(0xFF & packedAxis));
        packet[10] = ((byte)(int)(packedAxis >> 8 & 0xFF));
        packet[11] = ((byte)(int)(packedAxis >> 16 & 0xFF));
        packet[12] = ((byte)(int)(packedAxis >> 24 & 0xFF));
        packet[13] = ((byte)(int)(packedAxis >> 32 & 0xFF));
        packet[14] = ((byte)(int)(packedAxis >> 40 & 0xFF));

        //Add time info.      
        var now = DateTime.Now;
        packet[15] = (byte)now.Hour;
        packet[16] = (byte)now.Minute;
        packet[17] = (byte)now.Second;
        packet[18] = (byte)(now.Millisecond & 0xff);
        packet[19] = (byte)(now.Millisecond >> 8);
       
        calcUCRC(packet,4);//Not really needed.
       
        //calc crc for packet.
        calcCrc(packet,packet.Length);
       
        return packet;
    }

Krag · Apr 12, 2018

Success! I just flew around my office with nothing but a PC and an xbox controller.

bluejune · Apr 12, 2018

I had a chance to look into stick command [cmdID=80 (0x50)]

The axis range is between 363(left, down) and 1684(right, up) while center is 1024.
axis1 : aileron (RIGHT_X)
axis2 : elevator (RIGHT_Y)
axis3 : throttle (LEFT_Y)
axis4 : rudder (LEFT_X)
axis5 : 0 = slow mode, 1 = fast mode (only 4 bit is available for this axis)

Krag said:

I tried to figure out how the joystick bits are packed into a message based on the packet captures I did. I got it to work but I am tired and I must be doing something wrong. The sticks seem to be centered at 1024 and range from 364 to -364. Which doesn't make any sense.

Code:

        short axis1=1024;//RightX center=1024 left =364 right =-364
        short axis2=1024;//RightY down =364 up =-364
        short axis3=1024;//LeftY down =364 up =-364
        short axis4=1024;//LeftX left =364 right =-364
        short axis5=1024;//Unknown.
     
        long paramLong1 = ((long)axis1 & 0x7FF) | (((long)axis2 & 0x7FF) << 11) | ((0x7FF & (long)axis3) << 22) | ((0x7FF & (long)axis4) << 33) | ((long)axis5 << 44);
        a[9] = ((byte)(int)(0xFF & paramLong1));
        a[10] = ((byte)(int)(paramLong1 >> 8 & 0xFF));
        a[11] = ((byte)(int)(paramLong1 >> 16 & 0xFF));
        a[12] = ((byte)(int)(paramLong1 >> 24 & 0xFF));
        a[13] = ((byte)(int)(paramLong1 >> 32 & 0xFF));
        a[14] = ((byte)(int)(paramLong1 >> 40 & 0xFF));
        Console.WriteLine(BitConverter.ToString(a).Replace("-",","));

Krag · Apr 12, 2018

Krag said:
Success! I just flew around my office with nothing but a PC and an xbox controller.

And crashed(not bad)! If you try sending joy info make sure you include a dead band. And expect it to go out of control. My controls seem to stick sometimes. Probably something wrong on my side. But still. Be careful.

Krag · Apr 13, 2018

I posted my proof of concept C# code to github. I highly recommend NOT using this program unless you are prepared to crash your Tello.

Kragrathea/TelloPC

If you are brave.
-Compile with Visual Studio.
-Turn on Tello and connect the PC to the Tello network before running the program.
-Run program.
-Take off is Y on a 360 controller.
-Landing is A.

There is latency. And sometimes the controls stick. You have been warned.

Krag · Apr 13, 2018

I think my latency and sticking were primarily because of network problems. I was using a cheap dongle wifi. When I use it with my laptop it is smooth as silk. I have never flown it with a joystick before and I was blown away with how much easier it is to fly.

Sorry for the lousy video. Hard to hold a camera and fly.

deadprogram · Apr 13, 2018

Incredible effort from Krag here, thank you!

My Golang Tello package's initial functionality is now working thanks to the collective efforts on this forum.

Here is an example using a PS3 controller to fly the Tello:
hybridgroup/gobot

Next step, figure out how to get video, which it sounds like Krag and a couple of others already know?

deadprogram · Apr 13, 2018

npn said:
There is one more command (to be sent on 8889) needed to get a playable video stream:
0xcc, 0x58, 0x00, 0x7c, 0x60, 0x25, 0x00, 0x00, 0x00, 0x6c, 0x95
It will request an I-frame generation, together with the essential SPS/PPS info - Tello will send them on the video link, port 6038.

Sending it once at the beginning is enough, but could be better to send it periodically (let's say every second) to "repair" the stream in case of lost packets. I was able to play the stream with ffplay, by removing the two-byte header from every datagram and feeding them to the player.

DO you have any code that shows this?

Thank you!

deadprogram · Apr 13, 2018

bluejune said:
I also just found..
YES.. IT WORKS !!

Any code showing specifics of how to do this?

Krag · Apr 13, 2018

deadprogram said:
Next step, figure out how to get video, which it sounds like Krag and a couple of others already know?

I actually haven't got video working. In fact I cant reproduce the test I did below with Packet Sender. I don't know what happened, but I don't get video on 6038 anymore.

When I got video packets before.

deadprogram · Apr 13, 2018

Krag said:
I actually haven't got video working. In fact I cant reproduce the test I did below with Packet Sender. I don't know what happened, but I don't get video on 6038 anymore.

When I got video packets before.

You might have a typo here Kragrathea/TelloPC for the port?

Krag · Apr 13, 2018

I quickly changed it to 6138 before I updloaded because I didn't want it to start working and flood anyones screen with garbage. I should have comment out the listener. I don't think it works even if it is 6038. I would love to be proved wrong.

deadprogram · Apr 13, 2018

Krag said:
I quickly changed it to 6138 before I updloaded because I didn't want it to start working and flood anyones screen with garbage. I should have comment out the listener. I don't think it works even if it is 6038. I would love to be proved wrong.

I am getting the packets in my Go code:

Received 1460 bytes from 192.168.10.1:62512
frame received
Received 1460 bytes from 192.168.10.1:62512
frame received
Received 1460 bytes from 192.168.10.1:62512
frame received
Received 1460 bytes from 192.168.10.1:62512
frame received
Received 1460 bytes from 192.168.10.1:62512
frame received

However, I do not know what to do with the packets. My overall goal is to get the video into OpenCV's Mat type. Initial goal is just to see the video on my computer in any way.

Krag · Apr 13, 2018

I'd say it was something in my code but I can't get Packet Sender to work either. I'll keep at it but I am stumped at the moment.

deadprogram · Apr 13, 2018

Krag said:
I'd say it was something in my code but I can't get Packet Sender to work either. I'll keep at it but I am stumped at the moment.

Reboot everything

Krag · Apr 14, 2018

I give up. I am not getting video packets at all anymore. I get all other messages fine but nothing ever comes in on 6038. Either in my C# or Packet Sender. I don't know how you are managing it in Go. I am begining to think I never did have it working and it was just a fluke.

deadprogram · Apr 14, 2018

Krag said:
I give up. I am not getting video packets at all anymore. I get all other messages fine but nothing ever comes in on 6038. Either in my C# or Packet Sender. I don't know how you are managing it in Go. I am begining to think I never did have it working and it was just a fluke.

Do you also need to uncomment these lines to send the 'start video' command?

Kragrathea/TelloPC

deadprogram · Apr 14, 2018

I'm still trying to get the h264 video display working. I have the data, but not sure what to do with the packets...

Possibly use gstreamer? README\rtp\gst - gstreamer/gst-plugins-good - 'Good' GStreamer plugins

To do that would need the rtp "caps", it would seem. I ran "gst-launch-1.0" and started to get the packets from the Tello, but it was not able to decode correctly:

Code:

$ gst-launch-1.0 -v udpsrc port=6038 ! rtph264depay ! avdec_h264 ! videoconvert ! autovideosink sync=false             
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstRtpH264Depay:rtph264depay0: No RTP format was negotiated.
Additional debug info:
gstrtpbasedepayload.c(484): gst_rtp_base_depayload_handle_buffer (): /GstPipeline:pipeline0/GstRtpH264Depay:rtph264depa
y0:
Input buffers need to have RTP caps set on them. This is usually achieved by setting the 'caps' property of the upstrea
m source element (often udpsrc or appsrc), or by putting a capsfilter element before the depayloader and setting the 'c
aps' property on that. Also see http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/gst/rtp/README
Execution ended after 0:00:00.017995161
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...