Tello localization using downward and frontal cameras, landmarks map, and a particle filter

[pgminin]

Monte Carlo Localization for Tello

Tello downward camera is used for motion estimation:
position is integrated from optical flow velocity estimation (dead-reckoning).

Tello frontal camera is used for observation estimation:
landmarks (bottles) are detected using deep learning, and drone localization from known bottle positions is made using photogrammetry.

Estimations fusion:
motion and observation estimations are combined using a particle filter.

Video References:

Code Reference:

GitHub - AtsushiSakai/PythonRobotics: Python sample codes for robotics algorithms.

Python sample codes for robotics algorithms. Contribute to AtsushiSakai/PythonRobotics development by creating an account on GitHub.

github.com

 

[AIDrone]

Hi,

I have been following your work with Tello & how you have been pushing the boundaries of drone with power of AI and ML. I can find so many uses cases of the Tello Localisation work you have done here.

I am a developer and have built a Drone AI Software for flying Tello from macOS and Windows. May be we can collaborate on some ideas. My software allows bunch of cool ML features. Tello Localisation implementation will be one level up.

Let me know your thoughts.

 

[pgminin]

Hi,

I have been following your work with Tello & how you have been pushing the boundaries of drone with power of AI and ML. I can find so many uses cases of the Tello Localisation work you have done here.

I am a developer and have built a Drone AI Software for flying Tello from macOS and Windows. May be we can collaborate on some ideas. My software allows bunch of cool ML features. Tello Localisation implementation will be one level up.

Let me know your thoughts.

Hi, thank you for your nice words. I have more ideas than time to develop all of them. Now I'm working on Simultaneous Localization and Mapping and the idea is to port those algorithms on my Android App.
At the same time, I'm evaluating the bundle adjustment problem with dense reconstruction. A lot of things and for now I don't think I've time for a collaboration. Maybe in the future!

 

[Saemon]

Hi Aidrone!
I am glad that I met this news on the forum) the application is interesting, I will test it. Mobile version apk to run own script will be great.Please do not stop developing.

 

[Saemon]

Hi, thank you for your nice words. I have more ideas than time to develop all of them. Now I'm working on Simultaneous Localization and Mapping and the idea is to port those algorithms on my Android App.
At the same time, I'm evaluating the bundle adjustment problem with dense reconstruction. A lot of things and for now I don't think I've time for a collaboration. Maybe in the future!

Did you try build a 3d reconstruct? My fast experience in 3d pdf file

 

[pgminin]

Did you try build a 3d reconstruct? My fast experience in 3d pdf file

Hi, what sw did you use? The scene.pdf seems to be blank

 

[Saemon]

Hi, what sw did you use? The scene.pdf seems to be blank

Can use any 3d pdf reader, not just pdf reader. 3d model is not good, i dont know how use a position of drone, collaborate this coordinate with foto. This should increase the quality.
I used one flight of the drone, half a circle, split into frames and used 3d reconstruct app.

 

[pgminin]

Can use any 3d pdf reader, not just pdf reader. 3d model is not good, i dont know how use a position of drone, collaborate this coordinate with foto. This should increase the quality.
I used one flight of the drone, half a circle, split into frames and used 3d reconstruct app.

Ok now I see it.
Overall it's not so bad, I did the same outdoor with a full orbit and the quality of my 3d was very low (worst than yours) so I did not publish the result yet.

Here is a quick screen image of the 3d model using COLMAP for 3d reconstruction from video frames of the orbit (converges only on medium quality):

 

[Saemon]

Ok now I see it.
Overall it's not so bad, I did the same outdoor with a full orbit and the quality of my 3d was very low (worst than yours) so I did not publish the result yet.

Here is a quick screen image of the 3d model using COLMAP for 3d reconstruction from video frames of the orbit (converges only on medium quality):

View attachment 6919

I think building models is a matter of time. just using 3df zephyr, with default settings gives this effect. and this is without neural networks and any ml. now it seems to me important to associate frames from the video with the distance to the object.

 

[pgminin]

I think building models is a matter of time. just using 3df zephyr, with default settings gives this effect. and this is without neural networks and any ml. now it seems to me important to associate frames from the video with the distance to the object.

Very interesting, I will investigate 3df zephyr performance on my orbit, I see the 50 images limitation for the free version but your result is very promising.
I think that my distance estimation and localization are pretty noisy and not so good to improve the full bundle adjustment implementation of a complete 3d sw. But I'll see

 

[Hacky]

Any of you who knows how to access the downward facing cam stream? This is mentioned as new feature in the new firmware update for the Tello EDU.

 

[AIDrone]

Monte Carlo Localization for Tello

Tello downward camera is used for motion estimation:
position is integrated from optical flow velocity estimation (dead-reckoning).

Tello frontal camera is used for observation estimation:
landmarks (bottles) are detected using deep learning, and drone localization from known bottle positions is made using photogrammetry.

Estimations fusion:
motion and observation estimations are combined using a particle filter.

Video References:

Code Reference:

GitHub - AtsushiSakai/PythonRobotics: Python sample codes for robotics algorithms.

Python sample codes for robotics algorithms. Contribute to AtsushiSakai/PythonRobotics development by creating an account on GitHub.

github.com

How were you able to access the downward facing camera? Can you please share your thoughts on it.

 

[Hacky]

@pgminin Thank you for that link to an interesting collection.

@AIDrone
As I understand pgminin's post, he is not actually accessing the video stream of the downward facing camera (I would also be interested in that...). He is using the movement information generated through the optical flow logic and accessible from the drone status.