AI Autonomous Navigation for Tello with ROS 2

[cstath]

Hi all.

I recently found some time to tinker with a side project I’d been putting off for a while.

Interested in how far AI perception can be used with a Tello for real-world robot navigation, I built a simple navigation system for it.

In order to guide the Tello, the system uses a hierarchical setup where an LLM agent performs high-level reasoning and selectively invokes vision AI models for depth estimation and object detection.

Given only human-style navigational instructions via a language prompt, the drone autonomously navigates an unseen indoor environment, locates a target object, and completes the task, without maps or environment-specific training.

The code currently supports linux, but it can be easily tweaked for windows as well.

I’ve published a short demo video and the full code here:

YouTube demo:

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

GitHub repo: GitHub - cstath/tello-ros2-ai: An experimental repo for integrating SOTA AI features with the DJI Tello via ROS 2

Feel free to try it out, or even contribute to it! All the best to you all.

 

[GFields]

Interesting. However, it took a long time to execute.

 

[cstath]

Interesting. However, it took a long time to execute.

Thanks. Yes, indeed. Response times could definitelly be improved. Possibly by using a better GPU. I am running it on a lower end nvidia on a laptop. The heaviest part is the LLM though, that is running on the cloud. One could run an LLM locally, but that would require a very high-end GPU I guess.