- Joined
- Jan 7, 2019
- Messages
- 103
- Reaction score
- 100
How do I receive, decode and process video stream in Android?
It's time to answer this very common question!
Here I give my solution that works with Tello in SDK mode.
All what you need is ffmpeg!
It can receive, decode and save the streaming video.
Moreover it can extract and save a frame as a bitmap at a fixed frame rate.
Exactly what you need to perform real-time image processing as object detection!
And it can do all this in just a command provided in the right framework!
THE FRAMEWORK
Clone in Android Studio this excellent project from github:
github.com
It has an easy UI to put the Tello in SDK mode and start the video streaming.
It has also these buttons: "USE FFMPEG FOR VIDEO" - "STOP FFMPEG".
They let you decode and save on Android the video stream.
But it does not show you a live video.
We must go to the code and modify it!
THE CODE
In MainActivity.java spot the method:
public void on_click_btnCaptureVideoStream(View v) {}
The ffmpeg cmd that receive, decode and save video streaming is this:
String[] cmd = {"-i", "udp://127.0.0.1:11111", "-vcodec", "copy", output};
FFmpeg ffmpeg = FFmpeg.getInstance(this);
// to execute "ffmpeg -version" command you just need to pass "-version"
fftask = ffmpeg.execute(cmd, new ExecuteBinaryResponseHandler()
SAVING A FRAME AS A BITMAP
After a bit of ffmpeg documentation study and StackOverflow search you can end with something like this:
String[] cmd = {"-y", "-i", "udp://127.0.0.1:11111","-vf", livePreviewFPS_S, "-update", "1", outputImg, "-vcodec", "copy", outputVid};
where:
"livePreviewFPS_S" is the frame rate (as a string) at which ffmpeg saves the BITMAP
"outputImg" is the bitmap filename
"outputVid" is the video mp4 filename (saved at the original frame rate of 30 fps)
LIVE VIDEO AND PROCESSING
Now that a bitmap is extracted as a file from the streaming, you just have to:
- make a bitmap copy (as a buffer to avoid update blocking of the next frame),
- process the bitmap
- display the bitmap in a simple ImageView!
I hope this will be helpful for somebody.
Coding for Tello is very fun!
If you want to see this solution at work take a look at my Android app:
tellopilots.com
It's time to answer this very common question!
Here I give my solution that works with Tello in SDK mode.
All what you need is ffmpeg!
It can receive, decode and save the streaming video.
Moreover it can extract and save a frame as a bitmap at a fixed frame rate.
Exactly what you need to perform real-time image processing as object detection!
And it can do all this in just a command provided in the right framework!
THE FRAMEWORK
Clone in Android Studio this excellent project from github:
GitHub - Djacob502/TelloThird
Contribute to Djacob502/TelloThird development by creating an account on GitHub.
github.com
It has an easy UI to put the Tello in SDK mode and start the video streaming.
It has also these buttons: "USE FFMPEG FOR VIDEO" - "STOP FFMPEG".
They let you decode and save on Android the video stream.
But it does not show you a live video.
We must go to the code and modify it!
THE CODE
In MainActivity.java spot the method:
public void on_click_btnCaptureVideoStream(View v) {}
The ffmpeg cmd that receive, decode and save video streaming is this:
String[] cmd = {"-i", "udp://127.0.0.1:11111", "-vcodec", "copy", output};
FFmpeg ffmpeg = FFmpeg.getInstance(this);
// to execute "ffmpeg -version" command you just need to pass "-version"
fftask = ffmpeg.execute(cmd, new ExecuteBinaryResponseHandler()
SAVING A FRAME AS A BITMAP
After a bit of ffmpeg documentation study and StackOverflow search you can end with something like this:
String[] cmd = {"-y", "-i", "udp://127.0.0.1:11111","-vf", livePreviewFPS_S, "-update", "1", outputImg, "-vcodec", "copy", outputVid};
where:
"livePreviewFPS_S" is the frame rate (as a string) at which ffmpeg saves the BITMAP
"outputImg" is the bitmap filename
"outputVid" is the video mp4 filename (saved at the original frame rate of 30 fps)
LIVE VIDEO AND PROCESSING
Now that a bitmap is extracted as a file from the streaming, you just have to:
- make a bitmap copy (as a buffer to avoid update blocking of the next frame),
- process the bitmap
- display the bitmap in a simple ImageView!
I hope this will be helpful for somebody.
Coding for Tello is very fun!
If you want to see this solution at work take a look at my Android app:
Tello Vision 1D Demo
Get it on Google Play: https://play.google.com/store/apps/details?id=com.vision.pgminin.tellovision1d
tellopilots.com
Last edited:
