Breakthrough!
I now have TelloFpv running on one phone, and a special "viewer-only" version on another phone. I haven't measured latency yet but does not seem to have a tremendously negative effect.
Control works only through the main phone running TelloFpv. Both master & student controllers have to be connected to this device. The TelloFpv-Viewer app shows video, telemetry including home position indicator, and a couple of messages that are brokered by
Tello (like handland timers and most warnings).
A few things will not be shown on the viewer: the red/green indicators for Video / Sport mode and active controller, autopilot messages, maybe a few more things I haven't thought of yet.
The viewer and main phone both can be put into normal or VR mode independently. So it would be possible to use TelloFpv on the phone in your hands for control, and use another phone with the viewer in the VR goggles. Or use the main phone in VR mode in your goggles, and use a tablet running the viewer for some audience. Or fly with your main phone and let your kids / wife / SO passively enjoy the flight with VR goggles.
All kinds of combinations and use cases are possible.
TelloFpv viewer is completely passive, that is you can't fly
Tello with it, can't connect a controller, or trigger any functions or autopilots. The only "function" it will offer is screen recording.
I will need a few days to iron the bugs out, and customize the layout for the TelloFpv Viewer app. So there will be a release in the next few days for the main app, and a new separate Viewer App.
Any smart ideas for the name of the viewer app version? Just TelloFpv Viewer? TelloFpv Slave? TelloFpv Monitor?