In the TelloFPV, the default mode to operate is the student mode. If you select the desired gamepad as the student mode, it will work.
As much as I understand, the Master (teacher) has always control over the Student, but the student has again control after 2 seconds of Master stopped to use his...