Exactly what my Idea is: turning the direction by headtracking ????
Forward/backward up/down and sideways should be made by gamepads.
In the Way that you explained I think it will be weird. To
"Look out of the Window" will be strange and dangerous, if you dont look in the direction you fly.
I didnt fly by headtracking before, but it results in the same Way like my 360° degree Videos. You always will Watch it in the direction the cameraman Walks. Looking backward in a 360degree Video make strange Feelings. Sometimes you Watch a bit to left and right, but mostly you follow the direction forward.
May be you can implement both versions: Headless Mode with headtracking(look out of Window) and the "Captain Pilot Mode". I prefer the "Captain Pilot Mode" ?
And this should be done in VR Mode with Glasses ?