It might be normal. The location function, of being able to hold position is purely visual.
If your floor, or any surface under your
Tello is too plain, it can drift. Try flying it over an area that has some visible contrast.
I once took mine on a walk through a park, and it performed very well, except on a bridge. The
Tello slid sideways, because it could lock on the edge of the planks on the bridge, but could not "see" that the wind was pushing it along the edge of the plank.