I guess with "swarm control" you mean some kind of formation performed on mission pads or the large "mission carpet"?
If you only perform the positional commands (goto, curve etc.), you do not need another cascaded PID control layer. You only need this, when you plan to perform smooth continous floating figures like full circles, 3D lissajous figures etc. In that case you use the coordinates reported by the Tello drones while the fly over a mission pad and continously correct the error between desired position and reported position. The PIDs you will need here look different to those PIDs, you may have found as a good fit when controlling the drone through the camera view and some OpenCV based stuff because you have different update frequencies and lags in those loops.
If you only perform the positional commands (goto, curve etc.), you do not need another cascaded PID control layer. You only need this, when you plan to perform smooth continous floating figures like full circles, 3D lissajous figures etc. In that case you use the coordinates reported by the Tello drones while the fly over a mission pad and continously correct the error between desired position and reported position. The PIDs you will need here look different to those PIDs, you may have found as a good fit when controlling the drone through the camera view and some OpenCV based stuff because you have different update frequencies and lags in those loops.