Not trying to explain anything away, but I don't see any plotholes here.

- Infinite directions doesn't mean infinite useful ways to go. Hyperspace navigation isn't exactly easy.

- Just because Ackbar orders them to retreat doesn't mean that they are able to retreat.

- IIRC, Lando could detect that the shield is working as he approached the death star. During the battle the rebels get closer to it, so they are all able to notice when the shield is gone.

- We don't see the whole battle from start to end, so it's not that surprising that the ships changed their positions.