Unless you almost completely unscrew it, It is designed to only move like 3 clicks before you start fighting it.
If you need to make a big swing in alignment, it is best to really unscrew the front screw, and maybe even before it is mounted, unscrew the ones on the other side of the bracket to get it in close proximity to the direction you want and then tighten the back ones and install and then move around for final placement.