Right of way does not let you steer at people. Whenever a right-of-way boat changes course, she must leave the keep-clear boat room to respond. If the keep-clear boat is already doing what she should and your turn makes that impossible, you have broken this rule.
16.1
Applies to every course change by a right-of-way boat, including luffs, bear-aways, and turns while rounding a mark. The test is whether the keep-clear boat, sailing well, could respond to the change in time. Course changes before the start are covered too; there is no "hunting" exception.
16.2
After the starting signal, when a port-tack boat is keeping clear by sailing to pass astern of a starboard-tack boat, starboard may not change course if port would then have to change course immediately to keep clear. It applies on any leg, and to a luff as much as to bearing away. This stops starboard from "hunting" a boat that is already ducking her. Appendix C narrows it for match racing to bearing away on a beat.
Common situations
Rounding a mark with an inside boat
A right-of-way boat that changes course while rounding still owes room to keep clear. A boat entitled to mark-room that breaks rule 16 as a result of the other boat failing to give that room is exonerated (rule 43.1(b)).
Common misreadings
"Any course change is a foul"
A right-of-way boat may change course as often as she likes. Rule 16 only bites when the change leaves the other boat unable to keep clear.