I hounestly don't think they take advantage of your freezing. They pick up on your apprehension. I know it's hard to mask fear and trepidation. But try to become pro active rather than reactive. If a horse senses fear you must become passive, non reactive, just, for want of better analogy, limp and impassive. Try to become indifferent to their fear.
The most impressive display of this was on a course with Magnús Larusson, (the great Icelandic horse trainer) A horse got tangled in long reins and went into a panic. The watching crowd reacted by trying to catch the horse which went completely wild. Thankfully Magnús has a loud enough voice and big enough presence to command a crowd. Every one was to stand still and look at the ground. And I mean COMPLETELY STILL,within seconds of everyone freezing,the the horse came to a standstill and he was able to catch it and de-tangle it. But imagine the temptation to do something. As he says they are 10 times stronger than you, yet they take the slightest cue from you as how to behave.
It is possible to move a horse with a feather, you don't need strength, just empathy, you have shown you have it as you knew your horse was frightened of the tie down.