As horrible as this is, sometimes part of the reason I hold back from talking is also because I am angry/hurt about something the therapist has said/done and I think she hasn't earned the right to get to hear my real thoughts. I know I am really only hurting myself by doing this, not making her suffer and that it is very childish.
I've done the same thing Buffy. And you're absolutely right - it just hurts us in the end. Sometimes, when I take a few steps back from what has been said, I finally understand what the psychologist has said to me and the reasons for it. And sometimes, I actually go back in the next time and address the comment or observation that made me angry or upset. And we talk. Often, when I give her this opportunity to clarify - I feel better about her reasons for saying what she did.
In the end, the psychologist is really there to help us - which includes giving us the observations. Irrespective of whether these hurt us or we agree with them. Tough but necessary for the process I think.