How to interpret the meaning of "I am surprised that..."?

When somebody says "I am surprised that...", does it mean to express not only surprise but also some level of disappointment or disapproval?

If somebody uses that expression when speaking with me, I feel somewhat annoyed and guilty. It feels like there is some subtle expression of disapproval and even annoyance on the part of the speaker that they can't openly express.

Would a native speaker of English interpret this expression the way I do? Or would it depend not only on the expression but also on the manner of expressing it?

Answer

In principle it might be expressing either approval or disapproval, depending on context; but contra Jasper Loy I think it is more likely to express disapproval, because if it expresses approval I think it would be likely to be accompanied by words making that approval clearer.

So

"Wow! I'm surprised you said that"

is probably impressed, but

"I'm surprised you said that."

is probably disapproving, though either of these inferences may be changed by a suitable tone of voice.