meaning - Is it very wrong to say "can you pass the salt?" expecting people to pass the salt?

It makes sense to just pass the salt when someone says, "can you pass the salt?", but how grammatically incorrect is that?

Are we supposed to just answer yes for that question?

This happens in various situations. For instance, even teachers say "can you find the odd one out?
1 1 1 2 1 1 "
So, should one just say "yes" or should they say "2" ?

I apologise if the question is lame.