nouns - "A friend of my father" vs. "a friend of my father's" — which one is right? Why?

I have seen them both in my reference book, which is written by a native speaker. I wonder why there are two different phrases to describe the same meaning and am looking forward to some reasonable details to explain why this is so.

• Next grammaticality - Is it correct to use "their" instead of "his or her"?
• Previous orthography - Is there a term to describe words whose pronunciation bears little relationship to their spelling?