grammar - Why do we say "ahead of" but not "behind of"?

It just occurred to me that even though "ahead/behind" mean opposite things, their usage is slightly different. Say we were talking about time zones. Why is it that I could say either

I'm two hours ahead of you.

or

You're two hours behind me.

But both of these are wrong:

I'm two hours ahead you.

You're two hours behind of me.

Since these words are perfect opposites, it seems like they should follow the same pattern. So that either "ahead of you" and "behind of you" would both be correct, or "ahead you" and "behind you" would both be correct.

There are even other cases where these two words do match each other in the prepositions they use. For example, if I'm no longer talking about time zones, and I'm instead talking about my progress at work, I could say either

I'm ahead on my work.

or

I'm behind on my work.