grammaticality - What exactly does it mean to say something is "grammatical?"

I often see the expression "That's ungrammatical" used to explain why something is not OK. For example, a user might post a question: "Is it OK to say, I are go to New York?" Most people would answer it is not OK because it is "ungrammatical" and then depending on their level of knowledge give reasons or site references to back up their claim.

However, if the sentence is "I am owning a car." or "I will speak with your pony yesterday." it is suddenly not so clear. In the first case, that's just not how native speakers learned to speak (although many ESL learners might say "Why not? It makes perfect sense and doesn't violate what I learned in school!), and in the second case, the sentence doesn't make sense. But, I would contend that they are both grammatical. Sentence order is standard English, the verb and subject agree, the prepositions are in the correct place, etc.

I ask because I think it's important to have a clear idea of what "ungrammatical" means. But I also ask because from time to time we get questions such as "Why is it incorrect to say "I have written a book since 2001."?" and they don't get much attention and when they do the answers are "Sounds wrong." "To my ear..." or "It's just plain ungrammatical."

So what does it mean to say something is "grammatical" or "ungrammatical?" Does it mean that the sentence follows "standard" syntax rules (whatever those are) or does it mean that the sentence is logical and "makes sense?"