Is an article acceptable in those sentences?

Can I say : We are not in a kindergarden. Should I say : We are not in kindergarden. Does usage of 'a' refer to a bulding?

Answer

Exactly as you noted, in this case, the use of article refers to a building.

Saying "we're not in a kindergarten" you're expressing the fact that you're not in the building itself, while saying "we're not in kindergarten" generally says you don't attend kindergarten anymore.

Another example would be:

I need to go to hospital. I'm extremely sick.

versus

Go to a hospital and steal some syringes.