"It was the kind of story that / where you had to be there." -- Are the relative words 'where' and 'that' interchangeable?

Consider this exchange:

A: Your story wasn't funny at all.
B: Maybe it was the kind of story where you had to be there.

I encountered something like that a few days ago, and wondered if the relative word where could be replaced with that:

^?Maybe it was the kind of story that you had to be there.

It sounded strange a bit, but I was reluctant to say it's wrong, because there are analogous examples where a that relative or bare relative could be used instead of a where relative clause:

• The place I went running was a few blocks away



• This is not the place I will die.


• It's pathetic to live in the place you grew up.


• If you would have told me a year ago that I'd be in the place that I am now, I would have been like, good joke.


• Why does poetry become the place that you can say it?

^{examples from COCA}

So, when is it possible to use the relative words that and where interchangeably?