Tense simplification in subordinate clauses - past perfect instead of would have

in Michael Swan's "Practical English Usage" he states in entry 580.6 ("past instead of would ..."): "would, like will, is avoided in subordinate clauses; instead, we generally use past verbs". He also gives the following example:

-Would you follow me wherever I went?

However, if I wanted to refer to an unreal past situation in a subordinate clause (and by that, I mean not only an if-clause), would it be correct to use a past perfect form. For example, if I modified Michael Swan's example so that it refers to an unreal past situation, would this sentence be correct?

-Would you have followed me wherever I had gone?

My point is, Michel Swan just talks about using past verbs instead of would in subordinate clauses, but he doesn't mention anything about using past perfect forms instead of "would have".

I really appreciate every answer. Thank you in advance