Is this a fair assumption and does anyone else have an insight into this?
Yes, exactly. Also sometimes past perfective is used as imperative (when applied to the second person). That is the most rude way to tell someone to do something.
"Пошел вон!" = "Get lost!" means literally "(You) have gone out"