I think you may be misinterpretting the "4 years" thing. It's an exemption for people who have to move for work for a short time. If that isn't your situation, then the exemption has nothing to do with you.
IE. I live in Calgary, and own a house. If my work transferred me to Dallas for a year, I could rent my house out for that year, and then move back into it when I came back to Calgary, and maintain my personal residence status.
I really don't think this is a loophole you'll be able to exploit for tax free capital gains treatment on your investment portfolio, and it's probably important that you don't try to do so.
If you're just talking about paying tax on your actual personal residence, you can stay as long as you want and not pay tax. It's just a straight exemption.
Regards,
Michael