Same thing happens to me - just ride faster than the car behind you
Slightly more seriously I find they do demist themselves as long as you don't have to ride in the cold zone that caused it for too long. So rather than smear the mirrors with my glove I tend to do much more thorough life savers until they clear again.