Cold is easier to take than hot. With cold you can add layers until you're warm again. Might not be able to move much, but you can stay warm.
And when it gets hot outside you see a lot of guys mowing their yards without a shirt on. And some of them REALLY need to put on a shirt, no matter how hot it is. >.<