I think it's because everyone in the south assumes that you need to run the AC all the time when it gets vaguely warm outside. So I freeze. I wear sweaters and long-sleeved shirts outside in 102 degree weather because it's more comfortable to melt for a few minutes between car/work/mall etc than to freeze all day.
The most irritating part? Everyone looks at me and says, "But you're from Canada! You're supposed to like the cold."