I have a 2003 Toyota 4Runner and just recently the blower stopped working in very cold weather.
I drive about one mile to work and the car sits for about 3 hours before I go to lunch, but when I do, the fan starts with no problem. It just won't start in the morning after a very cold night.
First, take a look at your owner's manual under air conditioning operation. The system may intentionally disable blower motor operation during very cold ambient temperatures -- who wants cold air blowing on their feet in a subfreezing climate?
If it's super cold out, the one mile stint on the way to work may not have been enough to raise coolant temperature and trigger the temperature sensor which activates the blower motor.
If the blower motor has been coming on sooner in the past, the coolant system's thermostat may be stuck open and not allowing coolant temperature to rise quickly. Perhaps ambient temperature was high enough by lunch time to eliminate the condition altogether.
Keep an eye on your temperature gauge (if equipped) next time out to be sure the needle is rising. If the blower motor's low-temperature shutoff feature is not the issue, it could be a number of areas that need to be traced and tested with a volt/ohm meter on a cold winter morning.
Possibilities are the blower motor freezing up, high resistance in the blower relay or resistor block, a poor ground circuit and so on.