It’s bitterly cold here in Winnipeg, Canada, as I am writing this article. Being in the center of the North America continent, our winters are usually uncomfortably dry and cold. Fortunately, the HVAC system for our offices is working fine, so our staff is warm as toast and our servers, workstations, and connectivity to the cloud are all up and running properly. Datacenters should be immune from potentially adverse effects of cold weather, right? After all the companies that build them (like Amazon) usually have money to burn, so they must be able to afford the best HVAC systems and other equipment to ensure everything is kept properly cooled. In fact, one might even think datacenters could benefit from cold weather outside by bringing in cold air from outside to keep those racks of servers from overheating and save on air conditioning costs.
Can you save on AC costs during winter?
Unfortunately, it’s not always so simple keeping your network and servers going when cold weather arrives. The idea that you can just shunt in cold outside air and dial down your AC system to save costs is usually an illusion. While I’ve heard of a couple of smaller datacenters that cool their systems on ambient air during cool weather, they usually have the opposite problem when the weather turns really cold, namely, how to keep their equipment from freezing solid. What some have tried is to cycle the hot exhaust from their server racks back into the datacenter to try and maintain an appropriate and constant temperature. You can imagine however how technically challenging this can be to achieve properly.
In fact, many datacenters run both supplemental heating and cooling during extremely cold weather. The additional heat is needed to keep the outdoor components of the HVAC system from freezing up and to keep the chillers pulling heat into themselves to ensure they don’t freeze up before the coolant returns. Wind complicates this in wintertime as it pulls heat out of your HVAC cooling loops more quickly by acting like a giant fan on your evaporators.
Another problem is that outside air becomes extremely dry in very cold weather. This means that even if you do cycle in outside air for cooling purposes, you effectively won’t save any money because the humidification coils in your HVAC system will need to be running overtime to maintain proper relative humidity in your datacenter. This is critical because changes in relative humidity may cause condensation problems which can be deadly as far as electrical equipment is concerned. So the money you can gain by utilizing outside air can easily be offset by the money you need to spend to maintain proper relative humidity.
Monitor external air flows
Since equipment and components on the outside of your building are more exposed to cold weather, problems are more likely to happen there than inside your datacenter. Outside air vents are one type of component that need to be continuously monitored during very cold weather. You want to ensure that no form of blockage occurs, either through ice or with snow. I know of one datacenter that was brought to its knees when the air exit vents froze shut on the building.
Watch out for electrical fires
When heating or cooling systems are running at maximum load for extended periods, for example during bitter winter spells or extremely hot summers, components of these systems are more likely to fail for various reasons. This is especially true for components mounted externally on your building where temperature stresses have more impact. Failure of an HVAC component can then result in electrical problems that can lead to a fire. The fire when detected can then result in automatic failover to UPS generators which can lead to system shutdowns affecting customers. I heard of one datacenter recently where a transformer fire happened because the weather had gotten so cold that the cooling oil inside the transformer was no longer circulating properly, causing the transformer to overheat resulting in a power surge that damaged other equipment.
A related issue is what happens when a fire happens and you need to put it out. The reality is that it’s very hard to deal with fires in extreme weather conditions. For example, heavy snow may make it slow the response time for the fire department to arrive. And when the firefighters arrive they may find that the fire hydrants beside your building are frozen solid and need to be unthawed before your fire can be dealt with. In fact, fire departments often send extra firefighters in extreme winter weather so they can rotate them in and out of the warming truck as the fire is being fought.
Feed your HVAC bunnies
Since a healthy HVAC system is so critical to the healthy operation of your datacenter when extreme cold (or extreme hot) weather arrives, it’s important to make sure you have a good HVAC team in place — or at least always available in case of an emergency. Most HVAC systems even for datacenters still require some manual intervention during extreme weather to ensure they perform properly and to identify potential points of component or system failure. So make sure your HVAC team is happy and well-paid by building management. If you like to knit then you might want to knit them some nice warm woolen caps too, because it’s no joke how important warm clothing becomes when people have to work outside in cold weather. Most datacenter business owners don’t think about this; they’d rather knit data caps for their customers instead.
Events beyond your control
Sometimes things can happen that are simply beyond your control, and they can derail even the most carefully implemented plan for ensuring cold-weather operation of your infrastructure. For example, I heard about a situation with a datacenter in Minnesota where the weather got so cold that the local utility company asked them to turn on one of their generators to reduce the load on the area grid. Unfortunately, when they tried to comply with this request from the utility company, the automatic transfer switch (ATS) involved failed and they lost the B feed of their dual-feed power system for their server racks, forcing them to power-off some of their systems for a few hours to maintain operating functionality. So even the best of plans laid by mice and sysadmins can go wrong at times. And of course, dual power feeds only work if they’re implemented properly.
Make sure you have backup power
My final tip should be obvious, and it’s something you can in fact control: make sure you have a reliable source of backup electrical power for your datacenters in case something happens in wintertime that cuts the power off for your building. The key here is the word “reliable,” however. What I mean is, when was the last time you actually checked that your backup generators still work?
Featured image: Shutterstock
1 thought on “Just chill out: Datacenters and cold winters”
Recently our data center had new CRAC units installed (which previously had Liebert air handlers, the building was constructed in the late 90s). The temperature change has been quite noticeable. Do you have any suggestions for those that work in the data center daily from freezing to death? After about 3 hours my fingers and nose are frozen.