Editor’s note: In response to the coronavirus crisis gripping the world, TechGenix is republishing a selection of recent articles, tutorials, and product reviews with relevant information for IT pros as their jobs change dramatically. In this article, originally published Dec. 16, 2019, we look at some remote management products that can help an IT admin avoid a trip to the datacenter, an especially important safeguard during the pandemic.
The modern datacenter is managed almost entirely remotely. There may be times when you have to come in and add new servers or storage pools to racks or update mission-critical security software or devices. But these times are rare unless your datacenter is rapidly gaining new clients or is undergoing other changes. I know this because there’s a large corporate datacenter building that I drive past at least a couple of times each week. In the daytime, I may see several cars or other vehicles parked behind the perimeter wire fence around the building. At nighttime, I never see more than one vehicle present, and often there’s none. Operating systems and applications are easy to manage remotely and easy it’s easy to reboot them when something goes wrong and the program crashes or freezes. But what do you do when the underlying hardware needs rebooting? System hardware platforms usually have an out-of-band (OOB) interface that allows vendor-specific management software to remotely reboot them. But some kinds of network devices don’t support such remote management. And what happens if the OOB interface itself becomes inaccessible? In that case, only a hard reboot — cycling the AC power for the device off and then on — can make it possible to get the device up and running again from a remote location.
Devices for remote power cycling
If you work in a datacenter or your company has a server room packed with devices you need to be able to power cycle remotely. Because the last thing you want to have to do as a sleepless admin or a busy datacenter drone is to get out of bed late and night to drive to your datacenter just to flip a switch up and down to do a hard reboot. Fortunately, there are several off-the-shelf solutions you can purchase that can keep you off the road when the hours are late and accidents tend to happen. Let’s look briefly at a few of these solutions that have been recommended to me by my peers who work in the datacenter trade. Then after that, I’ll end with a few tips on how to use them properly and to best effect.
The Ethernet Power Controller 7 from Digital Loggers Direct is recommended by several datacenter experts I know. You can use this device to instantly reboot, start, or stop your servers, routers, and other network gear. You can also use the built-in AutoPing feature to reboot devices automatically whenever a fault has been detected happening in them. I also like the fact that the device can monitor both voltage and current and identify power surges. Plus it has other features that make it customizable to meet various needs. Digital Loggers also has other products that can be useful for your datacenter or server room.
The line of netBooter products from Synaccess Networks is also popular within the datacenter community that I have contact with. These devices come in several configurations and allow AC power outlets to be controlled independently via Telnet, the Web, or through an external modem using a USB or serial port. AutoPing support is included and lets you monitor network connectivity so you can reboot devices automatically when they go down. You can use both Python and Perl to script customized solutions using these devices.
If you’re looking for something simple then the iBoot-G2 Web Power Switch from Dataprobe may be all you need to get for your environment. It offers a simple web interface and a free cloud service that lets you control multiple iBoot devices in different locations from a single portal using a single sign-on.
For more complex environments APC offers the NetShelter line of Switched Rack Power Distribution Units (PDUs) that provide advanced load monitoring together with remote on/off switching capability over individual AC outlets for power cycling. They also can provide outlet use management and delayed power sequencing, and they’re user-customizable in various ways as well. APC products are widely used in datacenters and server rooms. I know one customer that runs a small datacenter where they use multiple different APC devices arranged hierarchically to manage and monitor all their network devices and systems. Their setup includes feeding an AP4431 automatic transfer switch (ATS) into several AP7941 switched power distribution units (PDUs) each connected to a dozen or more devices via AP7723 devices for providing redundant power to single-corded equipment. All of this in a single rack enclosure.
One final product that’s been recommended by some is the mPower line of mFi Controllable Power Outlets from Ubiquiti. These devices provide controllable power outlets with WiFi capability, and they can also let you create customized power on/off rules that operate on a schedule. Like most modern remote power cycling products, these also include power metering capabilities so you can monitor your energy consumption to reduce costs.
Tips on using remote power cycling devices
While most of these products are easy to set up and a snap to use — which is exactly what a remote power cycling device should be since you generally only need to use it in an emergency when your blood pressure is high and you can’t think straight — there are some caveats you should keep in mind in order safeguard your network and ensure the devices can be used as they are intended. One thing I’ve learned for example — and learned the hard way — is that most of these devices use an internal battery to power them when AC power itself has become inaccessible. The problem with batteries, of course, is that they can run down — even if they’re not being used regularly. So be sure to replace the battery on these devices periodically. The instruction manual that comes with the device should tell you how often you need to do this.
If you are going to control these devices over a TCP/IP connection then beware of letting there be too much Layer 2/3 broadcast traffic on your network. Or SNMP traffic since many of these devices can also be operated by SNMP and you don’t want to overwhelm the device’s software. It’s best in fact if you keep all your power cycling devices on a single management network — a VLAN that you’ve dedicated for use by embedded-stack devices.
Another thing you should be very concerned about is the security of these devices. First, check the manufacturer’s website and documentation to find out if there are any hard-coded admin passwords or back doors, and change the built-in admin password before deploying the device in your environment. Update their firmware regularly as well whenever the vendor releases a new firmware update. And perhaps most importantly, never put infrastructure interfaces such as those used for remote power cycling on the public-facing Internet. Always access them instead through a high-grade VPN connection that uses two-factor authentication to control access. Because the last thing you want to have happen is some script-kiddie bringing your whole datacenter down from nosing around your remote AC power interface. That did happen to one datacenter admin I know, unfortunately.
Also, be sure to consider taking advantage of the delayed power-on capability if the device supports this. The reason for using this is because when a power outage happens in your datacenter and you need to reboot a bunch of systems, you don’t want to reboot too many of them all at once, otherwise, you may blow your main circuit breakers. If that happens of course then it’s time to get the car out of the garage and drive to your facility no matter how late at night it is.
Leave some room to grow
Finally, my advice if you’re looking to purchase one of these remote power cycling solutions is that you buy one that’s either expandable or gives you more headroom (can cycle a larger number of AC outlets) than you think you’ll need. My reason for suggesting this is that you’re probably going to find these devices so helpful that you’ll wish you had purchased a larger unit than the one you ended up buying. Because they give you peace of mind even if you don’t use them. And peace of mind is something that’s definitely in short supply for most of us who work in system and network administration.
Featured image: Shutterstock
