Uninterruptible Power Supply Integration for Video

Nov. 5, 2014
How to use UPS systems to control orderly shutdown and startup sequences

It is common for access control and alarm system deployments to use battery power to keep their equipment functioning for a specified period of time in the event of a power outage; however, due to the very high power requirements of the hard drives in video recording servers, it is usually not cost-feasible to provide four-hour or longer battery capability to keep video servers online. It also would not make sense to do that if the cameras would be offline during the power outage.

Yet, it can be catastrophic to ignore the impact of a power outage on a video server, NVR or even DVR, because a hard shutdown can leave video data in a corrupted state, due to files being left open or data being partially written. Even if a video server powers back up from a hard shutdown without any immediately-apparent issues, a corrupted video database may not store data properly going forward, and a significant amount of previously recorded video can be lost as well. Such problems are typically detected only during an investigation, when the loss of recorded video is discovered.

This is why video recording equipment should be integrated with an Uninterruptible Power Supply (UPS), to enable orderly shutdown upon a power outage and orderly startup afterwards. A large video server with terabytes or even petabytes of RAID hard drive storage can be protected from data corruption with only 10 or 15 minutes of UPS power — enough to allow the video management software, the server operating system and the storage expansion servers to shut down in the correct sequence.

Orderly Shutdown and Startup

Getting the sequence right is important, including its timing. If a storage expansion server is powered down before the video server it supports, data corruption can result. If there is a network switch between the primary video server and its expansion storage, the switch must remain powered on until both server and storage have shut down.

The shutdown sequence for a video server with one networked expansion storage unit should be something like the following (see Figure 1):

1. Video Management Server software. The VMS software should be shut down first, which may involve shutting down one or more VMS components running as operating system services. The VMS has to stop writing to the hard drives before any storage expansion unites can be shut down.

2. Server Operating System. Once the VMS has been shut down, the operating system can be shut down.

3. Server physical machine. The operating system triggers the hardware shutdown as the last step of its shutdown sequence.

4. Storage Expansion Unit operating system. Once the video server operating system has been shut down, storage expansion units can be shut down — these are typically Network Attached Storage (NAS) or Video Storage Area Network (SAN) servers or appliances. The shutdown of any storage management system services or application software will be performed automatically by the storage unit’s operating system.

5. Storage Expansion Unit physical machine. Typically the unit’s operating system triggers the physical machine shutdown.

6. Network Switch. If there is a network switch between the VMS server and the storage units that switch may now be shut down. Typically, network switches are left running and will either shut down when the UPS runs out of battery power, or remain running throughout the outage if battery power is sufficient.

Once the UPS notifies the servers that battery power has been initiated, shutdown actions should be started. Shutdown sequencing is implemented for multiple computers and appliances by using time delays to keep some of the equipment from shutting down sooner than others. It may take a bit of testing to discover the worst-case timing requirements for each element that has to be shut down or started up as part of a sequence.

The startup sequence must work in the reverse order of the shutdown, as in Figure 1. For example, if the network switch is not initialized first, the server and its storage expansion units will not be able to communicate. If the VMS tries to record data and the storage expansion unit is not up and running yet, the VMS will encounter errors writing to the data files and/or database, which may require manual intervention to restore recording functionality.

Startup and Shutdown Differences

The UPS manufacturer provides software that must be installed on each operating system that participates in the orderly shutdown sequence. The software is used to configure the UPS operation, and it also interacts with the UPS and receives notice when UPS transitions to battery power, and initiates the shutdown actions. Simple scripts can be used to implement multiple-step shutdown sequences, as explained in the UPS’s installation and user guide materials.

The shutdown sequence for each machine is software-initiated. It begins in software and ends with physical machine shutdown. The startup sequence, on the other hand, is hardware-initiated. It begins with hardware power-up. When a machine powers up, its operating system will boot up automatically and launch systems services and software according to the operating system’s startup configuration. Hardware startup sequencing can be accomplished by controlling the power-up sequence of UPS power outlets. This prevents all of the machines from powering up simultaneously, which could result in operating systems and software applications initializing out of sequence.

An alternative to UPS power outlet sequencing is the use of a Power Distribution Unit (PDU) — which is basically a networked smart power-outlet strip. These units come with software that is used to configure the outlet power-up sequencing when a power outage ends.

For UPS and PDU outlet control, a power outlet “turn on delay” setting is used to accomplish sequencing of each machine’s power-up. Most UPS units have two controllable outlet groups, allowing switches and storage units to be placed in the first startup group and servers to be placed in the second group. Switch initialization time is usually shorter than that of the other equipment, so placing it in the first startup group generally ensures that it will complete its initialization before other equipment.

When servers and storage are running in one or more virtual machines the same basic principle applies: the applications, system services, operating systems and virtual machines must be shut down and started up in the appropriate sequences.

Video Project Power Plan

Video projects should always include a Power Plan that identifies which equipment will be on emergency power and/or backup UPS power. It specifies the shutdown and startup sequence requirements. The Power Plan establishes and documents the minimum requirements of any UPS and PDU products for the project.

Ray Bernard, PSP, CHS-III is the principal consultant for Ray Bernard Consulting Services (RBCS), a firm that provides security consulting services for public and private facilities. For more info, visit www.go-rbcs.com or call 949-831-6788. Ray is a member of the ASIS Physical Security and IT Security Councils. He is also a member of the Content Expert Faculty of the Security Executive Council (www.SecurityExecutiveCouncil.com). Follow Ray on Twitter, @RayBernardRBCS.