Storage designs for large-scale video surveillance systems

Comparing traditional SAN designs vs. today's clustered storage solutions


Video data presents quite a challenge to this traditional storage design. The funnel design, which is rarely overwhelmed for business data, is poorly suited for the 100 percent write and sequential nature of video data and introduces risks for some of the data to be lost. Traditional designs tend to need either next generation networks such as 10 Gigabit Ethernet or proprietary, expensive storage networks such as Fibre Channel in order to meet the bandwidth requirements of large-scale video installations. Similarly, the master controller architecture is problematic for field maintenance since capacity, bandwidth, and performance changes are difficult, if not impossible, to dynamically upgrade in the field.

Clustered storage is optimized for video

Newer clustered storage architectures aggregate a set of distributed RAID controllers to provide large-capacity storage. While providing the same data protection as traditional designs, the clustered systems introduce a number of innovations that are particularly useful for video. Data is spread automatically across the cluster, performance is allocated dynamically without user intervention and the cluster manages failure conditions for drives, network connections, and complete controllers. In storage terms, these features are termed dynamic provisioning, automatic load balancing, and multi-pathing support. In the best cluster implementations, the bandwidth of all the clustered RAID controllers is aggregated together and available to all the incoming video data, thereby removing the traditional funnel limitation using cost-effective Gigabit Ethenet technology available today.

For video, this is the optimal architecture for providing the high-bandwidth that is essential for the always-writing, sequential nature of video data traffic. And because clustering provides high-bandwidth by aggregating many network links, the architecture can deploy off-the-shelf, Gigabit Ethernet switches and cables for low installation and support costs. Dynamic field upgrades are easily maintained by adding additional clustered components. A side benefit of the architecture is that automated system rebuilds, such as in the case of a hard disk replacement, are accelerated over traditional designs, because all system components contribute to a massively parallel recovery.

Total Cost of Ownership considerations for large-scale video deployments

For someone who is not a storage expert, there are management and support considerations between traditional and clustered storage systems that should also be thought through before deciding on a video storage system.

First and foremost, traditional large-scale storage systems were designed for specially-trained, dedicated storage administrators in Fortune 100 companies. Configuring these systems requires extensive upfront planning and training, often at the user's expense. Because of the fixed nature of the underlying RAID controller design, users generally purchase the maximum configuration upfront since early configuration decisions cannot be easily changed in the field. Like the most sophisticated scientific calculators, these traditional systems have a lot of specialized commands to learn. In most cases, field support is best left to the manufacturer and annual maintenance fees can be quite high.

In contrast, a storage cluster manages sophisticated tasks without user intervention and requires only a high-level understanding of capacity requirements. The expectation is that existing staff with basic PC server skills will manage the system on a daily basis and that any required field upgrades or replacements could be handled by on-site staff. The same customer or integrator that handles gigabytes of data today can effectively manage hundreds of terabytes with these new systems.

What to look for when evaluating clustered storage for video surveillance

Once you have made the decision to look at clustered storage systems, it is useful to understand differences in the available choices. Here is a quick guide on key features to look for when evaluating clustered storage systems for large-scale video surveillance installations.

* Complete Node Protection for High Availability - A clustered storage system should eliminate all single points of failure including the failure of an entire distributed controller node.

* Distributed RAID for Capacity Efficiency - A clustered storage system that uses distributed RAID will protect distributed controller nodes and provide up to double the usable capacity of a clustered system that copies (or mirrors) data between nodes.