Storage designs for large-scale video surveillance systems

Storage systems for video surveillance are getting larger as customers demand more cameras, higher-resolution cameras and longer video-retention times. As a result, security integrators are increasingly faced with the challenge of designing and supporting large-scale video storage that meets the specialized performance, availability and management goals, which are unique to video surveillance. This article walks through the specialized requirements presented by digital video surveillance. It shows how a new form of clustered storage offers enhancements to traditional storage designs that are especially helpful when integrators need to make the jump from small-scale recording systems to large-scale surveillance systems, supporting thousands of cameras and petabytes of storage.

Comparing video data with normal business data

For the purposes of this article, we will talk about two types of data: normal business data and video data. Normal business data consists of email, files, database transactions, and perhaps small graphic images. Video data, by contrast, consists of dense video images that are constantly streaming into the storage system. In both cases, large storage systems deploy the concept of RAID to protect against any disk drive failures in the field. But the way in which these RAID controllers are architected can have a dramatic effect on the performance, cost, and manageability of the systems. Let's look into the characteristics of these two data types since they are markedly different and have large ramifications for storage systems.

Data Set Size: The first characteristic to look at is simply the raw amount of data that needs to be stored. A large database today is roughly 100 gigabytes and only very large companies generate general business data that reaches 1 terabyte (1,000 gigabytes) of information. By contrast, a video surveillance system with just a few hundred cameras can easily generate more than 1 terabyte of stored data in a single day.

Read vs. Write activity: A second characteristic to consider is read versus write activity. Normal business data, whether it is payroll database entries, legal file updates, or email messages generally follows the 80/20 rule, where 80 percent of the time data is being read and only 20 percent of the time is dedicated to write activity. Video data has nearly the exact opposite characteristics with data being written 100 percent of the time and read activity happening only rarely as incidents occur that need to be retrieved and reviewed.

Random vs. Sequential: Next, a storage system expert would look at how data is sent to the storage system and how the system handles that traffic. In the normal business case, data is written randomly in short bursts. This is termed "small block transfers" in the storage lexicon. Video data, once again, is radically different as video data arrives in a sequential and constant stream of dense information. As camera resolutions increase, and as the trend toward megapixel surveillance increases, the density of the data stream increases geometrically as more pixels are used to convey each image.

Planned Downtime: Normal business data generally follows a business cycle with a natural rhythm of downtime when maintenance, upgrades, and changes can be made to the system without affecting business activity. Video data has no such downtime cycle since the incoming stream of video is unrelenting and any changes or maintenance must be made dynamically without affecting the incoming stream.

Traditional SAN storage systems are designed for business data, not video

A traditional large-scale storage system is designed to accommodate business data. The architecture of these systems is well-established and is centered on the concept of a master RAID controller that has a fixed amount of network bandwidth, which controls a fixed maximum number of drives. In a traditional system, this master RAID controller acts as a funnel for incoming video data and is a limiter for the maximum capacity of the system. More advanced systems introduce a secondary controller for redundancy, but the maximum bandwidth and capacity remain fixed.

Video data presents quite a challenge to this traditional storage design. The funnel design, which is rarely overwhelmed for business data, is poorly suited for the 100 percent write and sequential nature of video data and introduces risks for some of the data to be lost. Traditional designs tend to need either next generation networks such as 10 Gigabit Ethernet or proprietary, expensive storage networks such as Fibre Channel in order to meet the bandwidth requirements of large-scale video installations. Similarly, the master controller architecture is problematic for field maintenance since capacity, bandwidth, and performance changes are difficult, if not impossible, to dynamically upgrade in the field.

Clustered storage is optimized for video

Newer clustered storage architectures aggregate a set of distributed RAID controllers to provide large-capacity storage. While providing the same data protection as traditional designs, the clustered systems introduce a number of innovations that are particularly useful for video. Data is spread automatically across the cluster, performance is allocated dynamically without user intervention and the cluster manages failure conditions for drives, network connections, and complete controllers. In storage terms, these features are termed dynamic provisioning, automatic load balancing, and multi-pathing support. In the best cluster implementations, the bandwidth of all the clustered RAID controllers is aggregated together and available to all the incoming video data, thereby removing the traditional funnel limitation using cost-effective Gigabit Ethenet technology available today.

For video, this is the optimal architecture for providing the high-bandwidth that is essential for the always-writing, sequential nature of video data traffic. And because clustering provides high-bandwidth by aggregating many network links, the architecture can deploy off-the-shelf, Gigabit Ethernet switches and cables for low installation and support costs. Dynamic field upgrades are easily maintained by adding additional clustered components. A side benefit of the architecture is that automated system rebuilds, such as in the case of a hard disk replacement, are accelerated over traditional designs, because all system components contribute to a massively parallel recovery.

Total Cost of Ownership considerations for large-scale video deployments

For someone who is not a storage expert, there are management and support considerations between traditional and clustered storage systems that should also be thought through before deciding on a video storage system.

First and foremost, traditional large-scale storage systems were designed for specially-trained, dedicated storage administrators in Fortune 100 companies. Configuring these systems requires extensive upfront planning and training, often at the user's expense. Because of the fixed nature of the underlying RAID controller design, users generally purchase the maximum configuration upfront since early configuration decisions cannot be easily changed in the field. Like the most sophisticated scientific calculators, these traditional systems have a lot of specialized commands to learn. In most cases, field support is best left to the manufacturer and annual maintenance fees can be quite high.

In contrast, a storage cluster manages sophisticated tasks without user intervention and requires only a high-level understanding of capacity requirements. The expectation is that existing staff with basic PC server skills will manage the system on a daily basis and that any required field upgrades or replacements could be handled by on-site staff. The same customer or integrator that handles gigabytes of data today can effectively manage hundreds of terabytes with these new systems.

What to look for when evaluating clustered storage for video surveillance

Once you have made the decision to look at clustered storage systems, it is useful to understand differences in the available choices. Here is a quick guide on key features to look for when evaluating clustered storage systems for large-scale video surveillance installations.

* Complete Node Protection for High Availability - A clustered storage system should eliminate all single points of failure including the failure of an entire distributed controller node.

* Distributed RAID for Capacity Efficiency - A clustered storage system that uses distributed RAID will protect distributed controller nodes and provide up to double the usable capacity of a clustered system that copies (or mirrors) data between nodes.

* Maximum Bandwidth for Performance - Clustered systems that eliminate master controllers and aggregate the network ports provide the highest bandwidth. For video installations, this means that they support more cameras and more video data without requiring expensive proprietary Fibre Channel or 10 Gigabit Ethernet.

* Fast Training - A clustered system should automate the management of the system to simplify configuration and management. An integrator or user with basic PC skills should need only 30 minutes to learn and be comfortable with the system.

* Investment Protection for New Drives - All clustered solutions provide some flexibility to "pay as you grow". Look for systems that can introduce newer, bigger disk drives into existing systems to take advantage of larger capacity drives as they become available.

* Cost Competitiveness over Time - Clustered solutions that use off-the-shelf hardware components provide the best assurance of cost-competitiveness over time. Be wary of proprietary hardware components that may be expensive or difficult to upgrade or replace.


Large-scale storage solutions are not equally well-suited to store and retrieve video data. Newer clustered storage systems provide enhancements over traditional storage designs that are especially helpful for the characteristics of the video data itself, and also meet the total cost of ownership requirements of the exploding video market. Overlooking these considerations can result in a serious mismatch of storage performance, cost, and long-term manageability.

About the author: Lee Caswell is co-founder and chief strategy officer of Pivot3, a firm which has developed clustered storage arrays designed for video surveillance applications.