Lee Caswell, co-founder and chief strategy officer for Pivot3, weighs in on clustered RAID storage arrays vs. traditional SAN designs in terms of applications for large-scale video surveillance systems.

Storage systems for video surveillance are getting larger as customers demand more cameras, higher-resolution cameras and longer video-retention times. As a result, security integrators are increasingly faced with the challenge of designing and supporting large-scale video storage that meets the specialized performance, availability and management goals, which are unique to video surveillance. This article walks through the specialized requirements presented by digital video surveillance. It shows how a new form of clustered storage offers enhancements to traditional storage designs that are especially helpful when integrators need to make the jump from small-scale recording systems to large-scale surveillance systems, supporting thousands of cameras and petabytes of storage.

Comparing video data with normal business data

For the purposes of this article, we will talk about two types of data: normal business data and video data. Normal business data consists of email, files, database transactions, and perhaps small graphic images. Video data, by contrast, consists of dense video images that are constantly streaming into the storage system. In both cases, large storage systems deploy the concept of RAID to protect against any disk drive failures in the field. But the way in which these RAID controllers are architected can have a dramatic effect on the performance, cost, and manageability of the systems. Let's look into the characteristics of these two data types since they are markedly different and have large ramifications for storage systems.

Data Set Size: The first characteristic to look at is simply the raw amount of data that needs to be stored. A large database today is roughly 100 gigabytes and only very large companies generate general business data that reaches 1 terabyte (1,000 gigabytes) of information. By contrast, a video surveillance system with just a few hundred cameras can easily generate more than 1 terabyte of stored data in a single day.

Read vs. Write activity: A second characteristic to consider is read versus write activity. Normal business data, whether it is payroll database entries, legal file updates, or email messages generally follows the 80/20 rule, where 80 percent of the time data is being read and only 20 percent of the time is dedicated to write activity. Video data has nearly the exact opposite characteristics with data being written 100 percent of the time and read activity happening only rarely as incidents occur that need to be retrieved and reviewed.

Random vs. Sequential: Next, a storage system expert would look at how data is sent to the storage system and how the system handles that traffic. In the normal business case, data is written randomly in short bursts. This is termed "small block transfers" in the storage lexicon. Video data, once again, is radically different as video data arrives in a sequential and constant stream of dense information. As camera resolutions increase, and as the trend toward megapixel surveillance increases, the density of the data stream increases geometrically as more pixels are used to convey each image.

Planned Downtime: Normal business data generally follows a business cycle with a natural rhythm of downtime when maintenance, upgrades, and changes can be made to the system without affecting business activity. Video data has no such downtime cycle since the incoming stream of video is unrelenting and any changes or maintenance must be made dynamically without affecting the incoming stream.

Traditional SAN storage systems are designed for business data, not video