Lee Caswell is founder and chief marketing officer for Pivot3, a company which specializes in data storage solutions design around the use case of video surveillance.
In this guest column, Pivot3's Lee Caswell discusses the concept of storage virtualization as compared to other storage models for video data.
Photo credit: Image courtesy stock.xchng/desigu
A recent report by IMS Research found the video surveillance storage market is roughly $1.2 billion. This market is growing at more than 60 percent annually based on the facts that more cameras are being installed, higher resolution cameras are in demand and longer retention times are required. The recently thwarted New York City bombing demonstrates the predominant growth of surveillance - 58 private camera views were available following the incident. And this number is far less than the number of views that would potentially be available in London, arguably the city with the most surveillance cameras deployed globally.
As the amount of video data grows, the viability of traditional DVRs and NVR with local storage is breaking down. With these systems, a fixed amount of capacity is available for video streams that may vary in needs over time. Determining what capacity to buy is a challenging task since the amount of storage depends on an estimate of resolution, frame rate, compression and motion detection by camera type. Therefore, the nature of the DVR or NVR infrastructure generally leads resellers to over-provision capacity for each system by up to 50 percent since fixed capacity is not easily expanded and has relatively low reliability.
More advanced compression technologies such as H.264 help bring storage requirements down when there is minimum motion in the field of view. However, the variable nature of the bit rate exacerbates the capacity estimation problem since requirements can be off by as much as 4x if motion is higher than expected. The market will continue to see improvements in compression standards, but the growth in the number of cameras, resolution and retention times will continue to drive the need for scalable and highly reliable storage solutions.
Effective ways to increase storage beyond a DVR or NVR
A storage area network, or SAN, is ideally suited for a centralized site such as an airport, casino or prison where there is plenty of bandwidth on the local area network. In a SAN environment, video recorders or archivers maintain their own file system, such as Windows, and these appliances talk to the shared SAN storage as if it were a local disk. Many archivers can share the storage and the SAN platform introduces more reliability over NVR/DVR systems because there is no single point of failure.
A network attached storage system offers shared storage for many archivers with an important difference - there is a file system on the NAS platform as well as on the local archiver. The advantage of the file system on the NAS platform is that it is easier to support a hybrid storage case as some storage occurs locally on self-contained NVRs/DVRs and extended storage is sent to a specific file on the NAS. Both SAN and NAS offer highly available shared storage, but the performance limitations of NAS file systems preclude their use for high camera count, high bandwidth installations. As a result, NAS products are typically used in lightly loaded distributed environments or when data is being sent across a WAN connection.
If shared storage is not required, then it is possible to simply add storage to a self-contained DVR/NVR with an external connector, which is normally fiber channel or serial attached SCSI (SAS). This is called direct attached storage because the capacity is tied directly to one server and cannot be shared. External DAS is best used when capacity requirements are well known, will not expand over time and where the performance requirements are fixed. For growing environments that might, for example, replace analog cameras with megapixel cameras over time, DAS offers little investment protection since a system cannot be expanded without a "forklift upgrade".
The beauty of SAN or NAS solutions is that these options don't require exact calculations since storage can be shared and scaled as needed. It is not easy to predict the amount of data generated by an individual camera because bit rates are dependent on assumptions on how much motion will be detected. The amount of motion can dramatically affect the amount of data needs and what the configuration should be.
Standardized for surveillance
Conventional storage products were designed around an IT data center workload, which is a far cry from a typical surveillance installation. Capacity points in even large IT installations are typically small relative to surveillance. While a medium-size surveillance environment can easily exceed 100 Terabytes (or 100,000 Gigabytes), most large databases are less than 100 Gigabytes. Similarly, an Exchange system for a medium-size business rarely exceeds 100 Gigabytes.
Conventional storage products also use proprietary storage networking hardware to reach the high bandwidth needs of video surveillance. These proprietary storage networks, called Fibre Channel, require special switches, optical cables, server host bus adapters and weeks of training to deploy and manage. These systems are commonly deployed in the financial services market where specialized storage administrators are tasked with configuring, installing and maintaining systems.
An alternative approach is called a scale-out storage model, which repurposes server hardware to provide both storage and server resources in a common hardware platform. This approach meets both the capacity and bandwidth needs of the surveillance market by aggregating the resources of many appliances and presenting the combined resources as highly available capacity that can be shared among many servers. In addition, the scale-out appliances have more processing power than is required for just the storage aggregation. Therefore, it is possible to integrate server virtualization software on each appliance so that archiver software or NAS software can share the CPU and memory resources that are already in place to run the scale-out SAN.
This concept of running embedded virtual servers saves power, cooling and rackspace over the use of physical servers. It also adds high availability since the application can be automatically restarted on another appliance. The virtualization provides application failover without the need for standby servers with additional software licenses. This introduces high reliability with an appliance model which is easy to manage and cost-effective.
Due to the increase in demand for more cameras and enhanced video quality, surveillance users cannot simply add conventional servers and storage. Server applications that share storage hardware resources to reduce overall power and cooling are critical to reduce costs and secure high availability and reliability. NVR servers generally drive 50 percent of power and cooling requirements. Whether a small sheriff's office is looking to store 30 days of video on 20 TB of storage or a large metropolitan airport requires 1,200 cameras and 500 TB of storage, savings in power and cooling from a combined solution are immediate and long-term.
Virtualization is a key technology that can help bridge the gap between budgets and customer needs. Storage providers that incorporate virtualization technology can reduce acquisition costs by up to 25 percent and operational costs by up to 40 percent. With storage representing such a high cost component of installations, virtualization represents an important tool for resellers and customers looking to meet budgetary constraints.
About the author: Lee Caswell is founder and chief marketing officer at Pivot3. Prior to founding Pivot3, Lee was EVP Marketing and Business Development at VMware and held a series of senior management roles at Adaptec (ADPT) ending as VP and General Manager of Adaptec's $350M Storage Solutions Group. Prior to Adaptec, Lee was VP Sales and Marketing at Parallax Graphics, which introduced the first real-time video capture products for the video surveillance market. He spent the first five years of his career in management programs at GE. Lee has a BA from Carleton College and an MBA from the Tuck School at Dartmouth College.