Terabytes, petabytes, exabytes, zettabytes, brontobytes...what's our security world coming to?
When you combine the undeniable trends of more surveillance cameras, higher bandwidth requirements, fatter bandwidth pipes and the continuing decline in storage costs, you get near-exponential growth in bytes stored. The question now becomes how to manage this vast amount of video data. I've discussed these questions with several large-scale storage vendors recently and learned some very interesting things.
First, there are well-developed content management systems available; however, managing "regular" data is different from managing video data — particularly streaming surveillance video. Whereas enterprise data consists mainly of small random blocks that are primarily reads rather than writes, video data are large blocks of data that are written far more than they are read. Further, you must organize the files and handle Metadata (tags) that go beyond date and time — a task that VMS and video storage vendors deal with on a regular basis.
This helps explain why those that can tag — such as analytics providers, and high volume storage providers — regularly seek to partner with VMS suppliers, who are the gateway between users and the external storage connected to the network (NAS, DAS, SAN). Innovation and implementation success occurs around these relationships. The manner in which the information is marked, cataloged, and shipped off to storage will directly affect the ease and speed of pulling it back out for review.
Companies providing storage solutions are ready. Pivot3, a familiar face in the industry, offers a purpose-built platform that integrates server virtualization technology from VMware to minimize or eliminate the need to provision, power and maintain dedicated storage servers. Their environment consists of 3-12 appliances, each with its own operating system and capable of running a virtual server infrastructure, hosting a VMS. The operating system is designed to handle large blocks of compressed video and the inevitable bandwidth spikes that occur when, for example, a scene becomes rich with motion or, perhaps, where the lighting is inadequate for the camera. It can scale to 432 TB, but it also capable of integrating with existing NVRs and DVRs.
NetApp, the second-largest storage provider and vendor to major data center operations, has packaged raw storage horsepower into a very small footprint: 180 TB in a 4 RU space ( 60 drives), scalable to 1080TB. The system’s read/write speeds are impressive — 4.4 GB/sec for Read, 2.4 GB/sec for Write per 4 RU unit. The video applications and control reside elsewhere. This is sheer storage and lots of it, efficiently packaged to limit space requirements in control rooms or data centers.
Most video is stored for a limited period (less than 30 days is common). This may be because the user’s situation or needs do not demand anything longer, but the decision could be driven by storage costs. In a prior product generation, the digital tape "jukebox" was an answer to long term archival — now, tape may be back.
New entrants to the industry, such as Crossroads Systems, working with tape library provider Spectra Logic, promote a technology based on LTFS (Linear Tape File System) to create a storage center that sits ready on the network, looks like a network device, holds a ton of video, and costs about 80 percent less than hard drive storage, with better reliability, higher density, and less energy consumed. Tape capacity is similar to SATA drives,1.5 or 2.5 TB.
The catch is lower read-write speeds (transfer rates of 160-600 MB/sec), which Crossroads has achieved by front-ending the tape system with a StrongBox (server with HDD storage) that acts as a cache to manage tape storage and speed information retrieval. While read-write capability is more limited than pure HDD systems, it looks like a good solution for long-term archival.
At the end of the day, provisioning storage should involve a long look at Total Cost of Ownership (TCO), as well as system features including performance and redundancy. Consider not only acquisition costs, but electrical usage, heat generation, space requirements and expandability.
A petabyte (PB) is 1000 trillion bytes (1000 TB) of data, and a storage system for this much data costs $750,000 to $2 million vs. $200,000 for a tape-based system. In case you were wondering, a brontobyte is a trillion PB (1 followed by 27 zeroes).
Ray Coulombe is Founder and Managing Director of SecuritySpecifiers.com, enabling interaction with specifiers in the physical security and ITS markets; and Principal Consultant for Gilwell Technology Services. Ray can be reached at [email protected], through LinkedIn or followed on Twitter @RayCoulombe.
RESOURCES
Pivot3: www.securityinfowatch.com/10214659
NetApp: www.securityinfowatch.com/10797240
Crossroads Systems: www.securityinfowatch.com/10797241
SpectraLogic: www.securityinfowatch.com/10797249
