Surveillance Storage in the Age of AI

Nov. 9, 2018
The rise of cognitive systems has brought a massive need for more robust ways to store and analyze video

As connected digital devices replace standalone analog devices, they are generating vast amounts of data that, in turn, enable us to refine and improve systems and processes – both security and in general – in ways that could not be imagined in the past.

Eventually, Big Data and AI metadata will be as critical as the introduction of electricity – touching nearly every aspect of our lives – and the consequences will be significant. By 2025, an average connected person anywhere in the world will interact with connected devices – computers, mobile devices, smart TVs and more – nearly 4,800 times a day. That equates to essentially one interaction every 18 seconds. In the next few years, an additional 26 billion sensors will be installed worldwide to help manage, monitor and improve our daily lives.

In this increasingly connected world, where access to real-time information has more or less become a necessity, our thirst for data has never been greater than it is today, and that thirst will only continue to grow. This is especially true in the video surveillance space, where data is being analyzed more than ever.

Like the datasphere itself, data analysis is expected to grow substantially in the next few years. IDC estimates the amount of data generated worldwide that is subject to data analysis will grow by a factor of 50 to 5.2 zettabytes by 2025, with the amount of data that is “touched” by cognitive systems growing by a factor of 100 to 1.4 zettabytes in that same time.

The ultimate value of data generated by an increasingly wider variety of disparate security and non-security sensors, devices and systems is its ability to be analyzed and distilled into actionable intelligence that can be used to increase security and improve operations. This virtual flood of data and the growing requirement for analysis pave the way for a new set of technologies including machine learning, natural language processing and artificial intelligence, or AI.

Collectively these terms are known as cognitive systems, and they are capable of facilitating the shift of data analysis from a relatively uncommon and after-the-fact practice to a more proactive driver of strategic action and decision-making. All this data can be analyzed and used to introduce unique user experiences and mitigate potential risks, opening up a new world of business opportunities.

With data from AI-enabled security systems, important tasks can be completed, such as seamless, efficient traffic flow management among connected vehicles to prioritize traffic protocols for emergency vehicles, detecting fraud in real time, or employing facial recognition to improve security at sporting venues or transportation facilities.

With the data gathered from AI systems, manufacturers can recognize operational efficiencies in production lines and can make immediate adjustments. Medical staff at hospitals can recognize unusual trends in a patient’s health and take action to significantly reduce mortality. Retailers can better understand their customer’s behavior as well as peak hours for foot traffic.

From a security standpoint, AI opens the door for intelligent real-time video analysis that can transition today’s ultra-high-resolution video from a tool used primarily for reactionary, post-incident investigation to a more proactive tool that enables preemptive action.

Instead of reviewing countless hours of typically uneventful video, like a car moving in the parking lot, AI enables security departments to identify specific events and triggers in the footage, such as, a blue bike riding south or a grey-haired man walking towards the building with a dog. The ability to capture scene footage with this knowledge and insight enables more accurate alerts and forensics, dramatically reducing the time to act and analyze the video manually.

New Frontiers in Surveillance Storage

From a video storage solutions perspective, these trends have instigated a shift from primarily write-only applications to constant, ongoing deep learning and analysis that produce unstructured data. Until recently, these systems relied on the processing power of cloud data centers to manage this analysis and heavy workload, but that model was fraught with latency issues.

Today, AI is being built into more NVR systems to enable them to process, analyze and recognize patterns on-site in real time at the edge, rather than dealing with the latency associated with transferring data and video off-site for analysis.

What has spurred this AI evolution, particularly in edge video surveillance applications, is significantly cheaper and faster GPUs with practically limitless storage. Hard disk drives (HDD) must be capable of writing data at high speeds to keep up with these AI applications and simultaneously support both AI and video workloads.

Software-defined storage technology has enabled rapid creation and migration of edge storage environments at the point where live data and Big Data analytics intersect to meet the need of local and mobile analytic workloads. The growing amount of data creation across the IoT universe in a mobile, real-time world is a fundamental driver of edge storage.

Recognizing the benefits of edge-based processing and analysis, storage solutions providers are developing or currently offer drives that offer high throughput and enhanced caching to deliver low latency and excellent read performance to quickly locate and deliver video images and footage analysis that does not require the power of an off-site datacenter.

Selecting a Hard Drive

The seemingly unlimited demand for data and the sheer amount of video being captured by surveillance systems are driving the need for more advanced and cost-effective storage options that are optimized for machine learning, deep learning, high-resolution video, advanced analytics streaming and much more. End-users and systems integrators alike are looking for reliable, high-capacity drives that are purpose-built for surveillance that support multiple cameras, 24/7 availability and the capability to maximize streaming and frame rate performance.

Given the increasing criticality of video for security and other uses, deploying the right recording and storage solution for the right purposes is critical. Organizations simply cannot afford to lose any data or video integrity, which surveillance AI systems hinge their accuracy and predictive powers on, and not all storage hard drives are created equal.

Additionally, storing more data for longer lengths of time is necessary for AI systems to become “smart,” increasing their predictive analytic capabilities. With all these factors in mind, there are a few best practices integrators can employ to ensure they are selecting the most appropriate hard disk drive for a particular customer application – and the workload associated with it – to ensure long-term success.

1. Select a solution purpose-built for the demands of video surveillance applications. Purpose-built drives with advanced technologies to allow both video streaming and AI analysis ensure the best possible drive reliability and performance, while delivering optimal TCO and the most valuable, actionable intelligence possible.

Consider that a typical desktop PC hard drive – which has no camera support – is built to work only eight hours a day, five days a week, or 2,000 hours over a full year. A surveillance hard drive, on the other hand, gets no down time, working 24 hours a day, which is more than four times as many hours as a desktop drive over the course of one year.

The differences in the way these two types of drives operate means they must be capable of performing under the extreme workload stresses to which they will be subjected in a deployment. Choosing anything less than a surveillance-optimized drive for video applications will result in a sub-optimal outcomes and even failures for video streaming quality, workload performance and analytical capabilities.

2. Look for health monitoring. Drives that offer some sort of health monitoring should be considered to provide prevention, intervention and recovery from potential data loss related to unexpected physical damage as a result of vandalism or natural disasters.

3. Account for proper installation, cooling and vibration. It is important to remember that beyond picking the right drive, proper drive handling and installation in surveillance systems is critical to overall performance, reliability and TCO. Always keep in mind that the drive must be cooled properly and the proper tools must be used for drive installation to avoid multi-bay vibration issues.

Jessica Burton is Global Surveillance Product Manager for Seagate Technology. Request more info about the company at