Troubleshooting Video’s Worst Performance Bottlenecks

Aug. 21, 2014
With more functionality being placed on video systems, better planning is crucial for successful deployments

When it comes to video recording for security and surveillance purposes, it’s all about performance. How quickly and reliably can you capture and retrieve video data? When a system is responsible for safeguarding property, assets – and even lives – the ability to reliably retrieve and review recordings is the most important objective of any video surveillance deployment.

In this article we will examine some of the issues that cause poor video performance, the effects subpar video can have on a system and finally, steps that can be taken to improve the situation.

Causes – Why aren’t surveillance systems performing up to par?

Today, the growing adoption of HD/megapixel technology, which delivers clearer images and more vibrant colors, is making a significant impact on video recording performance. Ultra-high resolution video, continuous streaming  and extended retention times are maxing out the capabilities of traditional storage approaches, such as using analog cameras and VCRs. Mobile devices are also a contributing factor, as major manufacturers of security systems are providing accessories for phones – giving users access to their systems remotely and further increasing the need for performance optimization.

Users want it all and they want it now – they want to centrally manage and store video, effectively handle the higher resolution video from HD/megapixel cameras, record at higher frame rates, and scale to massive capacities – as well as accommodate future growth.  Additionally, the wider adoption of video analytics that detect specific behavior in real-time and provide valuable information to organizations increase the demand for optimized performance. With the market moving to the high end, particularly in high-security environments, where detailed video is critical, performance bottlenecks – those pesky issues that slow or halt performance – are becoming a critical concern that must be addressed. 

Effects –How does performance impact reliability?

Performance bottlenecks can lead to frame drops (video loss), meaning the recording service is likely discontinued and live view or playback won’t be available to user requests in time to make timely risk assessments or react to alert events.   It can also lead to latency (down time). A loss of performance throughput can decrease the number of supported cameras on your network. It can also present more opportunities for bottlenecks as increased loads on individual cameras place additional capacity on the processor, network infrastructure or storage, and in recording/display applications.

A video surveillance system does many things at once, and user requests – playback, for example, make the load that much heavier. If you are running intelligent video analytics (i.e., facial recognition, license plate identification), the surveillance system now takes on an even heavier load. Other common culprits when it comes to bottlenecks include: core processor power; host Input/output (I/O) interface between server and storage; media for video data storage; networking; storage array background activity; video management software (VMS); and different use cases. Processor capability, frame rate configuration, type of storage media, array controller settings, connecting interfaces and network traffic conditions will all affect performance as well.

Let’s take a look at a typical scenario: In a large scale airport deployment, a newly configured storage array in a surveillance system begins storing video from multiple cameras throughout the site. At this point, streaming performance is steady, as there is nothing previously stored in the disk volume.  The controller just has to accept video feeds and write them simultaneously to the designated logical volume until the video data fills up the disk space.

Now that the disk volume has been filled, the controller starts to manage things differently. It’s still handling the video feeds from the cameras, but now it also must erase old video archived in the disk array. While all of this is going on, airport security receives a tip and starts to play back a designated timeframe at a security checkpoint from the footage it has archived. Being state-of-the-art, the system includes intelligent video analytics, and is looking for specifics that can be run through a facial recognition system.  If you add in dealing with and rebuilding from errors – all while staying online and continually capturing and storing live feeds – it’s easy to see how bottlenecks that result in highly unstable performance can occur.

Solutions – Optimizing performance

What can be done to eliminate bottlenecks? In general, there are quite a few strategies that can help reduce the load on the surveillance system – and thus reduce the chances for bottlenecks to occur. Upgrading your core processor is a good place to start. Offloading things like networking, graphics, encoding and decoding is another step you can take to relieve the workload on the processor.  An absolute necessity is employing a high availability RAID array optimized for video surveillance storage that delivers consistent and dependable performance that can create the foundation for a robust surveillance infrastructure as well as bring reliability and stability to handle errors and decrease service downtime.

Adopting a faster host I/O interface and storage media is another option when aiming to reduce performance bottlenecks. Additionally, you can also use port trunking (i.e. Ethernet link aggregation); reduce the number of variables present (narrow application categories, fix configuration scenarios) and improve storage I/O error handling and throughput – while decreasing I/O latency.

When looking to improve performance, where have some surveillance systems gone wrong? They often do not offer robust reliability, with appropriate disk I/O error handling to ensure continuous recording without dropping frames. Solutions can lack stability by not correctly managing disk I/O latency – particularly when it comes to write functions. They may not offer higher levels of software integration capabilities – with SDK or plug-ins, for example. Systems also may not come with the recommended configurations for video recording. In addition, video won’t come in at the same size and format, since in a deployment; it’s unlikely all cameras used provide the same image quality and recording rate.

 Some cameras record at a higher megapixel level, while some only need VGA level recording, and this leads to the need to simultaneously archive different video packet sizes, which creates a heavy load on the processor to write to the disk array. This can ultimately increase the possibility of frame dropping. Be aware that some surveillance systems may not be able to scale to meet future user needs, and are costly and complicated to deploy. This is particularly evident when users deploy onboard storage to archive video without considering the possibility for future expansion needs through external storage. This means the user has to replace the entire existing infrastructure to scale to a larger base, which is very costly and time consuming.

When talking storage, stable, flexible, scalable, high-availability, easy to deploy and affordable are all attributes that should be in the surveillance solution equation. Simple management and serviceability can be achieved by looking for things like hot swappable drive bays and redundant power supplies which eliminate unscheduled downtime. Look for storage systems that are proven with resilient performance – that ensure that video will continue to record – even when the system is undergoing array rebuild. It is best to see if the system can bring reliable throughput for multiple, simultaneous video feeds – without dropping frames.

About the Author:

Eddie Huang is the Product Planning and Marketing Manager for PROMISE Technology, which is a recognized global leader with 25 years of experience in the storage industry. PROMISE has developed its own enterprise-class hardware and software storage architecture, in addition to complete SAN, NAS and Unified storage platform product lines.