Video surveillance has changed. The days of a minimum-wage security guard watching a few cameras, looking (or not) for something unusual are gone. We can't just add "dumb" cameras to increase security; that only results in a need for more guards, who become ineffective after only 20 minutes of watching video monitors. Think about it: After a guard has been viewing surveillance monitors for millions of seconds, the likelihood that he would notice a two-second terrorist event, comprehend it and react effectively is reduced to pure chance.
The good news is that the market for intelligent video, often described as motion, pattern and behavior analysis, is growing. This software is intended to watch everything and send alarms only when it notices "interesting" situations. Guards respond only to what the analysis tools find alarming, suspicious or unusual. Although challenges remain, that's the promise.
Of course, despite the emphasis on technological innovation, the whole configuration is mediated by humans. If there are too many false alarms, for example, guards will turn off the system-no matter how intelligent it is reported to be. That said, there is still much to admire in the progress that the fledgling smart video industry has made over the past year.
Getting Perspective on the Technology
Video-based behavior analysis is the interpretation of events and patterns that are determined by detection and tracking of objects. Once an object is detected, its movement and location within a scene are used to determine if there is an event-say, crossing a threshold or speeding. This might require handoff and tracking of the same object among multiple cameras.
Ideally, the object can be identified through some set of attributes, which may also help in the interpretation of the behavior. Nonetheless, the most basic design choices in the video and optical system-for example, progressive vs. interline scan, field of view, compression, sensitivity and lighting-will make or break the ability of the system to provide such an advanced capability.
The system determines behavior by detecting a series of events that have a pattern or meaning in space or time. Example: A vehicle is detected repeatedly circling a critical building, counter to the normal traffic pattern. In order to provide an effective alert, the system must (1) have a clear enough picture to recognize identifying features of the car across all cameras; (2) track the car and associate these features among any number of cameras; and (3) receive or automatically generate and apply a threshold that determines the normal frequency of the event, and then compare it to the occurrence.
Once the information is detected and translated into data at the event level, the video itself is no longer relevant to the analysis. Now software from other fields, like radar, can be used to interpret the data.
The querying and forensic analysis of pre-recorded video can be just as important as the processing of real-time data. A good, content-based retrieval system allows you to scan through weeks of pre-recorded video within minutes, facilitating the location of critical information about potential security breaches. Ultimately, this can be used to understand the behavior of the attacker: a critical component of forensic analysis.
The State of the Art Today
The state of the art is rapidly advancing as both government and industry pump funds into developing better analysis tools. Purchasers should breadboard solutions before they proceed into full-fledged implementation. Military agencies are currently scrambling to develop and provide guidance and standards for video-based security systems. Until they succeed, the market will remain a formidable mixture of promise and performance.
There are four major aspects of intelligent video that reflect the effectiveness of the total solution: moving object detection, object tracking, recognition, and behavior and event pattern analysis.