Fredrik Nilsson is General Manager, Americas, for Axis Communications and a regular contributor on topics of IP video for SecurityInfoWatch.com, IPSecurityWatch.com and Security Technology & Design magazine.

An object-based IV system detects and segments a moving object first, then classifies it and provides a set of metadata that describes the object.

Picture courtesy of iOmniscient Intelligent Surveillance.

Surveillance system applies queue management to airport ticket lines.

Picture courtesy of Vidient Systems.

Surveillance system detects tailgating into access-controlled facility.

Picture courtesy of iOmniscient Intelligent Surveillance.

Surveillance system monitors crowd congestion.

A virtual fence can be set up to detect and send an alert if someone enters the rail or subway track.

Picture courtesy of Agent Video Intelligence, Inc.

Object-based analysis can detect abandoned objects.

A multi-camera tracking system tracks a person as he/she moves from one camera view to another.

In deploying intelligent video systems, camera placement determines accuracy.

The previous column in this series (see Applying Pixel-Based Intelligent Video) discussed pixel-based intelligent video (IV) applications that send alerts to security personnel when a camera detects a significant number of pixel changes in the view window. This month's article focuses on object-based IV applications and the sophisticated algorithms that enable systems to recognize and categorize objects in a video frame. Next month, we will discuss specialized intelligent video applications.

Object-based intelligent video applications fall into two broad categories: object recognition/classification and object tracking. Whether you decide to deploy this video intelligence at the server or distribute it to your surveillance system endpoints depends on the equipment you use and the demands of the environment in which it is being operated. (For more information on intelligent video architecture, see the article Intelligent Video Architecture: Deciding whether to centralize or distribute your surveillance analytics.)

When to use object-based analysis

Object-based analysis is a useful surveillance tool for observing the behavior of objects, such as monitoring the flow of traffic in a given area. It is also useful in counting objects within a scene, such as the length of a checkout line in a supermarket. Keeping track of objects, such a suitcase abandoned in the middle of a busy airport or someone loitering in front of an automatic teller machine, is another important capability of the technology.

How classification works

IV applications go through three steps: detection, segmentation and classification. In detection, the system analyzes all the pixels in a sequence of video frames, comparing pixels in each frame to a reference frame in order to determine what objects are moving. In segmentation, the system extracts the moving objects that were detected and assigns them descriptive signatures such as color, size, direction of motion and time. In classification, the system takes the segmented objects and categorizes them into different object types, such as a person or a car, and assigns them a set of descriptors that characterize them by such attributes as color, size or direction of motion. This descriptive information is called metadata.

Once the system creates metadata about objects in an image area, it then applies a set of criteria. For instance, if the metadata shows a person walking the wrong way, an individual abandoning a bag, or a car entering a restricted area, it can raise an alert in real-time or retrieve the appropriate video from storage to be used as evidence in criminal proceedings.

Counting objects

Once the IV system detects and classifies an object, you can leverage the information for a variety of purposes. People counting-especially in retail environments-is one of the primary applications for this technology because it provides a wealth of data to assist store managers in optimizing store layout and customer service.

• Analyzing customer traffic patterns - Retailers can count the number of people entering and exiting the store, passing through certain aisles or stopping by a particular merchandising display to leverage sales of impulse items or groups of items.

• Managing queues - Management can count the number of people standing in a queue for service, such as at a checkout counter or at the airport ticket counter or passport/security control point to calculate when to open more stations.

• Detecting tailgating - Object-based IV applications are particularly valuable in access control because the system can send an alert when one person swipes an access control badge to open a door and multiple people enter the facility.

• Crowd control - The ability to count the number of people gathered in a particular area of a scene means the IV system can send a warning when a capacity threshold is about to be exceeded or when something is impeding traffic flow.