Eye on Video: Specialized intelligent video applications

Analyzing surveillance footage for license plates, faces and even fires

How it works
The process for facial recognition is similar to automatic license number recognition, except that you cannot pre-define what the system should look for, such as a string of numbers and letters in a specific order. Instead, the facial recognition system draws on a database of "wanted" faces, such as a passport photo database or a police register. Getting that data is often the biggest challenge in deploying the system. Once that database is accessible, facial recognition undergoes a multi-step process to match the video image to a stored picture:
1. Find the person in the image.
2. Isolate the face from the rest of the body.
3. Locate and identify the various features - eyes, nose, mouth, chin, skin color, hair color, etc.
4. Construct a unique pattern of an individual face.
5. Match the extracted face with signature information from a database to identify the individual.

In some applications, simply finding a face is sufficient. For example, in an airport, the IV application can be used to measure the queue time between entering and exiting a check-in point. In this case, the actual identity of the individual is not important, just the ability to separate individuals from each other.

Challenges in deployment
Facial recognition applications are particularly challenging to deploy. Even under perfect lighting conditions, people generally move around and block each other. Appearances change over time-or can be easily modified with glasses or hair dye, a wig or facial hair. Also, people who freely move about rarely look straight into a camera. Three-dimensional recognition compensates for this problem by extracting 3D information from video streams and matching it with a database.

Fire and smoke detection
Fire and smoke detection systems search for visual cues of fire and/or smoke in the video stream. They react as soon as flames are visible in a room-or when reflected firelight is detected from an obstructed view-without having to wait until a certain level of ambient smoke appears or a pluming smoke cloud reaches the ceiling. This gives facilities managers a much earlier warning of problems than a simple smoke detector. But liability issues stemming from archaic fire codes and the labor intensity of sending engineers on-site to customize the algorithm's parameters to the specific installation have limited widespread adoption of the technology.

How it works
The IV systems processes video images and reacts when it detects the combination of color, light and movement that typically indicate the presence of fire and/or smoke. This could be the flickering frequency of certain pixels, the spatial dimensions of certain blob contours, or the existence of turbulent phenomena. Once detected, the IV system sends an alarm and live video images to the guard station or alarm center. Besides supplying vital situational information, the recorded video also provides forensic evidence for future fire investigation.

Challenges in deployment
The technology still faces a number of challenges. Foremost is matching images to an endless variety of smoke patterns. For instance, the visual pattern of smoke in a windless, open space differs from a noticeably windy environment. Upward smoke puffs look different from smoke spreading horizontally or downward. Because of the difficulty in adapting to light conditions, density and background scenes, setting thresholds for smoke detection can be very subjective.

Addressing privacy issues
Some view IV applications such as facial recognition and people counting as invading people's privacy. One way to counter this concern is to regularly purge the actual pictures or videos of the faces. In some ways, IV applications can even enhance privacy. A people tracking application, for example, may be able to find and mask out all the people in a video surveillance recording of a public area and then allow only law enforcement personnel to "unlock" these images when they are required in an investigation.

Keep in mind, however, that some countries place restrictions on audio and video surveillance. Before designing and installing any IV application, be sure to check with local authorities first to determine what is permissible.

Pointers for successful deployment
As with other intelligent video applications, whether you decide to deploy this video intelligence at the server or distribute it to your surveillance system endpoints depends on the equipment you use and the demands of the environment in which it is being operated. (For more information on intelligent video architecture, see the SecurityInfoWatch.com article Intelligent Video Architecture: Deciding whether to centralize or distribute your surveillance analytics.)