Eye on Video: Specialized intelligent video applications

Analyzing surveillance footage for license plates, faces and even fires

In the previous two months, I devoted columns to two broad categories of intelligent video (IV) applications: pixel-based video intelligence and object-based video intelligence. (See the articles Applying Pixel-Based Intelligent Video and Applying Object-based Intelligent Video). Pixel-based IV applications generally apply to detecting motion and camera tampering. Object-based IV applications are generally used to classify and track objects and people. A third category of IV applications combines techniques from both pixel- and object-based analytics to extract information from a video for specialized applications. These specialized IV tasks cover the range from automatic recognition of license plate numbers and individual faces to smoke and fire detection.

License plate recognition (LPR)
Having the intelligence to automatically recognize a license plate number can boost a facility's perimeter and interior security. It can also provide a wealth of information for astute retailers to leverage for target marketing.

• Access control. LPR could be used to restrict entry to vehicles with particular plate numbers, an appropriate application for high-security locations like embassies.
• Criminal investigation. LPR could help law enforcement locate a vehicle suspected of being involved in a crime by automatically look for vehicles with a particular plate number.
• Intelligent parking. Using LPR to automatically track vehicles entering a parking lot would be less expensive than a parking ticket kiosk. The technology could also automatically monitor how long a particular vehicle stays in a parking lot, which could help attendants determining if a vehicle has been abandoned or a shopper has monopolized reserved space beyond the legal time limit.
• Retail marketing. LPR could help retailers identify frequently cars parked in front of certain stores. This information could be used to analyze shopper demographics or design marketing programs to reach consumers in the right geographical areas.

How it works
License plate recognition is a multi-step process:
1. Find the car in an image - either through blob recognition or video motion detection.
2. Isolate the actual license plate - through object recognition and classification.
3. Extract the letters and numbers from the license plate - through image analysis.
4. Transform the collection of pixels into a stream of letter and numbers - using optical character recognition.
5. Store the resulting string in a database or compare it with existing entries - through data processing.

Challenges in deployment
Bad weather, light shining from headlights, dirty or bent license plates can all affect the accuracy of the analysis. Also, license plates have different physical attributes in different parts of the United States and around the world. So the IV application needs to be customized to local conditions and fine-tuned to specific implementations. Many integrators use specialty cameras in order to provide their clients with the best images possible.

Facial recognition
The profusion of crime scene investigation shows on television has made facial recognition one of the more high-profile applications of intelligent video. Police use the technology to receive alerts when certain people of interest are seen in public places or in sensitive areas. Companies use the technology to enhance access control, allowing only certain individuals to enter specific areas. Forensic investigators use the technology to search for individuals in stored video recordings. Casino owners use facial recognition to catch blacklisted players. And border agents improve checkpoint control by augmenting manual passport checking with automatic searches for individuals of interest.

This content continues onto the next page...