The Interpretive Camera: Surveillance Cameras Become More than Just an Eye

New surveillance systems use pattern recognition to differentiate between normalcy and potential crimes


With two cameras and a laptop computer set up in a conference room, Chellappa and a team of graduate students recently demonstrated how intelligent surveillance works.

A student walked into the middle of the room, dropped a laptop case, then walked away. On the laptop screen, a green box popped up around him as he moved into view, then a second focused on the case when it was dropped. After a few seconds, the box around the case went red, signaling an alert.

In another video, a car pulled into a parking lot and the driver got out, a box springing up around him. It moved with the driver as he went from car to car, looking in the windows instead of heading into the building.

In both cases, the camera knew what was normal - the layout of the room with the suspicious bag and the location of the office door and parking spots in the parking lot. Alerts were triggered when the unknown bag was added and when the driver didn't go directly into the building after parking his car.

Similar technology is currently in use by Marines in Iraq and by the subway system in Barcelona, according to ObjectVideo, a Reston, Virginia, firm that makes surveillance software.

ObjectVideo uses a "tripwire system" that allows users to set up virtual perimeters that are monitored by the cameras. If someone crosses that perimeter, the system picks it up, sends out an alert, and security staff can determine if there is a threat.

Company spokesman Edward Troha predicts the technology, currently designed primarily to protect borders, ports and other infrastructure, could be adapted to help prevent retail theft or guard private homes.

The Jacksonville Port Authority uses ObjectVideo software as part of its security measures to watch the perimeter of the Florida port that handles 8.7 million tons of cargo and thousands of cruise ship passengers each year. The surveillance system sends real-time video from anywhere at the port of possible intruders to patrol cars.

Still, industry officials say the technology needs to improve before it can be widely used. There are liability issues, such as if someone is wrongly tagged as a threat at an airport and misses a flight, said Bordes. Troha warns humans are still essential to intelligent video, to tell, for example, if a person in a restricted area is a danger or just lost.

And the cameras can only see so much - they can't stop some threats, like a bomber with explosives in a backpack. They can't see what you are wearing under your jacket - yet.

"That is an eventual goal, but we're not there yet," said Chellappa.

[Associated Press WorldStream -- 02/26/07]