Given the key role that surveillance video played in the aftermath of the Boston bombing, I’ve been giving some thought to how present and future surveillance technologies could more quickly and efficiently investigate such a tragedy. What law enforcement officials were able to quickly discern from the pile of video footage with the tools they had available is truly amazing. I can only imagine the number of hours expended on manual video review.
Two areas strike me as being appropriate to consider, and they both involve video analytics. The first area is identifying suspicious activity. Traditional video analytics provides certain tools, such as motion, direction, speed, loitering, object left behind and size. With these, one could ask the following types of questions: Is anybody in an area walking quickly relative to their neighbors? Is there erratic movement? Is there an unusual width to height ratio (suggesting a large backpack, or any backpack)? Was something put down and left behind? Did that placement follow a period of loitering?
While none of these answers would be conclusive in themselves, they appear to be the types of issues that video investigators would look for.
Defining Normal
While it may be a challenge, behavior-based analytics, from companies such as BRS Labs, might apply. Behavioral analytics looks for the abnormal, but it takes time to stabilize in order to define “normal.” Scenes that are brand new and involve constant change would be a challenge; but, what if a system could determine average speed, define average or lower speed as normal, and flag anything moving faster? Or, what if it could determine that a certain size backpack or hand-carried item exceeded the threshold for normal? It’s not that you would alarm on such events, but rather mark them to assist further review. Ideally, you would mark them for the entire duration the subject was within the camera system’s view.
Further, data could be correlated with non-video sensors. John Convy of BRS tells me that all of this is feasible. This could be especially powerful if coupled with video synopsis techniques.
Another approach parallels what was discussed for the use of facial analysis at Super Bowl XXXV (Tampa, 2001). Take subjects of interest and compare them with a data base to evolve a match. Several issues immediately arise. How good are the video images? Can they be converted to frontal poses for face matching? What and how relevant is the data base? If you could achieve some number of valid matches, it would constitute a set of candidates for investigators, and you could correlate those with results from traditional analytics to further narrow that set. Certainly, you would want to be able to mark that video once a facial match was achieved to allow later investigation.
Better Technology Means Better Investigations
Advances in technology and actual city surveillance deployments will help. Start with the cameras — in Boston, many cameras that just happened to be in the vicinity of the bombings provided the bulk of the video footage for review. Whether they were for building surveillance, or news, or whatever, the cameras were likely not optimal for use with analytics.
Megapixel cameras with rich analytic features and embedded storage, thoughtfully placed and with appropriate lenses, would be required. The video network could be supplemented with CBRN (Chemical, Biological, Radiological and Nuclear) sensors for added coverage. Already on the way are enhanced analytic and processing capabilities for the edge cameras, providing the means to mark the video with metadata allowing later review or use with Big Data systems.
With regard to facial recognition, there are several issues to be addressed beyond cameras. According to Joe Rosenkrantz, CEO of FaceFirst, “Both software technologies and choice of hardware, consciously deployed, are required for a successful system.”
While systems are theoretically capable of real-time (less than one second) identification, the number of matches (face to database) operations could be overwhelming, even though technology that synthesizes the best face from multiple frames of video. Options for dealing with a large number of matches include limiting the scale of the database, analyzing matches post-event, or tapping into more computational power to handle the match load.
Sadly, it takes a tragedy to both create motivation for certain technology developments and to soften public resistance for such deployments. A CNN/Time April 30 poll determined that 81 percent of respondents favored expanded video surveillance on streets and in public places, while less than half were in favor of giving up some of their civil liberties to be safer. Perhaps video is now being seen as less infringement and more necessity.
Ray Coulombe is Founder and Managing Director of SecuritySpecifiers.com, enabling interaction with specifiers in the physical security and ITS markets; and Principal Consultant for Gilwell Technology Services. Ray can be reached at [email protected], through LinkedIn at www.linkedin.com/in/raycoulombe or followed on Twitter @RayCoulombe.
