This year began with a focus on technologies with high relevance to the security industry, and a record-breaking 170,000 attendees at CES experiencing 4K/UHD sources and displays to expanding embedded systems and emerging trends like autonomous vehicles actively leveraging the video analytics and mapping industries. Many of these have industrial technology counterparts, contributing to more secured cities and also used by another significant initiative, the “smart” city.
Indeed, cities are the future. In the United Nations “World Urbanization Prospects” Report[i] stated that in 2008, for the first time in history, more people are living in cities than in rural areas. By 2050 nearly two-thirds of the world’s projected 9.7 billion people will be urbanized. Dense populations put great stress on city services, leading to greater focus on technologies that improve operations, prevent loss, improve safety and security and are essentially “smarter.”
It is no accident that on November 15-17, Houston, Texas will host the leading public safety security technology conference, Secured Cities (www.securedcities.com). Houston is a city with nationally recognized medical, public safety and commerce leadership, together with the ability to keep pace with the United States’ second highest growth rate of 6.7 percent from 2010-2014. It has one of the most sophisticated Emergency Operations Centers and efficient Next Generation 911 Centers in North America. Finally, a successful datacasting pilot program has given first responders and city management an innovative alternative infrastructure for communications and video surveillance.
The Internet of Things concept clearly emphasizes mobile devices, connectivity, video surveillance, wearables and actionable data solutions as the technology trendsetters for 2016. But how do we identify and differentiate technology’s often cyclic hype and long-term acceptance?
A “hype” cycle[ii] is a graphical representation of the stages a technology goes through from conception, marketability, expectations management, maturity and finally, widespread adoption. Initially a “Technology Trigger” may lead to “Inflated Expectations,” an adjustment, followed by gradual, sustained market acceptance, or the technology “promised land” (See Table, “The Hype Cycle: A Technology Assessment Tool”)
Some technology, like video analytics, even with the great sophistication of solution providers and great benefits already realized by the security and public safety industries, in past years and some markets had been “stuck” in a valley of lower acceptance, but is now in a gradual, sustained growth phase (see Hype Cycle – Data Sciences).
Others, like autonomous vehicles, may be in for a gradual evolution and shake-out, whereas field autonomous vehicles in supply transport, warfare, and planetary excursions are more widely accepted (see “Hype Cycle, Emerging Technologies). Autonomous vehicles benefit from powerful embedded systems with GPU (graphic processing units), 4K/UHD camera systems, advanced video analytics and geospatial intelligence, all of which are making a greater impact on the security industry.
4K and UHD
4K and Ultra High Definition (UHD) video content has approximately four times the resolution (4 times the number of actual pixels) of 1080p full HD. 1080p content has more than twice than 720p resolution. Add audio, metadata, potential multiple camera streams and you are really referring to data-rich Digital Multimedia Content (DMC). You will also be looking seriously at methods of limiting bitrate, calculating and reducing the storage footprint and the ability to find objects of significance, especially where this content is actually being used as Digital Multimedia Evidence (DME). Include the ability to search through metadata, and you are more likely to be comfortable with large amounts of unstructured video data.
A “next generation” codec includes an encoder that offers efficient, real-time compression of video, audio and metadata for more efficient streaming, decoding, and storage, ultimately taking up less disk space. The decoder extracts the audio or video information from the compressed video stream in real time or for forensic review purposes.
Next-generation video codecs, when applied to 4K/UHD, 1080p Full HD or 720pHD content can result in three areas of efficiency:
- From 46 – 73 percent reduction in bitrate or reduced storage over HEVC, depending on use case (The results are presented in the table “Comparison of HEVC bit rate savings over AVC and Zipstream bit rate savings over AVC”[iii])
- Increase the number of cameras as allowed by the bitrate or storage savings
- Increase frame rate or image quality as allowed by the bitrate or storage savings
Next generation codecs make it possible to deliver mission-critical video streaming at reduced bitrates. These efficient video compression technologies make it possible to use higher resolution and increase forensic detail while reducing storage cost and enabling longer recordings. The core competence of a next generation codec is the ability to enable high bit rate in scenes with significant detail in combination with low bit rate when the scene is relatively static.
4K/UHD content from surveillance cameras is providing law enforcement professionals with richer forensic content able to render greater detail during investigations. More license plates can get decoded; more face reference points identified for facial recognition programs and richer scene data for 3D modeling.
From NextGen to Fog and Edge Computing
Whether you are Law Enforcement, Fire, and EMS or in Corporate Security, richer video content means more sophisticated transport of DMC that could save a life. How long will the first responder have to wait for this content to reach their consuming device? This period of time is the “latency” which should be kept at a minimum at all costs. Also to reduce latency, the use of Mobile Edge Computing (MEC), an accelerating trend in the Video on Demand (VoD) industry will reduce that latency and get critical data to a first responder’s smartphone fast.
MEC architecture makes use of power embedded devices for storage of content closer to the “edge” or DMC source to reduce latency when the feeds are requested. (See Figure)
You may have heard of “Fog” computing architecture, similar to MEC, but supporting a centralized data center where higher computational analysis takes place.
We will see growth between 10 and a hundred video streaming devices for each mobile communications user – even now many people have a phone, tablet, laptop and a few Bluetooth-enabled devices. MEC gets the security content to these mobile users faster and with the opportunity to have analyzed actionable content.
Video Analytics Rise in Multiple Industries
Video analytic tools developed by various collaborators for transportation data processing, in particular, road traffic, motorized and non-motorized. The maturity of these will pave the way, literally to autonomous vehicles, a rapidly expanding industry. The open source projects include tools for the most typical transportation data type, trajectories, i.e. temporal series of positions. The original work targeted automated road safety analysis using video sensors.
Virtually every smart city presentation lists the advanced warning of traffic congestion and optimized vehicle parking as great opportunities for cities to save money and get safer through video analytics, but the greatest impact will apply this tech to reduce critical events like wrong way drivers, highway/street racing, and obstacles. In fact, recently at the ASIS New York City Show, several new products were introduced by device manufacturer Optex using detection sensors that leverage camera-based video analytics for confirmation. These perimeter multi-spectral intrusion detection sensors communicate directly with cameras having embedded advanced video motion sensors for verification and greatly reduce false alarms. Both city agencies and corporations will find these now mature technologies essential deployment practice. This same company just launched a product that uses LIDAR to report on railway obstructions. Use these sensors and video analytics together and we’ve taken a leap forward to having the possibility of autonomous long distance rail transport.
Person, object and vehicle video analysis and modeling tools have entered the open source communities in 2016. This will allow more users to model and analyze the motion of objects in security videos. By overlaying simple dynamical models directly onto the security DMC, everyone from operations managers in busy airports to entry screening security officers may see how well a model matches the real world.
Analytic processing of DMC leaped forward in 2016, yielding significant process improvements for public safety, corporate security, retail operations and loss prevention, school security and the central station monitoring industries:
- Combined video, audio, and location data with acoustic signature analysis (see Louroe Electronics and Sound Intelligence)
- Camera-embedded and server-based retail activity mapping and path analysis (See Prism solutions)
- Camera-embedded and full-featured edge processing of vehicle license plate data (See IP Configure IoT solutions)
- Camera-embedded and cloud-based analytics with prescriptive, ensemble and deep learning (See AgentVI and Innovi)
- Camera-embedded facial recognition at personnel entry turnstiles, and biometric input devices (See Orion/BlueLine/Morpho integrated solution)
Most of the above fall into the now-mature “Forensic” category: what has happened, security evidence, analyzed quickly. The Sound Intelligence/Louroe acoustic signature analysis and omnidirectional microphone adds gunshot recognition, “aggression” or “angry voices” detection, glass breakage and vehicle accident acoustic signatures. Gunshot detection systems are gaining wide appeal; in a recent news conference, NYPD officials reported that the “Majority of shootings picked up by NYPD’s ShotSpotter system would otherwise go unreported,” and only 48 percent of those shooting incidents had 911 calls connected with them. Now, users have the ability to confidently deploy solutions based on acoustic signature at a single camera location or location triangulation using multiple cameras.
Workload Management
There were multiple announcements of video management companies incorporating “retail” style analytics like activity mapping and direction/temporal visual data at recent trade shows, but the real technology to watch put some of these processes in edge devices like network cameras.
The use of activity tracking and “heat mapping” temporal analysis is applicable not only to retail but well sought after by airports and logistics markets. Cameras are being repurposed for passenger “flow” over time, which helps allocate industry’s biggest cost, personnel. Prism has camera-embedded activity mapping applications that may help retailers better understand their customer’s buying habits. Several transportation centers are using these Forensic and Predictive analytics to determine passenger flow and security screening queue line management.
The Predictive analytic processes do much of just that - security models created are reviewed on the basis of what might happen. Actionable security data is used and a feedback system that tracks the outcome successful or not, produced by the action taken.
IPConfigure’s Orchidä IoT platform leverages the powerful edge processing built into mid-high end network cameras and microcomputers such as the Razberi Pi-3, making it possible to perform complex NCIC[iv] queries.
The first analytic solution to use deep learning and cloud-based analysis, AgentVI’s Innoviä use a patented image processing architecture that distributes the image processing task between a camera edge device carrying an “agent” and a server. This unique architecture eliminates the need to stream video to the cloud, useful in low bandwidth environments and allowing the numbers of cameras to scale and be in geographically diverse locations, connected to the analytics service. Computerized vision, deep learning, and a pay-per-use model encourage the growing breed of central monitoring operators taking advantage of analytics for video verification and false alarm reduction.
In the Data Sciences and Analytics Hype cycle, we can see multiple technologies at different stages, with video analytics and ensemble learning well on the way to maturity.
Orion Entry Controls, Morpho biometrics, and Blue-Line external camera-based Facial Recognition Software combined to provide multi-factor entry verification using high-speed optical turnstiles. This trend of placing multiple, powerful devices at the “edge,” or in this case at a building entrance or opening can result in throughput of one person per second, combined with biometrics and real-time camera-based multiple facial recognition processes, perfect for high traffic lobby security.
Security Data Science is a Mentality
What used to be called “big data” as an emerging trend has actually fallen off the “hype cycle” and is now impacting many different markets including security, with examples mentioned in our analytics trends overview. To understand what could be the most important trend requires a change in the perspective of the security professional to the core assets of data, collecting edge devices and the processes that make the data “actionable.” Are you security analytics-driven or challenged?
One approach is to “store first, ask questions later,” and use security data science to ask the right questions of your unstructured data. The perspective that all security data is useful may move decision making from legacy processes to progressive data-driven approaches. From audio forensics to activity mapping, we can see how both big and small data impacts security management.
As data continues to grow and security hardware gets more capable, those requiring the “best” data quickly can generally forecast their security data capacity demands, and are willing to dedicate hardware. Also, we’re ingesting all this security data, but where will it go and who will use it? Introducing the Internet of “Security Things,” composed of producers, analysis, storage and consumers of all things related or leveraged by security data (see figure).
All this data produced by the “Security Things” like network cameras, access control devices, and perimeter sensors might not yet be actionable, but will be once our forensic-predictive-prescriptive analysis is performed. Then the data is on to consumption, but is it really being leveraged to its full potential?
The variety of software and tools to treat data will continue to expand, but the value of the data will be realized on “data democratization” or the creation of a security data ecosystem where it is leveraged for purposes contributing to a “smart city.”
- Save Energy – use video and audio sensors to determine occupancy and pedestrian flow and network resource allocation
- Reduce Investigation Time: use video analytics to search for the object type in question, not everything
- Improve surveillance coverage, so that when an amber alert is active, you can find this child.
- Improve parking efficiency automatically, and reduce emissions and costs
- Increase traffic safety: advance congestion warnings, wrong way driver, speeding detection
- Network – reduce storage costs – improve storage and connectivity requirements of video sources like IP cameras
- Perform city-wide tracking of the most important assets
- Scale the city ecosystem through agility: do more with less in more places
- Detect symmetric or asymmetric attack profiles; not everyone will come in single file into the bank branch just replenished with cash
- Reduce energy consumption and accommodate urban growth
With all of these smart and secure city goals, connectivity remains as the enabler and proliferator of security sensor deployment.
Datacasting
During the simulated events in the datacasting pilot, the City of Houston and University of Houston successfully pushed their content over the KUHT television signal to their own and multi-jurisdictional users. All content was encrypted using AES-256 encryption.
KUHT continued to broadcast normally. The general public watching television experienced no quality of service interruptions and had no idea that encrypted public safety data was also being transmitted at the same time.
Simulated events included suspicious individuals entering a dorm on the UofH campus and gang activities at NRG Stadium with. In both cases, subjects were tracked as they moved about and fled the scene. Multiple cameras were used to follow the suspects fleeing the scene of simulated criminal activity. Building blueprint data was transmitted to all responding agencies. Images of the suspects were emailed back to Command and then datacast to all public safety responder/participants. (Attend the session: "Datacasting: Public Safety Broadcasting Capabilities to Responders in the Field" on Wed., November 16 at 3:40 PM at Secured Cities.)
The coordinated event was deemed a success.[1]
Conclusion
Not every technology follows the “hype cycle,” but it is extraordinarily important to recognize those useful technologies often just beginning the climb to maturity and multi-industry acceptance. A great example of this is video analytics, whose developers have leveraged established and emerging technologies to create real cross-industry collaboration tools. Next generation codecs are another; after all, what good is less data when such sophisticated analysis is available. And what of these technologies will best accommodate safety, security, and sustainability in our world’s global urbanization trend? The answer may lie in that professional engineer’s statement on the need to work together on common causes, special causes, and practice Epistemology, the theory of knowledge: “In God we trust; all others bring data.”
About the Author: Steve Surfaro is the Industry Liaison for Axis Communications. He also serves as Chairman of the Security Applied Sciences Council for ASIS International and Vice Chair of the Security Industry Standards Council.
References:
[1] “City of Houston Collaboration Leads to Innovative Solution for First Responders,” date last updated (21 December 2015). Web. Date accessed (06 May 2016).
[i] “World Urbanization Report,” United Nations, 2014 revision. Web. http://esa.un.org/unpd/wup/Publications/Files/WUP2014-Highlights.pdf
[ii] The hype cycle is a branded tool created by Gartner, an information technology (IT) research and consultancy company.
[iii] Video Quality Evaluation Methodology and Verification Testing of HEVC Compression Performance, IEEE Transactions On Circuits and Systems For Video Technology, Vol. 26, No. 1, January 2016, http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7254155
[iv] The Federal Bureau of Investigation (FBI) compares license plates against its National Crime Information Center (NCIC) database. As law enforcement agencies take advantage of advanced technologies, the opportunities for using data to help with investigations increases greatly. LPR cameras locate license plates within an image, decode them using automatic number plate recognition and character recognition.
