Tech Trends: Why Big Data is a Big Deal

Feb. 12, 2020
A turbo boost from Artificial Intelligence has opened up many new uses, and many new questions as well
This article originally appeared in the February 2020 issue of Security Business magazine. When sharing, don’t forget to mention @SecBusinessMag!

For some time, I have been hearing about how “Big Data” is a top technology trend in the security industry; however, for most of us, it is hard to see how Big Data tangibly relates to what we practice every day.

At CONSULT 2019, I arranged a panel moderated by Mark Schreiber of Safeguards Consulting that included experts Dr. Werner Metz of Intel, Greg Skarvelis of Verint and independent consultant Ryan Parker to connect the dots. Here’s what I learned:

Big Data Defined

As its name implies, Big Data represents a collection of interrelated or loosely related data that, when analyzed, provides a basis for analysis or decision making. According to Dr. Metz, the process means “looking for patterns in data – it is as simple as that.”

We have all looked for trends in data one way or another – whether it is the stock market or the weather. Using the stock market as an example, it is impossible to infer total market performance from the performance of just a few stocks; however, the S&P 500, which represents 500 companies and much more data, can give someone a much better idea of where the market is headed.

Applying it to Security

In security, there is a lot of data to be analyzed, particularly when surveillance video is added into the mix. I view the security application of Big Data in two main areas: Predictive analytics and situational awareness.

Predictive analytics: Many may be familiar with the concept of heat maps used in law enforcement – using extensive data about the frequency, location and time of committed crimes, police can build a model of when and where certain crimes are most likely to happen. This assists the pre-emptive deployment of resources to be able to better prevent or interdict events, such as break-ins, robberies, and drug deals. The effectiveness of the tool is based on having enough data to make valid determinations, as well as the algorithms (analysis tools) to process that data.

Heat maps have also been used in retail, but more data combined with better algorithms will improve them, assisting in loss prevention, merchandising and customer service efforts.

While big data has meant big money in professional sports analytics, expect it to also improve stadium/arena security via improved traffic and people flow analysis, as well as behavioral analysis.

In implementing systems such as these, the expert panel emphasized the importance of determining customer needs from the beginning to help direct the data collection and analysis and to inform any associated RFP.

Situational awareness has become an industry norm, for good reason. Products like the Verint Situational Awareness Platform provide a means to fuse data and video from many different sources in order to present a coherent and integrated display tailored to a customer’s particular needs. Verint touts its “single pane of glass” to “maximize analysis, response, investigation, evidence gathering, incident reconstruction and debriefing.” What fills that glass is data, and more data means better performance…or does it? It really comes down to the techniques and algorithms used to filter and process that data.

Artificial Intelligence (AI) – despite its current status as an over-used buzzword – when properly developed and implemented, relies on Big Data to train its underlying models to recognize people, objects or occurrences. The more data analyzed, the more effective the training and performance becomes.

Video has always been a challenge, and billions of operations per video frame are required to extract, filter, sort, analyze, etc., to ultimately give a process the ability to make an accurate inference from surveillance video.

Parker thinks that data from microphones will be useful within five years, adding to the many terabytes that must be crunched. To make this process manageable, Dr. Metz argues for significant computing to be performed at the edge, for which Intel has made a sizable commitment. In fact, it is that continuous march of processing power – edge, centralized, cloud – and more and cheaper available storage that has brought us to this point.

Big Data and Privacy

A significant elephant in the room is data privacy. Increasingly, in response to the unabashed actions of those who have collected personal data from the trusting, unsuspecting public, many are waking up to and demanding to know how their personal data and images are used.

Several questions arise, starting with the training set of data. Where did it come from? Is the origin of the data broad enough and relevant enough to be applicable to its intended use? Who owns the data, and do they have a legitimate right to sell or license it? Was it collected in a private or public venue?

Suppose an integrator is working on subjects and behaviors in a sporting venue. What rights does the team or league have? I was involved in a project for the Department of Homeland Security a few years back, where a particular airport owned the cameras and all the data, prompting the need for a strong agreement between the airport authority and the government about the use of that data and its security.

More questions: How is the data being secured? Cloud or local storage? Who has access? What happens to the original data once training is complete? Do local police have access? Has personally identifiable information (PII) been scrubbed from the data set so it cannot be correlated with a particular individual? If so, the privacy risk is mitigated.

These questions lead us to the topic of governance, where the possessors of the data must understand the interrelationships of associated business operations, compliance, and policies and procedures.

For all of us in the security food chain – product and service providers, integrators, consultants, end-users – it is important to understand that this technology is upon us, and it is being made more powerful thanks to the AI turbo boost. With its many uses and benefits comes a responsibility that must be taken seriously.

Ray Coulombe is Founder and Managing Director of SecuritySpecifiers and the CONSULT Technical Security Symposium. Reach him at [email protected], through LinkedIn at or follow him on Twitter, @RayCoulombe.