Inside the NPS

April 12, 2012
A look inside the judging process

For more than five years I’ve been a judge for SIA’s New Product Showcase (NPS) and I wanted to share my insights into the program, which has grown and blossomed into one of the best-known security industry innovation awards programs. This is thanks in large part to hard work by all the participants, a credo to be fair and unbiased and a commitment to furthering the industry by presenting accolades to those products that ultimately assist the integrators and end users with pertinent solutions for their market.

This year, we had quite a few new judges participating, because the NPS process is extremely time-consuming and people change jobs and positions. All judges must sign an affidavit of no conflicts of interest and follow the SIA NPS code of ethics.

Having new judges is a good thing—it brings a fresh perspective to the program. In addition, if a judge does not fulfill the criteria that SIA establishes, they will be asked not to participate in subsequent years.

Judges are divided into teams based on their knowledge of the product categories, with one lead judge to speak for the group and judges review products prior to ISC West. Sometimes, it does happen that a judge has to abstain from voting on a particular product. This happened last year when one of the judges disclosed his commitment to a certain product and the fact that his company installed the device all the time. He abstained from voting or voicing any opinion on this particular product and let us make our own decision based on the merits of the product.

Judges listen to 10-minute presentations to get an overview of the product. Not all judges see all products and presentations, which would be impossible in the time allowed. After the first day of deliberations we gather and talk about what looks promising. The next day we set out on the show floor to view and see demonstrations of those products that were not part of our presentations but may be on the potential preliminary list to receive an award. That way we can balance the one-on-one demonstrations with in-person visits to understand why the other judges may have liked a product, or why we have nixed their selection. We take most of the day on the first day of the show to do this—and it’s worked out quite well. Not only do the judges expand their repertoire, so to speak, but it really levels the playing field for all the entrants.

This year’s judges, who are listed on the SIA NPS website at: http://www.sianps.com/en/ISC-West/2012-NPS-Judges/ along with the criteria for judging, included many more integrators than before and I think that’s fantastic. Our NPS Chair, Jennifer Martin, Manager, Worldwide Channel Programs for Pixim Inc., assists the judges in their goal to be fair and critical. In this role she does not vote or have any influences on judges’ decisions, nor does she participate in the product presentations. Martin is respectful of the process and always goes back to the SIA criteria for the awards during our deliberations, which are listed on the SIA NPS website. She does not interject when the judges all gather to evaluate what they have seen and why they believe a product to be a potential winner. She is the referee, when discussions get heated, and they sometimes do—because the overall gist of the awards program is not only fairness, but how this product really might work in the field and whether it works as stated and has been released to the market in the appropriate time frame. She’s an overseer and champions the SIA NPS cause, but is not part of the voting body of the NPS.

Let me say here that these are not testing awards, nor are they meant to be. These awards are designed to help winning and placing companies market more effectively the attributes of their products. That being said, the process is quite different from actual in-the-field testing by unbiased companies (is there anyone really doing this?) And that’s why the SIA NPS judging gets volunteer integrators from small to large companies, end-users, media and consultants, to evaluate what they might know about how these products behave in the field—how easy or hard they are to install—what the total cost of ownership might be to the end user and how that parlays into an effective solution, or not. In fact, some of the discussions I have been privy to when we gather as a whole to discuss possible winning entries, have centered on products and how they may take days to set up or program, or didn’t meet the customer’s needs or criteria, despite the manufacturer’s claims to having satisfied those same goals.

Winners are excited to be part of the honors and past winners have talked about the definite boost the awards have given to their marketing and branding efforts. For those who wanted to know more, for the first time this year SIA presented a Winner’s Forum at ISC West, and those in attendance learned how judges are motivated, what the process is, what winners have to say about what they gained or hoped to gain from this exposure.

The award process is quite transparent and actually follows the Wikipedia definition of such when it comes to openness, communication and accountability: “Transparency is operating in such a way that it is easy for others to see what actions are performed.” Sure, you can’t see directly into the judging chambers for the awards, but all of the rules and other criteria are open to see. Just check out these links in this story or visit www.sianps.com. It’s all there.

The SIA NPS fills a void that as far as I can see, couldn’t be filled by any other organization in such a professional and well-thought out manner. I’m proud to be part of this “academy awards” of the security industry, and grateful for the Security Industry Association, its staff and all the judges for helping elevate the stature of the security products industry.