(Editor’s note: A third panelist participated in this discussion. At the request of that panelist’s employer, all comments and references associated with that panelist have been removed from this article.)

Between the keynotes, the solution showcases and the standing-room product demonstrations, AI has become the organizing principle of nearly every security industry event. The capabilities are real. The momentum is real. But somewhere between the conference center and the operations center, things get complicated in ways that don’t make it onto the agenda.

Milestone Systems wanted to change that. When they approached me about moderating a panel at Milestone XPerience Days 2026 titled “No Hype Allowed: An Honest Conversation About AI,” the mandate was refreshingly simple: no vendor talking points, just practitioners speaking plainly about what they were experiencing in the real world and what they had learned.

I wanted to hear from people who had moved AI from a proof of concept into daily operation, hit the walls, made the mistakes and come out the other side with something to show for it.

Dan O’Neill, President and CEO of Advanced Data Risk Management, and John O’Connor, Managing Director of Administrative Services and Operations at Harvard Business School, fit that description.

What followed was one of the more grounded, useful conversations I’ve had on this topic.

Where everyone started

I opened by asking the panelists for a personal AI story: a moment where the technology made them stop and think. The answers were revealing, not because they were dramatic, but because they were honest about the arc from novelty to utility.

O’Neill’s entry point was deliberately low-stakes. About a year ago, he was sending emails written in the voice of Don Draper from the television series Mad Men. “It was really fun, got a lot of great responses,” he said. A year later, his company went all-in with AI for their Defender platform — using plain language interfaces, AI agents that open tickets automatically and health alerts driven by the technology. “It’s incredible how far it’s come in just a year,” he said.

O’Connor’s early experience with LLMs a couple of years ago produced something more complicated — a mix of excitement and something he described as a “moment of dread” about where the technology might go. As a people manager, he said, it was immediately clear there was real work ahead.

Two different entry points. Two different reactions. That range of experience is more representative of where the security industry actually sits than any unified narrative about AI readiness.

The data problem nobody talks about enough

When I queried the panel on what real-world deployment taught them that they hadn’t fully understood during the evaluation phase, O’Connor went straight to something that doesn’t get nearly enough attention: data.

“Across our operational technology, these systems are generating millions of rows of records on a daily or annual basis — just sitting in a database, locked away, maybe used for a report or a query at one point,” he said. The recognition that all of that information could now answer meaningful questions about operational efficiency had been genuinely eye-opening. But getting there was harder than expected.

“Finding the data that’s going to give us answers, then working through the specifics of how to get it from where it is today to where we want it to be in the future — that has been a lot more challenging than I initially thought,” O’Connor said. The specific frustration he described was taxonomy: one system calling an asset one thing, another system calling it something else entirely. Cleaning that up at scale, he said, had been a painstaking process.

This matters because organizations evaluating AI often focus on the algorithm or the platform and assume their data is ready. In most cases, it isn’t.

The 2x2 matrix and where to start

O’Neill talked about how his firm helps organizations figure out where to focus. The tool is a two-by-two matrix: high impact versus low impact on one axis, high effort versus low effort on the other. The goal is to identify where AI can move something from high impact/high effort to high impact/low effort.

In his world, that analysis has pointed to four areas where security programs consistently struggle: monitoring video effectively, searching video, managing false alarms on access control and maintaining camera and system health. That last one, he suggested, is often underestimated. “There’s nothing worse than going to get a video on a program you’ve spent tens or hundreds of millions of dollars on, and the camera hasn’t been working. All of that stuff is really fixable with AI.”

In practice, that approach has produced a morning workflow built around AI-generated prioritization rather than human triage. Instead of staff working through a checklist, the system surfaces what needs attention, guides technicians to the issue and in some cases dispatches agents to handle remediation automatically. For security operations teams managing large, complex environments, that shift from reactive to proactive is where the real operational value lives.

He also shared a case study that illustrated scale: a global client asked ADRM to assess their top 10 sites. They built AI-powered reports with summaries. The client used those reports to secure a million dollars in upgrades, implemented them, and then used AI-analyzed data to demonstrate how risk had been reduced across those ten sites — which led to funding for the next 20. “A few days of us putting together AI-powered reports — it created more funding for the security program,” O’Neill said. “Our company of 42 people moved the needle for a hundred billion dollar company. Just with using AI.”

The human factor is the hard part

If the data challenge is underappreciated, the human challenge may be even more so. O’Connor spent significant time on this, and it was one of the more candid stretches of the conversation.

When people first encounter these tools, he said, the underlying anxiety is direct: “Is this going to replace my job? How am I going to feed my family if my job isn’t there tomorrow?” His organization’s response was to invest in broad, faculty-led training — two full days for everyone, covering a range of AI tools. The explicit goal was to democratize the creativity: get the tools into the hands of people at every level of the organization and let ideas surface from the bottom up.

It worked, at least directionally. “We’ve gone from folks who are deniers — ‘I will never utilize these tools’ — to actually now surfacing some pretty neat ideas and new ways of making us more efficient to better serve our customers,” he said.

The companion idea he returned to was what he called “fail forward.” His argument was straightforward: no one has all the answers right now, and no one can accurately predict what AI will look like 12 or 24 months from now. The only path is incremental progress, which means accepting that failures are part of it. “Through a culture of creating a space where people can feel safe to fail, we’re going to try this, it might not work, we’re going to learn something, but we’re moving forward.”

Community pushback is real

O’Neill raised something I thought was important and is often left out of these discussions: the gap between what security professionals think is an obvious win and how the broader community receives it.

“One of the things we underestimated was potential pushback from the community,” he said. “It all sounds great for security professionals — you want to use technology to make the environment safer and more secure. But some of the customers, their customers — in this case students and faculty — really saw it as: ‘We’re becoming a surveillance state.’”

The delay was significant. Legal experts had to be brought in. Communication had to go back out to the affected population before implementation could move forward.

He also referenced an active shooter incident that occurred in a neighboring municipality where nearly 60 rounds were fired and three people were injured. The gunshot detection system did not play a role in the response. However, following the incident, the city council ended the community’s use of the technology after years of public scrutiny and debate. O’Neill said the episode underscored how privacy and surveillance concerns can walk back technologies that may appear to have strong security value.

That’s a real dynamic, and it doesn’t resolve itself just because the security case is strong.

Don’t lead with AI

O’Connor made a point near the end of the session that I think is the right frame for all of this: the smartest organizations are probably not leading with “AI” as the destination.

“We sit around the table in our operations suite and ask: what are the actual problems that we’re trying to solve?” he said. AI tools may be part of the answer to those problems, but they’re not the starting point. “Being crystal clear on the problems we’re trying to solve and the outcomes we’re trying to achieve — for the organization, for our guests, for the community — is key. Then bring in the right tools, resources, people and technology to actually solve those problems.”

That’s not a complicated idea, but it’s one that gets reversed a lot. Organizations get excited about a technology and work backward to find a use case. The ones that seem to be getting the most out of AI are doing it in the other direction.

What to do Monday morning

O’Neill and O’Connor closed with a concrete suggestion.

O’Neill’s: run the two-by-two matrix exercise with your organization. Identify what’s high impact and high effort, and then develop a plan to move those things toward high impact, low effort using AI. “If you can do that, it will impact your organization with AI.” And if that feels too heavy, he added — only half-joking — try writing an email in the voice of Don Draper. “That’s also really fun.”

O’Connor’s was directed at everyone in the room, regardless of their role: “There are software developers here, integrators, consultants, end users and leaders. We all have a responsibility to think about how the tools that we’re developing, creating, building or implementing are going to actually affect the business and affect our operations. Don’t lose sight of the person at 3 o’clock in the morning who’s in your security operations center that’s going to interact with these tools.”

That last image has stayed with me. All of the strategy, the matrix exercises, the data readiness work, the vendor relationships: it all eventually lands on a person, in a room, at an ungodly hour, trying to do their job better than they could before. If the technology helps them, it worked. If it doesn’t, no amount of impressive demos changes that.

That’s what “no hype allowed” actually means.