Shadow AI Risks Security Teams Can’t Afford to Ignore
Key Highlights
- Shadow AI introduces new risks through unapproved AI tools, public models and third-party integrations.
- Malicious model files, prompt injection and MCP servers are expanding the software supply chain attack surface.
- Security teams can reduce exposure by monitoring AI use and applying stronger governance to models, OAuth grants and integrations.
Security leaders spent years fighting what’s commonly known as “shadow IT”: software and cloud services employees adopted without the approval, or even the knowledge, of the IT department. That meant tightening policies and pushing back against rogue Dropbox accounts and unsanctioned Slack and Zoom installs, which exploded during the pandemic-era shift to remote work. Then came generative AI, and with it, a new challenge: shadow AI.
Shadow AI is often treated as shadow IT with a new coat of paint, but the two are distinct. Shadow IT is about unsanctioned tools: an employee using a personal Dropbox account instead of the company’s approved file-sharing platform. Shadow AI is about unsanctioned AI applications and features specifically: sign-ups for AI SaaS tools, browser extensions, and AI capabilities embedded into existing software and using “public” models all adopted without IT or security review. The distinction matters because the risk is different in kind, not just degree. An unsanctioned Dropbox account exposes files. An unsanctioned AI tool exposes whatever an employee types into it: proprietary code, customer records, financial figures, internal strategy, with no visibility into how that input is stored, retained, or used downstream.
Stack Overflow’s 2025 Developer Survey found that 84% of developers are using or planning to use AI tools in their development process, up from 76% the year before. Increased adoption causes widening gap between how much AI gets used and how much oversight it gets is exactly where shadow AI risk accumulates. Application security tooling hasn’t kept pace with either trend, and together, rapid adoption and thin governance have created an attack surface most enterprises aren’t equipped to defend.
The scale of the problem
Let’s dissect one part of the shadow AI, “public” models. To understand the threat, start with the source: where are developers getting their models? One popular option is Hugging Face, which today hosts more than 2 million models and has become a go-to resource for AI and ML development teams.
Hugging Face’s popularity has drawn attention from attackers, and the platform has seen a number of malicious campaigns. In one, dubbed “nullifAI,” two Hugging Face PyTorch model files exploited a compression quirk (using 7z instead of the standard ZIP format) that caused Hugging Face’s Picklescan security tool to fail to flag them as unsafe. The payload was a platform-aware reverse shell that connected back to a hardcoded IP address.
In a related campaign months later, RL researchers found malicious PyPI packages using the same Pickle-abuse technique, this time delivering fully functional infostealer code while posing as a Python SDK for a legitimate AI platform. What makes both attacks effective is that ML files haven’t traditionally been treated as a medium for distributing executable code, so they don’t get the same scrutiny as other software dependencies.
That gap points to something worth naming directly: pulling a model from a public hub without review and security scan is itself a form of shadow AI. A data scientist downloading a model from Hugging Face is functionally doing the same thing as a developer pulling an unreviewed package from npm or PyPI, and it deserves the same default posture: treat it as untrusted until proven otherwise.
Hugging Face does mark an organization’s page as “Verified” once it confirms ownership of a matching company domain, which is a reasonable first filter. NVIDIA and Meta (which publishes under the “facebook” and “meta-llama” organization accounts) both carry that badge, and other major labs, including Google, Microsoft, Mistral AI, and IBM, maintain similarly recognized accounts. Outside a short list like that, the safer default is to treat a model as at least suspicious, and route it through the same review as any other third-party dependency, until it’s cleared.
Model manipulation: The prompt injection problem
Malicious packages like those in the nullifAI campaign are only part of the picture. Attackers are also targeting AI models directly through model manipulation, a technique that is becoming increasingly common. Here, the goal is to corrupt a model’s behavior so it produces inaccurate, harmful, or attacker-favorable outputs.
The most prevalent technique for doing this is prompt injection, which OWASP’s GenAI Security Project ranks first on its 2025 Large Language Model Top 10 list. That’s because a prompt injection can cause a model to violate its own guidelines, generate harmful content, enable unauthorized access, or skew decisions in ways that serve the attacker rather than the user.
For example, a malicious comment embedded in a code repository could instruct an AI coding assistant reading it to suggest insecure code patterns or introduce subtle vulnerabilities into its completions.
Another variant is multimodal injection, where the attacker hides a malicious prompt inside an image rather than in text. The model processes both simultaneously and follows the hidden instruction, while security tools built to scan text inputs miss it entirely.
MCP servers: The newest entry point
Model Context Protocol (MCP) servers have emerged as critical infrastructure for AI-powered development workflows, and with that, another source of AI risk.
MCP is an open standard that connects AI applications to external systems, and adoption among software developers has been rapid. GitHub MCP servers, for example, have become a standard tool for code management because they’re convenient and powerful. Industry estimates put monthly MCP SDK downloads at roughly 97 million as of early 2026, up sharply from a small fraction of that at the protocol’s launch in late 2024.
As with Hugging Face, that popularity means MCP servers are squarely in the sights of malicious actors. In September 2025, researchers at Koi Security identified the first known instance of a malicious MCP server distributed via npm: a package called postmark-mcp that quietly BCC’d every outgoing email to an attacker-controlled address.
It was a milestone worth taking seriously, because where there’s one, there will be more. Like other AI tooling, MCP adoption is outpacing the development of security standards and detection capabilities for the protocol. Development teams are adding MCP integrations without the governance frameworks that would normally accompany new infrastructure, and that gap between adoption and controls is exactly the kind of opening attackers look for.
Finding shadow AI before it finds you
Most shadow AI doesn’t announce itself. Employees don’t file a ticket before signing up for an AI writing assistant or connecting a note-taking bot to their calendar; they just start using it. Detection has to work without cooperation, by watching for the signals that AI adoption leaves behind. For example:
- Strong policies regulating model use. Every company needs policies regulating model use that need to be widely known and enforced. Each model and training data needs to be scanned for malicious intent.
- Network and SaaS-usage discovery. Proxy logs, DNS queries, and firewall traffic to known AI service domains reveal usage patterns long before procurement does. Cloud access security broker (CASB) tools built for general SaaS discovery apply this same approach, and the same telemetry catches AI-specific tools.
- Identity provider and OAuth app inventories. Most AI SaaS sign-ups happen through “Sign in with Google” or “Sign in with Microsoft,” which means each one creates an OAuth grant in the identity provider’s admin console. Reviewing that grant list on a regular cadence surfaces AI tools that were never reviewed, without needing agent software on every endpoint.
- Browser-level monitoring. Browser extensions and enterprise browser tooling can flag AI site visits and file uploads to AI tools directly at the point of use, catching adoption that never touches a corporate network or SSO flow.
- Procurement and expense audits. AI subscriptions are frequently paid on personal or departmental credit cards specifically to avoid procurement review. A recurring audit of expense categories for AI tooling charges is a low-tech but effective complement to the technical controls above.
None of these alone is sufficient. Taken together, they cover the paths that shadow AI adoption tends to take: network, identity, endpoint, and spending.
Uncovering third-party leaks through existing platforms
A second, related blind spot sits inside the SaaS platforms organizations have already sanctioned. Most enterprise platforms, CRMs especially, expose a connected-app or OAuth-integration model that lets third-party tools request API access without a full procurement cycle. Salesforce is a useful example because its model is well documented: any connected app, whether it’s an official Salesforce integration, an AppExchange listing, or something a user authorized on their own, shows up in Setup under Connected Apps OAuth Usage, along with the scope of data access it was granted.
Setup Audit Trail logs when a new connected app was installed and by whom.Organizations with Event Monitoring enabled can go further and see the actual volume and pattern of API calls each connected app is making. (Salesforce is migrating this model to External Client Apps, which apply stricter security defaults by default, but the audit principle carries over.)
The practical takeaway: an AI meeting notetaker, sales-enrichment tool, or CRM copilot that an employee connected through an OAuth prompt is functionally a third-party integration with standing access to customer data, whether or not it went through security review. Treat the connected-app or OAuth-grant list in any core platform, Salesforce or otherwise, as an audit surface, not a settings page.
A recurring review, flagging unfamiliar apps, checking granted scopes against what the tool actually needs, and disabling anything that can’t be attributed to an approved use case, closes a gap that’s otherwise invisible from the network layer alone. For a stronger default, Salesforce’s API Access Control setting can block new connected apps from gaining access until an admin explicitly allows them, shifting the model from opt-out to opt-in.
Governance can’t be an afterthought
To date, attacks like these are a small slice of the overall threat landscape. But as development teams and enterprises deploy AI models deeper into their operational workflows, the consequences of a successfully manipulated model, or an unreviewed integration with standing data access, grow significantly. That’s why shadow AI and the threats lurking on AI infrastructure demand sustained attention from development organizations and their customers alike.
Practically, that means applying the same rigor to AI infrastructure that’s applied to other parts of the software supply chain:
- Treat AI models as software artifacts, subject to the same supply chain scrutiny as any other dependency, and default to suspicion for anything pulled from a public hub outside a short list of verified publishers.
- Build detection capabilities that account for ML file formats as a potential malware delivery mechanism, not just a data format.
- Establish governance frameworks for MCP integrations before they become a default part of the development stack.
- Audit connected-app and OAuth grants inside core SaaS platforms on a recurring basis, not just at initial rollout.
- Take shadow AI seriously as its own threat category: related to shadow IT, but with a materially different risk profile that demands updated thinking.
To be clear, this isn’t an argument against AI adoption. The productivity and capability gains of large language model AI are real. Security teams that try to stifle AI adoption with controls that bottleneck it, rather than help enable it, are likely to be sidestepped or ignored outright. The better approach is intentional adoption, where security controls keep pace with deployment rather than chasing it.
Organizations that recognize this now, and invest accordingly, will be far better positioned than those still reaching for yesterday’s playbook when the next supply chain incident hits.
The AI wave isn’t coming. It’s already here. The question is whether security architecture is built to move with it.
About the Author
Igor Lasic
Igor Lasic, Senior Vice President, Technology, ReversingLabs
Igor Lasic is senior vice president of technology at ReversingLabs, where he leads technology initiatives. He brings more than 20 years of experience with early-stage companies delivering SaaS and on-premises products, as well as experience across the telecommunications, enterprise and medical industries. He previously served as vice president of engineering at IHM Services, Active Broadband Networks, Cartiza Networks and BigBand Networks.