Securing the Human Layer: Building AI Security Awareness and Technical Defense

Key Insights

• Shadow AI is now a top insider-threat vector. The 2026 Verizon Data Breach Investigations Report found that the share of employees using personal AI accounts on corporate devices tripled in a single year, reaching 45% of the measured workforce. Most of those users access AI through consumer-tier accounts. Shadow-AI-related breaches add roughly $670,000 to total incident costs. [2]
• The primary IP-loss vector is convenience, not malice. Most AI-related data leaks occur when well-intentioned employees paste sensitive content into chat windows or upload proprietary documents for summarization. These transfers are invisible, difficult to audit, and unlikely to appear in any traditional data-loss-prevention console. [3]
• Security awareness programs require fundamental redesign, not incremental updates. The classic compliance curriculum (phishing recognition, password hygiene, and clean-desk policy) addresses outside-in threats. AI IP risk is largely inside-out: trusted employees voluntarily sharing sensitive data with services they trust. Training must shift from rule lists to risk-reasoning frameworks. [3]
• AI literacy is now a regulatory obligation. Under Article 4 of the EU AI Act, providers and deployers must ensure a sufficient level of AI literacy among staff and contractors. National authorities gain formal supervisory powers to enforce this requirement starting August 2, 2026. Awareness is no longer merely good practice; it is a documented compliance requirement. [7]
• Prompt injection and tool misuse have moved from theory to practice. Prompt injection is ranked the top risk in the OWASP guidance for LLM and agentic applications, and real-world exploits, such as zero-click image attacks and RAG-indexed instructions, have demonstrated data exfiltration through AI agents. [3, 6, 8]
• Agentic AI is outpacing the controls designed to secure it. IBM reports that while 79% of enterprises are deploying AI agents, 97% lack adequate security controls for them. The Cloud Security Alliance estimates that 40% of enterprise applications will embed AI agents by the end of 2026. The OWASP Top 10 for Agentic Applications (2026) was created to close this gap. [8, 9, 11]

This article focuses on the framework’s two operational disciplines: security awareness, redesigned for the AI era, and a layered technical defense. Part 1 covers the foundational disciplines, regulatory readiness, and governance architecture.

Introduction

Technical controls are necessary but not sufficient. Most AI-related IP-loss events will not trigger a DLP alert or be caught by an AI gateway; they will occur because an employee made a reasonable-seeming decision to use an AI tool in a way that exposed sensitive data. Every prompt submitted to a public model is a potential transmission, and the same behaviors that make AI useful, its ability to synthesize, contextualize, and improve, also make it a powerful conduit for intellectual property loss. [3]https://greenleafgrp.com/insights/ai-readiness-isnt-what-you-think-the-gaps-that-dont-show-up-in-assessments/

Closing that gap requires two complementary disciplines. A security-awareness program designed for AI-era risk builds the human judgment that no control can supply, while a layered technical defense gives the organization visibility into AI use and the means to contain it. The sections that follow build both capabilities, starting with the human layer, where most losses begin.”

Step 3: Security Awareness for the AI Era

Closing the human gap requires a security-awareness program tailored to AI-era risks, not the classic curriculum with an AI module bolted on.

Why Existing Programs Fall Short

The traditional security-awareness curriculum, including phishing recognition, password hygiene, physical security, and clean-desk policy, remains necessary but is no longer sufficient. It addresses threats that come from the outside in. AI IP risk is largely inside-out: well-intentioned employees voluntarily sharing sensitive data with services they trust, in good faith and in pursuit of legitimate productivity. [3]

Traditional awareness is also binary; a behavior is either safe or unsafe. AI use is contextual. The same prompt may be harmless or catastrophic, depending on the sensitivity of the data, the account tier, and the provider’s data-handling terms. Employees need judgment frameworks, not just rule lists. A program that cannot equip people to reason about risk in real time will fail to address the combinatorial variety of AI use cases that arise in practice.

Five Shifts Required in Security-Awareness Design

• From annual training to continuous micro-learning. The AI threat landscape evolves faster than annual compliance cycles. Replace or supplement annual modules with short, frequent interventions, in-app guidance in approved tools, and regular briefings on emerging threat patterns. The goal is AI security intuition, not checkbox behavior.
• From rules to risk reasoning. Employees cannot be handed a list of every prohibited action; the combinations are infinite. Teach a risk-reasoning framework: How sensitive is this information? Who controls the system I am sharing it with? What are the data-handling terms? What is the worst case if it is exposed or memorized? Regulated industries can anchor the framework in specific obligations.
• From generic content to role-specific scenarios. A software engineer’s AI risk profile differs fundamentally from that of a corporate attorney or a financial analyst. Develop role-based curricula with scenario simulations tailored to legal, engineering, finance, HR, and executive functions, and measure retention by role.
• From punitive culture to psychological safety. If employees fear discipline for admitting they used an unsanctioned tool, they will not report incidents. Near-miss reporting is among the most valuable inputs for an effective program, and it requires a culture in which disclosing mistakes feels safe.
• From employee-only training to the extended enterprise. Contractors, consultants, and vendors with access to sensitive data often fall outside the awareness program. Vendor contracts should include AI-security requirements, and onboarding should set explicit expectations for AI data handling.

AI Literacy as a Compliance Requirement

Security awareness has taken on a regulatory dimension. Article 4 of the EU AI Act, in force since February 2, 2025, requires providers and deployers to ensure a sufficient level of AI literacy among staff and explicitly extends this requirement to contractors and others operating systems on their behalf. National market-surveillance authorities gain formal powers to enforce it from August 2, 2026. Literacy is not a fixed standard: it must be proportionate to the context of use, the system’s technical complexity, and the role of the person concerned. [7]

Regulators have signaled that weak AI training will be treated as an aggravating factor in broader enforcement and that human-oversight obligations for high-risk systems presuppose trained operators. The practical implication is that organizations should document what training has been delivered, to whom, and which risks it covers, turning awareness from an informal effort into documented evidence of compliance. A program built around role-specific risk reasoning, as described above, satisfies both security and regulatory objectives simultaneously. [7]

Security-Awareness Maturity Model

Organizations can use a four-level model to assess and prioritize investment. At the Basic Level, an annual AI policy module is delivered, and an acceptable-use policy is published. At the Developing Level, role-based modules are deployed, the AI vendor registry is actively communicated, and DLP alerts are linked to real-time learning moments rather than to disciplinary action. At the Advanced Level, continuous micro-learning is in place, behavioral analytics track how knowledge is applied, a structured near-miss reporting program is in place, and third-party AI requirements are embedded in vendor contracts. At the Leading Level, real-time coaching is delivered through approved tools, AI phishing simulations are regularly red-teamed, and board-level metrics include awareness outcomes alongside technical control status.

Most organizations currently operate at the Basic or Developing level. Moving to Advanced requires deliberate design, integrating awareness triggers into existing security operations workflows, HR onboarding, and legal review cycles, rather than relying on a standalone training budget.

Step 4: A Layered Technical Defense

Securing AI is not a matter of a single control. It requires a comprehensive strategy spanning visibility, data governance, vendor management, architecture, and adversarial testing. Zero Trust applies directly: you cannot apply its principles to what you cannot see, so continuous discovery of AI usage is the foundation for every other control.

Visibility and Shadow-AI Discovery

AI adoption arrives through multiple channels simultaneously: employee chatbots and browser extensions, SaaS platforms that enable AI features with minimal change control, open-source models in developer tooling, and copilots embedded in Microsoft 365, Salesforce, and Zoom. [2]

Organizations should deploy the discovery of AI services accessed from the corporate network and on managed devices, risk-tier classification of those services, and logging sufficient to reconstruct which data was submitted to which service. This cannot be achieved by policy alone; it requires technical integration across identity management, browser controls, and network monitoring, including inspection of DNS and proxy logs, monitoring egress to known AI endpoints, and use of Cloud Access Security Broker tooling to flag SaaS apps with embedded AI. Unapproved tools should be blocked or flagged at the network and endpoint layers, not merely prohibited in an intranet document. [3]

Data Classification

Effective IP protection begins with knowing what needs protection. AI governance requires extending classification in three ways.

• First, define AI-sensitive categories explicitly. Trade-secret algorithms, unreleased roadmaps, M&A intelligence, proprietary pricing, and regulated personal data should each carry labels that trigger AI-handling rules — making clear to employees and systems alike what cannot be included in an AI prompt.
• Second, make the classification machine-readable and enforceable. Labels must integrate with DLP and the identity-and-access layer so that policies are applied automatically rather than left to individual judgment at the point of use.
• Third, keep the scheme frictionless. Employees must be able to classify data with minimal effort. A scheme too complex to use in practice provides only the illusion of governance.

AI-Aware Data Loss Prevention and Gateways

Traditional DLP was designed to intercept email attachments, USB transfers, and web uploads. It is poorly suited to the LLM use case, in which sensitive data may be submitted as conversational text in small increments across multiple sessions. Security teams should evaluate AI-aware DLP solutions that provide prompt-level inspection, integration with browser extensions and API gateways, and real-time enforcement based on classification labels. [3]

Organizations should also consider a corporate AI gateway, a managed proxy between employees and external providers, that can log prompts, enforce redaction, block submissions containing sensitive patterns, and route request types to approved services. Databricks’ AI Gateway demonstrates this capability at the data-platform layer, and comparable products exist for general-purpose LLM traffic.

AI Model Observability

Observability, knowing what AI systems are doing, with what data, and with what outcomes, is a foundational requirement that is frequently underemphasized. At the infrastructure level, it means capturing model inputs, outputs, and intermediate reasoning in audit logs accessible to security operations. At the application level, it means monitoring for anomalies: unusual prompt volumes, outputs referencing content not present in the session, or agent actions outside expected parameters. Gateways that log prompts and responses, model-serving infrastructure with audit trails, and agent frameworks that expose action logs should be procurement criteria evaluated before deployment, not retrofitted afterward.

Internal Models for High-Sensitivity Workflows

For the most sensitive IP, the appropriate architecture is an internally deployed model running on infrastructure the organization controls, with no data leaving the network perimeter. The rapid maturation of capable open-weight models, including sub-10-billion-parameter models that run efficiently on enterprise hardware, has made this practical for organizations that would have found it cost-prohibitive just two years ago. Internal models introduce their own considerations: model weights must be protected against exfiltration, inference infrastructure must be hardened, and fine-tuning pipelines must be treated as high-value targets because a fine-tuned model inherits the sensitivity classification of its training corpus. [4]

Governing Agentic and Automated AI Pipelines

As organizations move beyond chat interfaces toward agentic systems that browse the web, write and execute code, access databases, and send communications, the governance challenge expands considerably. An agent with broad permissions can aggregate and transmit sensitive information across multiple steps, actions that a single step would not flag as suspicious. With 40% of enterprise applications expected to embed agents by the end of 2026, this is a near-term concern. [9, 17]

The OWASP Top 10 for Agentic Applications, released in December 2025 and reviewed by experts from bodies including NIST and the European Commission, provides the current reference for these risks, spanning agent goal manipulation, tool misuse, excessive agency, and memory poisoning. It complements the threats-and-mitigations work of the OWASP Agentic Security Initiative and Singapore’s Model AI Governance Framework for Agentic AI, published in January 2026, which organizes requirements across risk assessment, human accountability, technical controls, and end-user responsibility. [6, 9]

Practical guidance has converged on a small set of controls. Recent industry research, including Meta’s “Agents Rule of Two” and Simon Willison’s “lethal trifecta,” holds that an agent becomes dangerous when it simultaneously has access to sensitive data, exposure to untrusted content, and the ability to communicate externally; mitigation means permitting no more than two of the three. The Databricks AI Security Framework maps this logic to enforceable platform controls. Governance frameworks for agentic AI should therefore require least-privilege access scoped to the minimum necessary data and systems, human-in-the-loop checkpoints for high-consequence actions, comprehensive action-level audit logging, and the ability to kill-switch or quarantine a misbehaving agent, with these measures embedded in development and procurement standards before agentic systems reach production. [14]

Prompt-Injection and Tool-Misuse Defenses

Prompt injection embeds malicious instructions in content that an AI tool will process. When an employee asks an assistant to summarize a web page, document, or image, hidden instructions can redirect the model to exfiltrate session context, including data shared earlier in the conversation. The attack surface spans web pages, images (including zero-click payloads that execute on upload), documents with concealed text, and the conversation itself. As agents gain outbound capabilities, prompt injection escalates from a data-exposure risk to an active exfiltration channel. CrowdStrike and others have documented indirect prompt injection, in which instructions hidden in retrieved content are indexed by enterprise RAG systems and executed when any employee runs a routine query. [3, 15]

No single technique fully prevents prompt injection, so defense must be layered: input validation and content sanitization, retrieval allowlists for trusted sources, output filtering and redaction, strict privilege minimization for tools, and real-time behavioral monitoring. Architectural approaches show promise. Evaluations of the CaMeL design on the AgentDojo benchmark mitigated 67% to 100% of injections while preserving task utility by separating trusted control instructions from untrusted data. A strong guardrail program translates policy into explicit, testable rules that specify which data sources are permitted, what classes of data may appear in outputs, and which tools may run under what conditions. [13]

Red Teaming and Adversarial Testing

Red teaming AI systems by simulating adversarial attacks to uncover vulnerabilities before real attackers do is becoming a standard for enterprises deploying AI in production. IBM reports that while 79% of enterprises now deploy AI agents, 97% lack adequate security controls for them; the gap between deployment velocity and security maturity is the most urgent near-term risk for many organizations. [11]

Microsoft distinguishes two overlapping objectives: safety red teaming (testing for harmful content and policy violations) and security red teaming (testing for data exfiltration, system compromise, and unauthorized tool use). NIST’s March 2025 update to AI 100-2 extended its adversarial-ML taxonomy to cover autonomous-agent vulnerabilities for the first time, including indirect prompt injection, agent memory poisoning, and supply-chain attacks, and established a three-tier evaluation hierarchy of model testing, adversarial red teaming, and field testing that organizations can adopt as a template. Red teaming should be built into the development lifecycle for AI-enabled products and conducted periodically for production systems, at a cadence that reflects each system’s risk classification. [10, 12]

Conclusion

An enterprise’s intellectual property, including its algorithms, strategy, client relationships, and proprietary processes, is now at risk through a channel that did not exist at meaningful scale just five years ago. The tools that make employees more productive are the same tools through which sensitive data can silently leave the organization. Managing that risk does not require prohibiting AI; it requires governing it through people and technology working together.

Unlike governance frameworks, which can be designed in advance, security awareness and technical defenses must remain dynamic, updated as attack techniques evolve, refined as new AI capabilities emerge, and stress-tested through red teaming and near-miss reporting. Organizations that treat this as a continuous capability rather than a completed project will capture AI’s productivity benefits without surrendering the IP that defines their advantage.

References 

1. reco.ai, 2025 State of Shadow AI Report. 2025. https://go.reco.ai/hubfs/2025%20Reco%20Shadow%20AI%20Report.pdf  
2. Verizon / Kiteworks, Shadow AI Now a Top Insider Threat ,2026 Data Breach Investigations Report analysis. https://www.kiteworks.com/cybersecurity-risk-management/shadow-ai-data-leakage-governance/  
3. Ferrara, E., Protecting the Crown Jewels: AI Use Presents Real Risks. Green Leaf Consulting Group, 2026. 
4. Carlini, N., et al., Extracting Training Data from Large Language Models. USENIX Security, 2021. https://www.usenix.org/system/files/sec21-carlini-extracting.pdf  
5. He, Y., et al., Security of AI Agents. arXiv, Cornell University, 2024. https://doi.org/10.48550/arXiv.2406.08689  
6. Park, S., Unveiling AI Agent Vulnerabilities: Data Exfiltration. Trend Micro, 2026. https://documents.trendmicro.com/assets/white_papers/ExecBrief%20-%20LLM%20Service%20Vulnerabilities%20p3.pdf  
7. OWASP GenAI Security Project, Top 10 for Agentic Applications (2026) and Agentic Security Initiative. December 2025. https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/  
8. European Commission, AI Literacy under Article 4 of the EU AI Act , Q&A guidance; see also Travers Smith, The EU AI Act’s AI Literacy Requirement. 2026. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai  
9. Cloud Security Alliance / Singapore IMDA, Model AI Governance Framework for Agentic AI. January 2026. 
10. NIST, AI 100-2 Adversarial Machine Learning Taxonomy and Terminology, March 2025 update. https://csrc.nist.gov/pubs/ai/100/2/e2025/final  
11. IBM, Securing the Agentic Enterprise, AI Agent Deployment and Security Controls. 2026. 
12. Microsoft, Lessons from Red Teaming 100+ Generative AI Products. Microsoft AI Red Team, 2025. 
13. Red Hat, AI Security: Defending Against Prompt Injection and Unsafe Actions (CaMeL / AgentDojo evaluation). 2026. https://www.redhat.com/en/blog/ai-security-defending-against-prompt-injection-and-unsafe-actions  
14. Databricks, Mitigating the Risk of Prompt Injection for AI Agents (Databricks AI Security Framework; Agents Rule of Two). 2026. https://www.databricks.com/blog/mitigating-risk-prompt-injection-ai-agents-databricks  
15. CrowdStrike, Indirect Prompt Injection Attacks: Hidden AI Risks. 2025. https://www.crowdstrike.com/en-us/blog/indirect-prompt-injection-attacks-hidden-ai-risks/  
16. Second Talent, Top 50 Shadow AI Statistics 2026. https://www.secondtalent.com/resources/shadow-ai-statistics  
17. Collier, Beyond the Prompt: Architecting Secure, Scalable AI Orchestration Workflows in Green Leaf Insights, M. Miner, Editor. 2026, Green Leaf Consulting Group: https://greenleafgrp.com/insights/beyond-the-prompt-architecting-secure-scalable-ai-orchestration-workflows/.