How security professionals can adapt their skills for generative AI

Your security expertise still transfers, with some assumptions to revise

Security professionals approaching generative AI face a narrower learning problem than it may first appear. Injection, supply-chain risk, broken access control, privilege escalation, auditing, and forensics still apply. The unnamed author of the short course GenAI Security Foundations, who reports fifteen years working in security, puts the continuity plainly: “The fundamentals we already know, such as injection attacks, supply chain risk, broken access control, privilege escalation, and the disciplines of audit and forensics, all are still relevant. They apply directly to AI systems.”

Those familiar concepts become useful once practitioners know where they fit in a generative AI system. A large language model (LLM), a model that processes prompts and generates language responses, interacts with instructions, retrieved information, users, and tools in ways that change some familiar security assumptions. As the course author says, “What’s different is the genAI architecture, and that’s what we need to understand to be able to secure it.”

That architectural difference defines the upskilling problem. A practitioner with strong conventional security knowledge can build on that expertise by learning enough architecture to place existing controls correctly. The next step is understanding the behaviors specific to generative AI because they show where familiar controls and assumptions need to change.

Start with the context window: the architecture determines the attack surface

The first architectural concept is the context window: the information available to an LLM while it generates a response. In the model described by the course author, the LLM has no persistent memory of its own. Its current context can instead contain earlier prompts and responses, instructions supplied by the system builder, and documents loaded for use during the interaction.

Because the model acts on that context, control over its contents becomes a security concern. When a human or another automated agent submits a prompt, the model reads the available context, generates its response, and stops. The author describes the risk this way: “The context window is the primary attack surface. If an attacker can influence what lands on that whiteboard, the attacker can influence what the model does.” The whiteboard in the quotation represents the model’s current context, so the operational question is who can put information on it.

Once context is treated as an attack surface, the trust level of each input matters. The author’s hierarchy places training embedded in the model at the highest-trust layer and compares that position to firmware in a conventional system. The system prompt, which gives the model instructions above the conversation, comes next, while human or agent input sits at the bottom and requires the same suspicion and sanitization applied to other user-supplied information.

That trust hierarchy matters more when organizations add Retrieval Augmented Generation (RAG). RAG retrieves information from an organization’s document libraries and places relevant material into the model’s working context so it can affect the answer. A security review must consequently ask which documents can enter the context, how they entered the organization’s trusted collection, and whether an attacker can manipulate material that the model later retrieves.

Retrieval expands the inputs the model can consume, while agents expand what the model can do with them. An agent can let a model invoke tools that send email, access the web, query databases, or execute code. Once those capabilities are available, manipulated context can influence a model with permission to change other systems, which means the security boundary includes both the model’s inputs and the authority granted through its tools.

Together, context, retrieval, and tools provide the architectural map needed to reuse established security knowledge. User input can be hostile, retrieved documents can be compromised, and tool permissions can exceed what a task requires. Experienced practitioners already know those kinds of risks, but genAI architecture changes where they appear and where controls have to operate.

Much of the threat model is recognizable once it is mapped onto that architecture

With the architecture established, prompt injection becomes easier to reason about. Prompt injection occurs when attacker-controlled input reaches the model’s context and changes its behavior. The course author compares it conceptually with SQL injection: in both cases, insufficiently controlled input reaches a component whose later behavior the attacker is trying to influence. That comparison gives practitioners a familiar security concept for identifying the input boundary and thinking about sanitization, even though an LLM processes input differently from a database.

The same input problem can arrive through material the user never entered directly. In indirect prompt injection, an attacker places a malicious instruction inside a document or webpage that the model retrieves later, allowing hostile material to enter context through content the application chose to fetch. The author compares this behavior with stored cross-site scripting because the malicious content can be planted first and encounter its execution environment later. Security teams therefore need to inspect the retrieval path alongside direct user input.

Following that retrieval path upstream leads to RAG poisoning. The author characterizes it as a supply-chain attack because the attacker compromises data the system trusts. The model itself can remain intact while an organization’s retrieval collection feeds it poisoned information and produces manipulated answers. Model integrity and information integrity are therefore separate security concerns.

Once model output can trigger actions, least privilege becomes directly relevant. A compromised agent with broad access to tools has, in the author’s comparison, the functional security characteristics of a compromised privileged service account. An agent that only needs to query one database should receive the privileges required for that task, while unrelated systems and higher-risk operations remain outside its authority.

Actions with greater consequences can use another established control: separation of duties. The author gives two arrangements: require both an agent and a human to participate, or divide responsibility between two agents that use different instructions or different models. Either arrangement reduces the ability of one compromised decision path to complete a consequential action by itself. Existing access-control practice thus remains useful when model output is connected to operational authority.

These mappings show why prior security experience has practical value in genAI, but they also expose where translation stops being sufficient. Prompt injection, retrieval poisoning, and excessive agent privileges fit familiar categories of hostile input, supply-chain compromise, and overbroad authority. The model making decisions, however, has properties that change assumptions about investigation, security boundaries, and testing.

Three genAI behaviors mark the boundary where familiar assumptions fail

That limit leads to three model behaviors the course author treats as distinctive: “There are three intrinsic behaviors of generative AI that don’t look like anything we’ve come across before.” They are opaque reasoning, the collapse of the separation between instructions and data, and non-determinism. Each changes an assumption security teams commonly use when investigating a system, establishing a boundary, or validating a control.

The first behavior, opaque reasoning, changes what an investigation can reconstruct. In the author’s account, a model’s decision-making cannot be logged and replayed in the way practitioners expect from conventional software. Logs can still capture surrounding events such as an input or an action taken through a tool, but those records do not reconstruct the model’s internal decision process. Incident responders can observe important evidence around a decision while still being unable to recover why the model made that particular decision.

That reconstruction limit also affects compliance work because audit and forensic disciplines depend on evidence that supports accountability. Existing expertise still tells a team which questions an investigation has to answer, while the model changes which evidence can answer them. Security teams need to separate the events they can capture around the model from the internal reasoning they cannot recreate after the event.

The second behavior changes the boundary between instructions and data. Conventional architectures give practitioners a useful distinction between commands and stored information: in the author’s examples, a database stores information without treating it as instructions to execute, while a processor has architectural rules governing which memory is treated as executable. Those separations let systems apply different controls to commands and content.

Language models weaken that separation because both instructions and information can arrive as language within the prompt and surrounding context. The author says a model cannot reliably distinguish code from data when the two are intermingled there. A retrieved document can consequently contain text intended to provide information to the user alongside language designed to direct the model’s behavior.

That shared language channel explains the special importance of indirect prompt injection. Sanitizing the human’s immediate request leaves other context inputs to examine because a webpage or RAG document can introduce instructions later in the same processing path. Security teams have to reason about every party that can influence context, including sources the application itself chooses to retrieve.

The third behavior, non-determinism, changes what testing can establish because the same attack against the same defense can produce different outcomes on separate runs. Repeatability supports many conventional security-testing judgments: teams often expect one test condition to produce stable evidence about whether a control resists a known technique. When model behavior varies, an observed result gives weaker evidence about what will happen on the next run under an apparently identical condition.

That variability changes the standard of confidence needed for red teaming and assurance. A test that fails to trigger harmful behavior once cannot establish reliable resistance on another attempt, while a successful exploit demonstrates at least one reachable failure without identifying every condition under which it appears. The course author argues that a simple one-time result may provide inadequate assurance.

From the same variability, the author makes the broader claim that the attack surface can never be fully enumerated. The problem goes beyond the number of possible inputs because behavior can change even when an attack and defense appear to remain the same. Security assessment must account for that variability when deciding what a finite set of successful checks can establish about model behavior.

Taken together, these behaviors define where established methods need revision. Opaque reasoning limits reconstruction, mixed instructions and data weaken a familiar security boundary, and non-determinism weakens conclusions drawn from individual test results. Those differences make architecture the starting point for genAI security work and determine which parts of existing practice can carry over unchanged.

Upskilling means learning where to reuse a control, and where to revise the model

Because the boundary is architectural, the learning path starts with architecture and then maps existing security knowledge onto it. Earlier comparisons provide practical anchors: prompt injection connects to injection experience, RAG poisoning to supply-chain thinking, and agent compromise to privilege management. The course author frames the professional consequence directly: “Your existing ways of thinking as a security professional are a valuable foundation when it comes to securing genAI. But staying relevant will mean learning at a pace faster than most of us have had to before.”

That learning pace matters because capabilities and attack techniques are changing while practitioners study them. Drawing on fifteen years in security, the author says, “GenAI capability, and the attack techniques targeting it, are moving faster than any technology shift I’ve seen in fifteen years in this field. The half-life of any technology we know feels like it’s decreasing more quickly than ever.” The statement is the author’s assessment from professional experience, and it explains why accumulated expertise has to be paired with continued learning.

Continued learning, in the author’s prescription, begins with the architecture and uses existing expertise as its structure. “As an existing security professional, the knowledge and skills you have built up to today still matter. You just need to add new ones quickly. My advice is to start with the architecture, map what you already know onto it, and keep learning, ideally faster than you ever have before.” For practitioners, that directs attention toward opaque reasoning, mixed instructions and data, and variable model behavior, where established assumptions require the most revision.

GenAI Security Foundations is the author’s own short course built around this learning problem. Its intended audience is security professionals who already understand security but have little experience with genAI beyond using it, and the author proposes it as an entry point for teams. The author has a direct commercial interest in adoption of the course, so the recommendation should be read as the course creator’s proposed learning route rather than independent validation.

That commercial recommendation also carries the author’s broader view of what security teams need to develop. The author concludes, “At the end of the day, the most successful organizations are not the ones with the most experienced security teams, they’re the ones whose teams learn the fastest. Hopefully my new course will be a good starting point for you and your teams.” On that view, the immediate task for a security practitioner is to keep adding architectural and model-specific knowledge quickly enough to apply established expertise to a changing genAI environment.

Main highlights

• Build on existing security expertise: Injection, supply-chain security, access control, least privilege, auditing, and forensics remain relevant to generative AI. Security leaders can focus upskilling on how these established controls map to AI architecture and workflows.
• Treat context as an attack surface: Prompts, retrieved documents, system instructions, and agent tools create paths for attackers to influence model behavior. Security teams need to govern what enters the context window and limit the authority available through connected tools.
• Map familiar threats onto AI architecture: Prompt injection, RAG poisoning, and overprivileged agents have parallels in established security practice. Teams can use existing expertise in input security, supply-chain integrity, and privilege management to design appropriate controls.
• Revise assumptions for model-specific behavior: Opaque reasoning limits forensic reconstruction, mixed instructions and data weaken traditional security boundaries, and non-determinism reduces the assurance provided by individual tests. Security and assurance programs need testing and evidence practices designed around these properties.
• Make continuous learning part of security capability: GenAI capabilities and attack techniques are evolving quickly, making architectural knowledge an ongoing requirement. Security organizations can prioritize learning that connects emerging model behavior with the controls and threat models practitioners already understand.