AI Security Roadmap — From AppSec to AI Security

Week 1

LLM Foundations I — Tokens, Prompts, Determinism

Build an accurate mental model of what an LLM is and how inputs shape outputs. Without this, everything downstream becomes cargo-culting.

Resources

Book Hands-On Large Language Models — Chapters 1-2 $ Alammar & Grootendorst (O'Reilly, 2024) The clearest chapter-1 framing in print. Skip appendices.
Video Intro to Large Language Models (1h) Andrej Karpathy Non-negotiable. Watch once at 1x, take notes.
Docs Prompt engineering overview Anthropic Current, opinionated, vendor-neutral framing under vendor docs.
Blog Things we learned about LLMs in 2024 Simon Willison State-of-the-field annual summary. Orients everything else.

Exercise

Install Ollama. Pull a small model (llama3.2:3b or similar). Send the same question 5 times at temperature 0 and 5 times at temperature 1. Save outputs.

Checkpoint

In 2 sentences: what is a token, and why does rewording a prompt change the output?

Week 2

LLM Foundations II — Attention, Context, Sampling

Understand the internals well enough to reason about attack surfaces. You do not need to implement a transformer — you need to know why injection works at the attention level.

Resources

Book Hands-On Large Language Models — Chapter 3 $ Alammar & Grootendorst (O'Reilly, 2024) Attention and transformer mechanics without the math overhead.
Video Attention in transformers, visually explained 3Blue1Brown Best visual intuition for attention. Pair with Karpathy from Week 1.
Blog The Illustrated Transformer Jay Alammar Canonical reference. Free version of Hands-On LLMs Ch. 3 intuitions.
Docs Sampling parameters — temperature, top_p, top_k OpenAI Platform docs Skim the sampling section. Same concepts apply across vendors.

Exercise

With Ollama, run the same prompt 10 times at temperature 0 and 10 times at temperature 1. Count how many outputs are byte-identical in each group. Explain the result.

Checkpoint

Explain attention to a backend engineer who has never seen it, in 3 sentences.

Week 3

OWASP LLM Top 10 — The Attack Surface

Map the attack surface as industry has agreed on it. This is the taxonomy every other AI Security document in 2026 builds on.

Resources

Docs OWASP Top 10 for Large Language Model Applications 2025 OWASP GenAI Security Project Read end-to-end. Every LLM01-LLM10 page matters.
Book The Developer's Playbook for LLM Security — Chapters 1-2 $ Steve Wilson (O'Reilly, 2024) Wilson co-leads OWASP LLM Top 10. Book = extended commentary on the spec.
Video OWASP LLM Top 10 v1.1 launch talk Steve Wilson — OWASP Global AppSec Author explains the rationale behind each risk class. 45 min.
Docs MITRE ATLAS — AI threat matrix MITRE Browse all tactics once. ATLAS is to AI what ATT&CK is to classic security.

Exercise

Pick a public LLM application (any chatbot, Copilot, search assistant). Write a 1-page threat model naming which of LLM01-LLM10 apply and why. No solutions yet — just enumerate.

Checkpoint

Recite LLM01-LLM10 from memory with a 1-line description each.

Week 4

Prompt Injection — Direct and Indirect

Understand LLM01 at research-paper depth. It is the most exploited and least defended class in production.

Resources

Paper Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Greshake, Abdelnabi, Mishra et al. (arXiv 2023) The paper that named indirect injection. Required reading.
Blog Prompt injection archive Simon Willison The single best running log of real-world injections. Sort by date.
Book Adversarial AI Attacks, Mitigations, and Defense Strategies — prompt injection chapters $ John Sotiropoulos (Packt, 2024) Most practical attacker-lens chapters available in book form.
Video Prompt Injection: What's the worst that can happen? Simon Willison — AI Engineer Summit 30 min. The plain-English version of the paper.

Exercise

Against a free-tier consumer chatbot, attempt 5 direct injection variants (role hijack, delimiter break, instruction override, Unicode smuggling, tool-call manipulation). Document what worked and what was blocked. Do nothing harmful.

Checkpoint

Explain direct vs indirect prompt injection with one concrete example of each.

Week 5

Sensitive Information Disclosure — LLM02

Understand where PII and secrets leak in an LLM pipeline and why generic DLP tools miss most of it.

Resources

Book The Developer's Playbook for LLM Security — Chapter on data leaks $ Steve Wilson (O'Reilly, 2024) Wilson's chapter on LLM02 is the cleanest short treatment available.
Docs LLM02: Sensitive Information Disclosure OWASP GenAI Security Project Official spec page. Short. Read.
Docs Microsoft Presidio — PII detection engine Microsoft Most widely used open-source PII detector. Read the recognizers list.
Blog Preventing data leakage with LLM pipelines AWS Machine Learning Blog Reference architecture mapped to OWASP LLM Top 10.

Exercise

Write a TypeScript (or Python) function that detects one government ID from your country using regex + checksum. Test it against 10 synthetic inputs (5 valid, 5 invalid). Do not use real data.

Checkpoint

List the 3 points in an LLM pipeline where PII can leak, in order of occurrence.

Week 6

Supply Chain and Model Risks — LLM03, LLM05

Understand the non-input attack surface. Most AppSec engineers underweight this because classic web apps don't have it.

Resources

Book The Developer's Playbook for LLM Security — Chapters 4-5 $ Steve Wilson (O'Reilly, 2024) Supply chain + insecure output handling, from the spec co-author.
Docs MITRE ATLAS — Supply Chain Compromise tactics MITRE Focus on "ML Supply Chain Compromise" and linked techniques.
Blog Securing the ML supply chain Hugging Face Practical guidance on model provenance and malicious models.
Paper Poisoning Web-Scale Training Datasets is Practical Carlini et al. (IEEE S&P 2024) Foundational paper on data poisoning. Section 1-3 is enough.

Exercise

Pick 3 random models from Hugging Face trending. For each, write a 5-line risk assessment: license, training data claim, author reputation, download count, license compatibility with commercial use.

Checkpoint

Name 3 supply-chain risks unique to LLM deployments that do not exist in classic web apps.

Week 7

Evals and Guardrails — Tooling

Move from theory to tools. You cannot call yourself an AI Security engineer without having run at least one eval harness against a model.

Resources

Docs Promptfoo — Getting Started Promptfoo Most practical eval harness. Works with any model. Install and use.
Docs Garak — LLM vulnerability scanner NVIDIA The closest thing to "nmap for LLMs". Ships with injection, jailbreak, toxicity probes.
Blog A statistical approach to model evaluations Anthropic Why naive eval scores are misleading. Short and necessary.
Docs NVIDIA NeMo Guardrails — Introduction NVIDIA Read the intro and architecture overview. Do not install — just understand the model.

Exercise

Install Promptfoo. Write a YAML config that runs 5 prompt injection probes against a local Ollama model. Run it. Commit the config and the output report to a personal repo.

Checkpoint

Explain the difference between static and adversarial evals in 2 sentences.

Week 8

Fuzzing, Red-Teaming Basics, First Public Output

Consolidate the 8 weeks and ship your first public artifact. Public output is how this plan becomes visible.

Resources

Paper An Empirical Study of the Reliability of UNIX Utilities (1990) Miller, Fredriksen, So — Communications of the ACM The fuzzing foundational paper. Short. Every security engineer should have read it once.
Blog Red teaming language models with language models Perez et al. — DeepMind / arXiv The paper that started automated LLM red-teaming. Abstract + Section 3 is enough.
Docs Garak probes reference NVIDIA Garak docs Run at least one probe class against a model you have locally.
Blog OWASP LLM AI Security & Governance Checklist OWASP GenAI Capstone reference document. Use it to audit what you now know vs what you don't.

Exercise

Run Garak against a local Ollama model with 1 probe class (suggested: "promptinject" or "dan"). Save the report. Fork or star one open-source LLM security project that was useful during these 8 weeks.

Checkpoint

Publish one public artifact: a blog post, LinkedIn post, or GitHub README summarizing the most surprising thing you learned across these 8 weeks. This is the first public output of your AI Security pivot. Link it in this checkpoint.

From AppSec to AI Security

Who this is for

How to use it

LLM Foundations I — Tokens, Prompts, Determinism

Resources

Exercise

Checkpoint

LLM Foundations II — Attention, Context, Sampling

Resources

Exercise

Checkpoint

OWASP LLM Top 10 — The Attack Surface

Resources

Exercise

Checkpoint

Prompt Injection — Direct and Indirect

Resources

Exercise

Checkpoint

Sensitive Information Disclosure — LLM02

Resources

Exercise

Checkpoint

Supply Chain and Model Risks — LLM03, LLM05

Resources

Exercise

Checkpoint

Evals and Guardrails — Tooling

Resources

Exercise

Checkpoint

Fuzzing, Red-Teaming Basics, First Public Output

Resources

Exercise

Checkpoint

Upcoming — to unlock

What this roadmap does not cover