Louis Columbus at VentureBeat published a piece yesterday on a gap that has been bothering me for a while, and he quoted me on it. Prompt injection sits at number one on the OWASP Top 10 for LLM Applications, where it has been for three years running. In the public incident record, however, it sits at 12th.
Both numbers are real. Hence, the rub.
What the gap measures
The instinct when two measurements disagree is to pick the one you trust and move on. I would rather accept the disagreement because it tells us a lot.
Here is how I put it to Louis:
We had two ways of measuring the same risk, expert judgment and the public incident record, and they disagree with each other. Neither one is the truth. Two witnesses are contradicting each other, and we can't tell you which one is lying.
The expert side is a survey of roughly 29 practitioners. The incident side is 6,639 labeled incidents drawn from a corpus of 7,714 pulled out of CVE, GitHub Security Advisories, OSV, and AIAAIC. The agreement between them, measured as Cohen's kappa, comes out at 0.20, with a 90% confidence interval running from -0.16 to 0.57.
An interval that crosses zero is worth pausing on. It means the data cannot rule out the possibility that the two rankings agree no better than chance. It also cannot rule out moderate agreement. The honest reading is that we do not have enough signal to say, and a number reported without that interval would imply a precision nobody has earned.
Why a low score is not a verdict on the experts
The reflex when a kappa comes back at 0.20 is to decide somebody is wrong. Usually the experts, because the incident record feels more objective. Counting things feels like measurement in a way that asking people does not.
That reflex skips a question. What are the two sides counting?
That number is telling you about our categories, not about our experts.
A CVE gets filed when a specific product has a specific flaw at a specific version. Prompt injection is not that shape. It is a model doing exactly what it was built to do with text it should not have trusted. There is no version number to pin, no patch to ship, and frequently no vendor who agrees anything is broken. The incident record undercounts it because the incident record was built to count a different kind of thing.
So a low agreement score between the two is partly a measurement artifact. The categories do not line up, and forcing them into the same ranking produces a number that describes the seams rather than the risk.
The part that should change what you do on Monday
If you take one thing from the piece, take this. The attack does not announce itself.
A model running on a poisoned instruction doesn't act broken. It acts certain. Certainty is what your monitoring treats as a healthy system.
That is the operational problem in one line. Most detection is tuned to find things behaving badly: errors, retries, anomalous exits, malformed output. A model working from an instruction it should never have accepted produces none of those. It produces a fluent, confident answer, and the confidence is the thing your dashboards read as health.
You will not scan your way to finding it, because there is no signature to match. What you can do is bound what the model is allowed to reach when it is wrong. Authorization gates on the actions an agent can take. Permissions scoped to the task rather than to the identity. A ceiling on blast radius that holds whether or not the instruction was legitimate.
That is architecture, not detection, and it is the work that pays whether prompt injection is genuinely number one or genuinely number twelve.
How I would spend the budget
Rankings are a poor way to allocate money. They are a summary of somebody else's environment, and yours is not theirs.
I'd prioritize spend where the expert vote and the incident record point the same direction, because that's two independent witnesses agreeing.
Where the two disagree, the disagreement itself tells you something: either the category is badly drawn or the evidence has not caught up. Neither is a reason to spend first. Where they agree, you have two independent lines of evidence pointing at the same place, which is about as much confidence as this field currently offers.
Then weight all of it against your own exposure data, which beats any published ranking for the simple reason that it is about you.
Read the piece
Louis talked to several people for this and the reporting is careful, including about what the numbers cannot support. Worth your time if you are trying to decide what to do about any of this.
Sources
- Prompt injection ranks No. 1 with OWASP and No. 12 in the incident record. The attack itself is invisible to a scan., Louis Columbus, VentureBeat, 25 August 2026
- OWASP Top 10 for LLM Applications
- OWASP LLM01: Prompt Injection