Research: Prompt Injection Is Role Confusion — "Destyling" Cuts Attack Success From 61% to 10%

Summary

New research from Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell (writeup at role-confusion.github.io, surfaced via Simon Willison) reframes prompt injection as a *role confusion* problem: large language models fail to reliably distinguish privileged internal text (tagged `<system>`, `<think>`, etc.) from untrusted user input because they prioritize the *style/formatting* of text over its actual content labels. An attacker can append malicious instructions written in the style of the model's