Prompting Science Report 5: This is an Excellent Paper: The Effects of Prompt Injection on Grading

<div> This is the fifth in a series of short reports that help business, education, and policy leaders understand the technical details of working with AI through rigorous testing. Here, we ask whether frontier reasoning large language models (LLMs) tasked with grading academic papers can still be influenced by prompt injection across several settings. Drawing from two distinct text corpora, we study one concise and one verbose prompt injection, both designed to artificially inflate grades. For each prompt, we further vary prompt placement, inserting the injected text at the start, in the middle, or at the end of a given paper. </div> <div> <br> </div> <div> We tested a total of four model families: </div> <div> <br> </div> <div> 1. Claude Opus 4.5 </div> <div> 2. Gemini 3 Pro&nbsp; </div> <div> 3. GPT-5.2 </div> <div> 4. GPT-4o mini </div> <div> <br> </div> <div> We found that:&nbsp; </div> <div> - Prompt injections did not meaningfully increase LLM-assigned grades in most of our experiments. Across roughly 40,000 trials using Opus 4.5, Gemini 3 Pro, and GPT-5.2, prompt injections increased scores by about 2.6 percentage points across two independent datasets. </div> <div> - Vulnerability varies across LLMs. Notably, Gemini 3 Pro stood out as the most vulnerable reasoning model, particularly for the longer-paper corpus, where beginning and middle injections produced effects exceeding 10 percentage points, while end-of-document injections had no effect. In our experiments, Opus 4.5 showed only minimal effects. Interestingly, GPT-5.2 with reasoning enabled was slightly more susceptible than the non-reasoning configuration, suggesting that reasoning did not provide additional protection against prompt injections. </div> <div> - GPT-4o mini, a smaller and less capable LLM, however, was highly susceptible, with an average increase of 19 percentage points. </div> <div> - We observed heterogeneity along two design dimensions. First, verbose prompt injections consistently outperformed concise ones by a factor of two to three, and, for GPT-4o mini, by as much as sixfold. Second, placement also mattered depending on the LLM used. While Gemini 3 Pro showed strong effects for beginning and middle placements and only minimal effects at the end, particularly in the longer-paper corpus, the remaining frontier model families showed little to no placement-specific effects. </div>