Structured Numerical Hallucinations in Financial LLMs: Deterministic Verification Outperforms Chain-of-Thought Prompting

<div> We study a specific but important failure mode in financial LLMs: numerical&nbsp; </div> <div> errors in task-specifically fine-tuned small models are statistically&nbsp; </div> <div> structured rather than random (chi-squared = 41.97, p &lt; 0.001), and this&nbsp; </div> <div> structure makes deterministic post-hoc correction more effective than&nbsp; </div> <div> reasoning augmentation in our controlled experiments. Scaling reasoning via&nbsp; </div> <div> chain-of-thought (CoT) prompting and retrieval-augmented generation (RAG) —&nbsp; </div> <div> the dominant paradigms for improving LLM numerical accuracy — both degrade&nbsp; </div> <div> performance in this setting (CoT: -9.0pp; RAG: -7.5pp; p &lt; 0.001,&nbsp; </div> <div> McNemar's test). </div> <div> <br> </div> <div> We propose a Deterministic Verification Layer (DVL) — a parameter-free,&nbsp; </div> <div> inference-time module that exploits this structured error distribution to&nbsp; </div> <div> improve execution accuracy from 1.0% to 42.61% (95% CI: 39.5–45.7%) on&nbsp; </div> <div> FinQA through document context and QLoRA fine-tuning. DVL logs 63&nbsp; </div> <div> corrections across three rule types with 100% correction precision on the&nbsp; </div> <div> audited subset (44/44 verified correct, zero false positives). All three&nbsp; </div> <div> rules fire on prediction magnitude, question keywords, and document-context&nbsp; </div> <div> unit headers — no ground-truth access at inference. DVL generalizes across&nbsp; </div> <div> architectures (Llama-3-8B: +10.31pp without retraining), across financial&nbsp; </div> <div> benchmarks (TAT-QA zero-shot: +9.47pp and +19.92pp; p &lt; 0.001, McNemar&nbsp; </div> <div> c = 0), and captures 45.8% of gains achievable by full symbolic&nbsp; </div> <div> recomputation (93.13% oracle accuracy) without gold program annotations.&nbsp; </div> <div> A cross-domain evaluation on DROP establishes that the structured bias is&nbsp; </div> <div> specific to financial training distributions, not a universal LLM property.&nbsp; </div> <div> A mechanistic analysis identifies computational drift as the CoT failure&nbsp; </div> <div> mode: output length increases 17x with zero extraction failures, and a&nbsp; </div> <div> token-level trace shows arithmetic precision degrading at step boundaries.&nbsp; </div> <div> Our system narrows the gap to GPT-3.5 (no CoT) to 5.4pp using a model 25x&nbsp; </div> <div> smaller, on a single consumer GPU, with deterministic auditable outputs. </div>