guardrail-rs A Fail-Open Reverse Proxy for Prompt-Injection Defense and PII Redaction in LLM Applications

Abstract Applications built on large language models (LLMs) typically forward user input to a model provider with no enforcement layer in between, leaving prompt-injection attempts and personally identifiable information (PII) to pass through unfiltered in both directions. We present guardrail-rs, an open-source reverse proxy, implemented in Rust, that sits between an application and its LLM provider and inspects every request and response before it crosses the network boundary. The system is built around a composable Stage/Pipeline abstraction supporting regex-based and optional ONNX-based semantic classifiers for prompt-injection detection, regex-based PII redaction, user-defined policy rules, and privacy-preserving observability (metrics, distributed tracing, and a metadata-only audit log). A central design goal is fail-open by default: a malfunctioning security stage degrades gracefully rather than becoming an availability liability. We describe the system’s architecture and implementation as a five-crate Rust workspace, all of which is published and available for use, and report an honest, continuouslytracked empirical performance characterization measured on commodity CI infrastructure, including a case where a component misses its own documented latency target at larger payload sizes. We also report several concrete engineering pitfalls encountered while hardening the system for production use — a security-relevant configuration-drift defect, a dependency-tooling schema change that silently broke continuous integration twice, a single-threaded async-runtime deadlock, and a supply-chain issue that silently blocked releases — each with the general lesson it implies for builders of similar infrastructure. We are explicit about what this work does not yet establish: no formal adversarial-robustness evaluation of the classifiers against established attack suites has been performed, and this is identified as the most important item of future work. guardrail-rs is available at https://github.com /Mattral/guardrail-rs, published on crates.io, and reproducible via a runnable notebook at https://colab.research.google.com/github/Mattral/guardrail-rs/blob/m ain/examples/notebooks/quickstart_colab.ipynb.