Beyond Sequential Hybrid Retrieval: A Parallel Framework for Accurate and Scalable RAG

Abstract Large Language Models (LLMs) frequently generate hallucinated or outdated responses when operating without external knowledge. Retrieval Augmented Generation (RAG) grounds model outputs in retrieved documents, yet existing hybrid RAG systems execute multiple retrievers sequentially, creating latency bottlenecks that limit scalability. Current research focused on PH-RAG, a Parallel Hybrid RAG framework that runs BM25 sparse retrieval and FAISS based dense retrieval simultaneously using a persistent thread pool, fuses their ranked outputs via Reciprocal Rank Fusion (RRF) or Weighted Score Fusion, and applies a cross-encoder reranker for improved top rank precision. This framework coupled a strong instruction tuned bi-encoder (BGE large en-v1.5) with an exact FAISS inner-product index and a lightweight MiniLM cross-encoder, forming a modular four-stage pipeline in which every component can be independently ablated. Evaluated on TriviaQA with 1,000 validation questions and a corpus of 5,470 Wikipedia passages, PH-RAG achieves Hit@1 of 0.656, Hit@5 of 0.797, Recall@20 of 0.833, and MRR of 0.718, surpassing the HybGRAG baseline (ACL 2025) on all four reported metrics without requiring a knowledge graph or an iterative critic module. A theoretical latency model, validated against per retriever measurements across seven corpus sizes, demonstrates parallel speedups of 1.01× to 1.64× for corpora between 1,000 and 20,000 documents, with the advantage growing as sparse-retrieval cost begins to dominate. Ablation studies quantify the individual contributions of fusion strategy, encoder strength, and reranking, showing that the weighted fusion of complementary sparse and dense signals is the primary driver of retrieval quality, while parallel execution provides a deployment time efficiency benefit that is orthogonal to accuracy. Our results indicated that a carefully engineered but architecturally simple hybrid retriever match or exceed substantially more complex agentic RAG systems on open-domain question answering.