A Lightweight Framework for Diagnostic Failure Detection and Adaptive Control in RAG Systems

Abstract

The Retrieval-Augmented Generation approach has emerged as a feasible solution to ground large language models on external knowledge. Nonetheless, the models may experience failures under different circumstances, particularly when retrieval and generation fail together. Most diagnostic techniques require "peeking" into the model or incurring high computational costs, making them difficult to utilize for real-world systems. We present a lightweight diagnostic approach to detect RAG failure without accessing the model’s internal workings. The retrieval similarity, lexical grounding, and natural language inference are collected for forecasting the failure. With this score, the system can choose on the fly to accept, re-retrieve, or abstain from a generated answer. We demonstrate our approach’s performance in detecting RAG failure on RAGTruth and HalluRAG, showing competitive performance with minimal latency impact. Our experiments demonstrate that robust RAG failure diagnosis is feasible solely from external signals, providing a promising solution for real-world RAG systems.