Preference-Aligned Passage Reranking Without Decoding for Efficient Retrieval-Augmented Generation

Abstract

Retrieval-augmented generation has emerged as a cornerstone paradigm for mitigating hallucinations and expanding the factual boundaries of large language models. However, the efficacy of the generation stage remains fundamentally bounded by the quality of the retrieved context. Consequently, passage reranking systems are routinely employed as an intermediate step to refine the output of first-stage retrievers. Modern high-performance rerankers increasingly rely on large language models utilizing autoregressive decoding to score passage relevance, a process that introduces severe latency bottlenecks and prohibitive computational overhead. To address this critical inefficiency, this paper introduces a novel methodology for preference-aligned passage reranking that entirely circumvents the decoding phase. By leveraging human and artificial intelligence preference alignment techniques directly on the internal hidden states of the model, our approach projects high-dimensional contextual representations into a calibrated preference space using a single forward pass. This decoding-free architecture eliminates the sequential token generation bottleneck while preserving the sophisticated reasoning capabilities of large-scale models. Through comprehensive empirical evaluations across multiple benchmark datasets, we demonstrate that the proposed framework achieves retrieval performance commensurate with state-of-the-art generative rerankers but operates with an order-of-magnitude reduction in latency. The findings suggest that direct preference optimization of encoder representations offers a highly scalable solution for enterprise-grade retrieval-augmented generation systems, fundamentally altering the trade-off between retrieval accuracy and computational efficiency.