Retrieval-augmented generation (RAG) models have recently demonstrated great success on language tasks by combining the scaling benefits of large neural language models with fast access to external knowledge.
Models achieve state-of-the-art performance on benchmarks by retrieving and conditioning on relevant knowledge to generate more informed responses.
It will become a system design to leverage generative AI in corpora settings.
However, despite their strong results, complex reasoning abilities remain a stubborn challenge for these models. RAG models still struggle with logical inconsistencies, biases, and lack of explainability that erode trustworthiness. Their underlying neural foundations make transparent reasoning difficult.
Prior methods such as chain-of-thought prompting have sought to tackle these limitations in large language models, but significant gaps remain.
To strengthen neural reasoning, we explore integrating orthogonal forms of symbolic guidance into RAG models:
- “Large Language Models as an Indirect Reasoner”: This technique proposes enhancing reasoning through logical contrapositives and proof-by-contradiction templates injected into model prompting.
- “In-Context Principle Learning from Mistakes”: This approach has models reflect on their own incorrect predictions to elicit high-level principles that prune faulty reasoning branches on future examples.
These methods provide useful but ephemeral signals for improving model logic. Our goal is achieving durable integration of such symbolic guardrails alongside existing vector knowledge.
We propose encoding logical rules and mistake-based principles directly into the knowledge graphs of RAG models using nodes and typed relationships.
The retriever identifies relevant symbolic reasoning chains to guide the generator at inference time.
We propose experiments to demonstrate this hybrid approach significantly boosts reasoning accuracy on HotpotQA, math, and BIG-Bench benchmarks compared to baseline RAG models…