The goal was to introduce novel strategies for employing Indirect Reasoning (IR) to address the constraints of direct reasoning. This approach offers an alternative and effective method for tackling practical problems.
The study also makes a number of prompt templates available which effectively stimulate LLMs to follow indirect reasoning.
The aim of the study was to keep the implementation light and prompt based, without any dependancy on external data. Hence approaches like fine-tuning, RAG-based implementations, or tool base (agent-like) were avoided.
LLMs often struggle to grasp complex rules, affecting their ability to use them effectively.
Consider the following:
Bob does not drive to work
If the weather is fine, Bob drives to work
Humans can apply the equivalence of contrapositive to deduce that the rule is equivalent to:
If Bob does not drive to work, the weather is not fine hence humans can deduce.
And this allows humans to conclude based on the rule, that
The weather is not fine .
LLMs can find this reasoning approach challenging, hence to address this issue, the study propose adding the contrapositive of rules to the rule set.
Hence applying at type of in-context learning, with few-shot learning.