LLM-Oracle Machines

Abstract

Contemporary AI applications leverage large language models (LLMs) to harness their knowledge and reasoning abilities for natural language processing tasks. This approach shares similarities with the concept of oracle Turing machines (OTMs). To capture the broader potential of these computations, including those not yet realized, we propose an extension to OTMs: the LLM-oracle machine (LLM-OM), by employing a cluster of LLMs as the oracle. Each LLM acts as a black box, capable of answering queries within its expertise, albeit with a delay. We introduce four variants of the LLM-OM: basic, augmented, fault-avoidance, and $\epsilon$-fault. The first two are commonly observed in existing AI applications. The latter two are specifically designed to address the challenges of LLM hallucinations, biases, and inconsistencies, aiming to ensure reliable outcomes.

1 Introduction

In an oracle Turing machine, the oracle embodies a decision problem. It acts as a hypothetical, all-powerful entity that can instantly determine whether a query, an instance generated during the computation, is a positive instance of the decision problem. OTMs have played a significant role in the development of both computation theory and computational complexity theory.

Drawing inspiration from the concept of using external knowledge to assist with computing tasks, and motivated by the recent advancements in LLMs with their powerful knowledge bases and inference capabilities, we use a cluster of LLMs in place of the oracle in OTMs.

We treat LLMs as black boxes, capable of answering queries within their domains of expertise, albeit with a delay. Both queries and responses are exchanged in natural language, encompassing formatting. Since each LLM is trained on diverse datasets using different technologies, their capabilities vary. Therefore, we use a cluster of LLMs as the oracle, selecting an appropriate LLM to respond to a query during an LLM-OM computation.

Unlike in an OTM, where the oracle represents a decision problem and a query asks whether an instance is positive, a query generated in the computation of an LLM-OM consists of a task and a sequence of specifications, with the response being a solution to complete the task.

In a nutshell, the computation of an LLM-OM takes an input representing a task to accomplish, generates queries, acquires answers to each query from the appropriate LLM in the LLM-oracle, and continues this process until the LLM-OM reaches a halting state with the final answer.

Unlike in an OTM where the oracle reliably provides an answer to a query, LLMs can generate fabricated or misleading information, resulting in incorrect or inadequate answers (e.g., see [1, 2]). LLMs may also provide answers with different meanings to the same query at different times. These issues of information hallucination (or better phrased as “information nonsense” because LLMs cannot distinguish between truth and lies), inadequacy, and inconsistency are common in LLMs.

While advancing technologies aim to mitigate these issues, complete elimination remains challenging. Therefore, we assume that there exists a probability that an LLM may provide an unacceptable answer to a query. Depending on the context of the application, an unacceptable answer could be one that is outright incorrect or one that, while not incorrect, fails to meet the required level of adequacy.

2 LLM-OM Basics and Variants

An LLM-OM can be viewed as a deterministic algorithm with access to an LLM oracle. Similar to an OTM, each computation step in an LLM-OM represents a transition. We denote an LLM-OM as $M$, the input representing the task inquiry (including optional specifications) as $Q$, and the output as the answer $A$ to $Q$. This output $A$ can take various forms, including human-like text, code snippets, or other representations. Both inputs $Q$ and outputs $A$ can be encoded as binary strings for compatibility with traditional computing models.

During the computation, $M$ generates a query in the form of $(x;y_{1},y_{2},\ldots,y_{k})$, denoted as $q_{x}$, where $x$ is an intermediate task for completing $Q$ and each $y_{i}$ is an attribute. These attributes can be

• Selected text to provide relevant context for $x$.

• Specific requirement to detail the desired outcome of $x$.

• Solution format to instruct $M$ on how to express the answer (e.g., text, code).

• Verification method to specify how to validate the answer’s correctness.

• Self-critique instruction to guide $M$ to evaluate its own response.

Other specifications with any additional details relevant to completing $x$ may also be attributes. Collectively, these attributes form a prompt that instructs the LLMs on how to approach the intermediate task $x$ effectively.

The computation begins with $M$ interpreting the input task $Q$. If possible, it decomposes $Q$ into a sequence of smaller, more manageable subtasks, denoted as $Q_{1},Q_{2},\ldots,Q_{m}$. However, if $Q$ cannot be further broken down, it remains the sole subtask.

2.1 Basic variants: adaptive and non-adaptive

The basic variant of the LLM-OM utilizes two query types: adaptive and non-adaptive.

In an adaptive LLM-OM: subtasks are interdependent. This means some subtasks require answers from previous subtasks before generating all queries. For a subtask $Q_{i}$, the LLM-OM generates a query $q_{i,x}$ specifying the task within the query. It then retrieves an answer $a_{x}$ from a chosen LLM in the oracle and uses $a_{x}$ to determine the next step. The final answer is derived by combining answers from subtasks, potentially involving the LLM-oracle again.

In a non-adaptive LLM-OM, subtasks are independent. Each subtask $Q_{i}$ is completed by sending a set of independent queries generated during the computation to the LLM-oracle. The final answer is produced solely from these answers, without further interaction with the LLM-oracle.

Ideally, the final answer $A$ should directly address the original inquiry $Q$ and remain relevant to the topic. Let $\text{toc}(X)$ denote the collection of topics within text $X$.

• We say that $A$ is relevant to $Q$, denoted by $A\lesssim Q$, if $\mbox{toc}(Q)\subseteq\mbox{toc}(A)$.

Remark 1. The basic functionality of many LLM web applications, like free versions of ChatGPT and Gemini, exemplifies the non-adaptive LLM-OM variant. In these applications, users submit queries, which directly translate to the subtasks in the LLM-OM. The system then independently generates a set of queries based on the user’s input and sends them to the underlying LLM-oracle. Finally, the application presents the user with the final answer derived solely from these responses, without further interaction with the LLM.

2.2 Augmented LLM-OM

The augmented LLM-OM takes a pair $(T,Q)$ as input. Here, $T$ represents an “augmented text” in natural language, acting as verified background knowledge or ground truth. $Q$ remains the task inquiry, specifying what information to extract or infer from $T$. This information can include specific sentences, topics, summaries, entities, relationships between entities, events, event relationships, logical consequences, or numerical consequences. Similar to the basic variant, queries generated during the computation can be either adaptive or non-adaptive.

Ideally, the answer $A$ to the inquiry $Q$ should align with the augmented text $T$. Here’s a formal definition of this alignment. Let $\mbox{mng}(X)$ denote the set of meanings of text $X$.

• We say that $A$ complies with $T$ with respect to $Q$, written as $A\lesssim_{Q}T$, if the following conditions hold:

  1. Relevance: $A$ is relevant to $Q$. Namely, $Q\lesssim A$.

  2. Inclusiveness: $A$ is inclusive in $T$. Namely, $\mbox{mng}(A)\subseteq\mbox{mng}(T)$.

Note that even if two texts $X$ and $Y$ have the same set of meanings (i.e. $\mbox{mng}(X)=\mbox{mng}(Y)$), it doesn’t necessarily mean they are identical. There are many ways to express the same idea with different wording.

To complete the task $x$ represented by query $q_{x}$, the LLM-OM $M$ first identifies relevant content, denoted as $C_{x}$, within the augmented text $T$. It then leverages this content $C_{x}$ along with an appropriate LLM from the LLM-oracle to generate an answer to $x$, denoted as $a_{x}$, ideally complying with $C_{x}$ with respect to $q_{x}$, namely, $a_{x}\lesssim_{q_{x}}C_{x}$.

Remark 2. Web applications like ChatGPT 4o that employ Retrieval-Augmented Generation (RAG) techniques with an LLM can be seen as a practical application of the augmented LLM-OM framework. In RAG-based applications, the retrieved information acts as the augmented text that confines the LLM’s response. This retrieved information helps the LLM generate answers that are more grounded in factual evidence and more likely to comply with the user’s query.

2.3 Fault-avoidance LLM-OM

The basic and augmented LLM-OM variants do not inherently guarantee consistency, correctness, or adequacy in the final answer $A$ for several reasons:

1. Limitations of LLMs: Even the most advanced LLMs can be susceptible to biases, factual inaccuracies, and hallucinations in their responses. These issues can directly translate into inconsistencies or errors in the final answer generated by the LLM-OM.

2. Incomplete or unclear input: If the initial user inquiry $Q$ or the augmented text $T$ in the augmented variant is ambiguous, incomplete, or misleading, it can lead the LLM-OM down an incorrect path and ultimately result in an inadequate or incorrect answer.

3. Dependence on LLM selection: The choice of LLM from the LLM-oracle can also influence the outcome. Different LLMs have varying strengths and weaknesses, and an inappropriate selection might hinder the generation of a consistent or accurate answer.

4. Query design challenges: Crafting effective intermediate queries $q_{x}$ is crucial. Poorly designed queries can lead the LLM-OM to misunderstand the intent or miss key aspects of the overall task, compromising the final answer’s adequacy.

For the purpose of illustration, we assume that both the user inquiry $Q$ and the augmented text $T$ in the augmented variant are well-defined and free from ambiguity, incompleteness, and misleading information. This idealized scenario allows us to focus on the limitations inherent to the LLM-oracle itself, independent of potential issues with the input data. We define these terms (consistency, correctness, and adequacy) as follows:

• $M$ is consistent if, for any user inquiries $Q$ and $Q^{\prime}$ posed at different points in time, if $\mbox{mng}(Q^{\prime})=\mbox{mng}(Q)$, then $M$ outputs an answer $A$ to $Q$ and an answer $A^{\prime}$ to $Q^{\prime}$ such that $\mbox{mng}(A)=\mbox{mng}(A^{\prime})$.

• Suppose $M$ is augmented with input and output being $(T,Q)$ and $A$. We say that $A$ is correct for $Q$ with respect to $T$ if the following conditions hold:

  1. Compliance: $A$ complies with the ground truth $T$ with respect to $Q$. Namely, $A\lesssim_{Q}T$.

  2. Exclusiveness: $A$ doesn’t include unnecessary information outside the scope of $Q$. In other words, if we let $\mbox{intent}(Q)$ denote the set of intended meanings of $Q$, then $\mbox{mng}(A)\subseteq\mbox{intent}(Q)$.

  3. Completeness: $A$ doesn’t miss any information contained in $T$ that are relevant to $Q$. In other words, $(\mbox{mng}(T)-\mbox{mng}(A))\cap\mbox{intent}(Q)=\emptyset$.

  4. Distribution agreement: If $A$ has more than one meaning, then $\mbox{dist}(A)=\mbox{dist}(T_{A})$, where $\mbox{dist}(A)$ denotes the distribution of meanings within answer $A$ and $\mbox{dist}(T_{A})$ denotes the distribution of meanings in the corresponding text for $A$ in the augmented text $T$.

  We say that $M$ is correct with respect to $T$ if for any input $(T,Q)$, $M$ returns an answer $A$ that is correct for $Q$ with respect to $T$.

• Suppose $M$ is an augmented LLM-OM with input $(T,Q)$ that generates an answer $A$. We say that answer $A$ is adequate for $Q$ with respect to $T$ if $A$ complies with $T$ with respect to $Q$. However, unlike correctness, adequacy allows for some flexibility in the answer:

  1. Non-exclusiveness: $A$ may contain information that extend beyond the specific information requested in the user inquiry $Q$. Namely, $\mbox{mng}(A)-\mbox{intent}(Q)\not=\emptyset$.

  2. Incompleteness: $A$ may miss information from $T$ that pertains to $Q$. Namely, $(\mbox{mng}(T)-\mbox{mng}(A))\cap\mbox{intent}(Q)\not=\emptyset$.

  3. Distribution discrepancy: $\mbox{dist}(A)\not=\mbox{dist}(T_{A})$.

  Essentially, adequacy acknowledges that the answer $A$ may not be perfect, but it provides valuable information to the query from the augmented text.

  We say that $M$ is adequate if for all inputs $(T,Q)$, $M$ returns an answer $A$ that is adequate for $Q$ with respect to $T$.

• If $M$ is not augmented, assume the existence of a set of texts, denoted by $U$, that represents the true knowledge and information for the areas of interest. We say that $M$ is correct if $M$ is correct with respect to $U$, $M$ is adequate if $M$ is adequate with respect to $U$. Set $U$ is referred to as the absolute truth.

Note that if $A$ is correct for $Q$ with respect to $T$, then $A$ is adequate for $Q$ with respect to $T$, but not vice versa.

A fault-avoidance LLM-OM is an augmented LLM-OM that is consistent and correct. A weak fault-avoidance LLM-OM is an augmented LLM-OM that is consistent and adequate. It is necessary for $M$ to identify the best-matched content $C_{x}$ from $T$ for each query $q_{x}$, and the chosen LLM from the LLM-oracle must comply with $C_{x}$ when generating an answer $a_{x}$ to $q_{x}$.

Verifying the correctness or adequacy of the answer $A$ for a given query $Q$ with respect to $T$ calls for concrete implementations of $\mbox{toc}(X)$, $\mbox{mng}(X)$, $\mbox{intent}(Q)$, and $\mbox{dist}(X)$ using techniques in natural language processing, machine learning, deep learning, and other methods.

Consistence, however, is harder to verify. We may aim to develop a method that provides a certain guarantee that $M$ is consistent with a desired high probability.

2.4 $\epsilon$-fault LLM-OM

An $\epsilon$-fault LLM-OM is a non-augmented LLM-OM that is consistent and correct with probability of $1-\epsilon$ with respect to the absolute truth for the areas of interest, where $\epsilon$ is some small positive parameter. Likewise, a weak $\epsilon$-fault LLM-OM is similarly defined by replacing correctness with adequacy.

Let $L_{1},L_{2},\ldots,L_{k}$ be the LLMs in the LLM-oracle, where each $L_{i}$ has a small probability $p_{i}$ of generating hallucinated answers to queries. Different LLMs may hallucinate on different queries. However, $M$ doesn’t know which LLM will hallucinate an answer to a given query, and $M$ alone cannot verify if an answer is incorrect, as the absolute truth is not provided as an augmented input.

We aim to investigate whether it is possible to utilize these LLMs to obtain an answer to a query with a certain level of guarantee that the answer is correct or adequate with a desired probability $1-\epsilon$ for some $\epsilon$.

Making certain reasonable assumptions may be useful in the quest to obtain such a result. For example, we may assume that for a given query, there is always an $L_{i}$ that produces a correct and adequate answer; we just don’t know which one.