Reasoning without Observation¶
In ReWOO, Xu, et. al, propose an agent that combines a multi-step planner and variable substitution for effective tool use. It was designed to improve on the ReACT-style agent architecture in the following ways:
- Reduce token consumption and execution time by generating the full chain of tools used in a single pass. (ReACT-style agent architecture requires many LLM calls with redundant prefixes (since the system prompt and previous steps are provided to the LLM for each reasoning step)
- Simplify the fine-tuning process. Since the planning data doesn't depend on the outputs of the tool, models can be fine-tuned without actually invoking the tools (in theory).
The following diagram outlines ReWOO's overall computation graph:
ReWOO is made of 3 modules:
- 🧠Planner: Generate the plan in the following format:
Plan: <reasoning>
#E1 = Tool[argument for tool]
Plan: <reasoning>
#E2 = Tool[argument for tool with #E1 variable substitution]
...
- Worker: executes the tool with the provided arguments.
- 🧠Solver: generates the answer for the initial task based on the tool observations.
The modules with a 🧠 emoji depend on an LLM call. Notice that we avoid redundant calls to the planner LLM by using variable substitution.
In this example, each module is represented by a LangGraph node. The end result will leave a trace that looks like this one. Let's get started!
0. Prerequisites¶
For this example, we will provide the agent with a Tavily search engine tool. You can get an API key here or replace with a free tool option (e.g., duck duck go search).
For this notebook, you should add a .env
file at the root of the repo with
TAVILY_API_KEY
:
import "dotenv/config";
Install dependencies¶
npm install langchain @langchain/community @langchain/openai @langchain/core
Graph State: In LangGraph, every node updates a shared graph state. The state is the input to any node whenever it is invoked.
Below, we will define a state object to contain the task, plan, steps, and other variables.
import { Annotation } from "@langchain/langgraph";
const GraphState = Annotation.Root({
task: Annotation<string>({
reducer: (x, y) => (y ?? x),
default: () => "",
}),
planString: Annotation<string>({
reducer: (x, y) => (y ?? x),
default: () => "",
}),
steps: Annotation<string[][]>({
reducer: (x, y) => x.concat(y),
default: () => [],
}),
results: Annotation<Record<string, any>>({
reducer: (x, y) => ({ ...x, ...y }),
default: () => ({}),
}),
result: Annotation<string>({
reducer: (x, y) => (y ?? x),
default: () => "",
}),
})
1. Planner¶
The planner prompts an LLM to generate a plan in the form of a task list. The
arguments to each task are strings that may contain special variables
(#E{{0-9}}+
) that are used for variable substitution from other task results.
Our example agent will have two tools:
- Google - a search engine (in this case Tavily)
- LLM - an LLM call to reason about previous outputs.
The LLM tool receives less of the prompt context and so can be more token-efficient than the ReACT paradigm.
import { ChatOpenAI } from "@langchain/openai";
const model = new ChatOpenAI({
model: "gpt-4o",
temperature: 0,
});
import { ChatPromptTemplate } from "@langchain/core/prompts";
const template =
`For the following task, make plans that can solve the problem step by step. For each plan, indicate
which external tool together with tool input to retrieve evidence. You can store the evidence into a
variable #E that can be called by later tools. (Plan, #E1, Plan, #E2, Plan, ...)
Tools can be one of the following:
(1) Google[input]: Worker that searches results from Google. Useful when you need to find short
and succinct answers about a specific topic. The input should be a search query.
(2) LLM[input]: A pre-trained LLM like yourself. Useful when you need to act with general
world knowledge and common sense. Prioritize it when you are confident in solving the problem
yourself. Input can be any instruction.
For example,
Task: Thomas, Toby, and Rebecca worked a total of 157 hours in one week. Thomas worked x
hours. Toby worked 10 hours less than twice what Thomas worked, and Rebecca worked 8 hours
less than Toby. How many hours did Rebecca work?
Plan: Given Thomas worked x hours, translate the problem into algebraic expressions and solve with Wolfram Alpha.
#E1 = WolframAlpha[Solve x + (2x - 10) + ((2x - 10) - 8) = 157]
Plan: Find out the number of hours Thomas worked.
#E2 = LLM[What is x, given #E1]
Plan: Calculate the number of hours Rebecca worked.
#E3 = Calculator[(2 * #E2 - 10) - 8]
Important!
Variables/results MUST be referenced using the # symbol!
The plan will be executed as a program, so no coreference resolution apart from naive variable replacement is allowed.
The ONLY way for steps to share context is by including #E<step> within the arguments of the tool.
Begin!
Describe your plans with rich details. Each Plan should be followed by only one #E.
Task: {task}`;
const promptTemplate = ChatPromptTemplate.fromMessages([["human", template]]);
const planner = promptTemplate.pipe(model);
const task = "what is the hometown of the winner of the 2023 australian open?";
await planner.invoke({ task });
AIMessage { "id": "chatcmpl-9z88bDgCFkpWbYitlBSkuEaUU0YA2", "content": "Plan: Identify the winner of the 2023 Australian Open.\n#E1 = Google[\"winner of the 2023 Australian Open\"]\n\nPlan: Find the hometown of the winner identified in #E1.\n#E2 = Google[\"hometown of #E1\"]", "additional_kwargs": {}, "response_metadata": { "tokenUsage": { "completionTokens": 55, "promptTokens": 438, "totalTokens": 493 }, "finish_reason": "stop", "system_fingerprint": "fp_3aa7262c27" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 438, "output_tokens": 55, "total_tokens": 493 } }
Planner Node¶
To connect the planner to our graph, we will create a getPlan
node that
accepts the ReWOO
state and returns with a state update for the steps
and
planString
fields.
import { RunnableConfig } from "@langchain/core/runnables";
const regexPattern = new RegExp(
"Plan\\s*\\d*:\\s*([^#]+)\\s*(#E\\d+)\\s*=\\s*(\\w+)\\s*\\[([^\\]]+)\\]",
"g",
);
async function getPlan(state: typeof GraphState.State, config?: RunnableConfig) {
console.log("---GET PLAN---");
const task = state.task;
const result = await planner.invoke({ task }, config);
// Find all matches in the sample text.
const matches = result.content.toString().matchAll(regexPattern);
let steps: string[][] = [];
for (const match of matches) {
const item = [match[1], match[2], match[3], match[4], match[0]];
if (item.some((i) => i === undefined)) {
throw new Error("Invalid match");
}
steps.push(item as string[]);
}
return {
steps,
planString: result.content.toString(),
};
}
2. Executor¶
The executor receives the plan and executes the tools in sequence.
Below, instantiate the search engine and define the tools execution node.
import { TavilySearchResults } from "@langchain/community/tools/tavily_search";
const search = new TavilySearchResults();
const _getCurrentTask = (state: typeof GraphState.State) => {
console.log("_getCurrentTask", state);
if (!state.results) {
return 1;
}
if (Object.entries(state.results).length === state.steps.length) {
return null;
}
return Object.entries(state.results).length + 1;
};
const _parseResult = (input: unknown) => {
if (typeof input === "string") {
const parsedInput = JSON.parse(input);
if (Array.isArray(parsedInput) && "content" in parsedInput[0]) {
// This means it is a tool result.
return parsedInput.map(({ content }) => content).join("\n");
}
}
if (input && typeof input === "object" && "content" in input) {
// If it's not a tool, we know it's an LLM result.
const { content } = input;
return content;
}
throw new Error("Invalid input received");
};
async function toolExecution(state: typeof GraphState.State, config?: RunnableConfig) {
console.log("---EXECUTE TOOL---");
const _step = _getCurrentTask(state);
if (_step === null) {
throw new Error("No current task found");
}
const [_, stepName, tool, toolInputTemplate] = state.steps[_step - 1];
let toolInput = toolInputTemplate;
const _results = state.results || {};
for (const [k, v] of Object.entries(_results)) {
toolInput = toolInput.replace(k, v);
}
let result;
if (tool === "Google") {
result = await search.invoke(toolInput, config);
} else if (tool === "LLM") {
result = await model.invoke(toolInput, config);
} else {
throw new Error("Invalid tool specified");
}
_results[stepName] = JSON.stringify(_parseResult(result), null, 2);
return { results: _results };
}
3. Solver¶
The solver receives the full plan and generates the final response based on the responses of the tool calls from the worker.
const solvePrompt = ChatPromptTemplate.fromTemplate(
`Solve the following task or problem. To solve the problem, we have made step-by-step Plan and
retrieved corresponding Evidence to each Plan. Use them with caution since long evidence might
contain irrelevant information.
{plan}
Now solve the question or task according to provided Evidence above. Respond with the answer
directly with no extra words.
Task: {task}
Response:`,
);
async function solve(state: typeof GraphState.State, config?: RunnableConfig) {
console.log("---SOLVE---");
let plan = "";
const _results = state.results || {};
for (let [_plan, stepName, tool, toolInput] of state.steps) {
for (const [k, v] of Object.entries(_results)) {
toolInput = toolInput.replace(k, v);
}
plan += `Plan: ${_plan}\n${stepName} = ${tool}[${toolInput}]\n`;
}
const model = new ChatOpenAI({
temperature: 0,
model: "gpt-4o",
});
const result = await solvePrompt
.pipe(model)
.invoke({ plan, task: state.task }, config);
return {
result: result.content.toString(),
};
}
4. Define Graph¶
Our graph defines the workflow. Each of the planner, tool executor, and solver modules are added as nodes.
import { END, START, StateGraph } from "@langchain/langgraph";
import { MemorySaver } from "@langchain/langgraph";
const _route = (state: typeof GraphState.State) => {
console.log("---ROUTE TASK---");
const _step = _getCurrentTask(state);
if (_step === null) {
// We have executed all tasks
return "solve";
}
// We are still executing tasks, loop back to the "tool" node
return "tool";
};
const workflow = new StateGraph(GraphState)
.addNode("plan", getPlan)
.addNode("tool", toolExecution)
.addNode("solve", solve)
.addEdge("plan", "tool")
.addEdge("solve", END)
.addConditionalEdges("tool", _route)
.addEdge(START, "plan");
// Compile
const app = workflow.compile({ checkpointer: new MemorySaver() });
const threadConfig = { configurable: { thread_id: "123" } };
let finalResult;
const stream = await app.stream(
{
task: "what is the hometown of the winner of the 2023 australian open?",
},
threadConfig,
);
for await (const item of stream) {
console.log(item);
console.log("-----");
finalResult = item;
}
---GET PLAN--- { plan: { planString: 'Plan: Identify the winner of the 2023 Australian Open.\n' + '#E1 = Google["winner of the 2023 Australian Open"]\n' + '\n' + 'Plan: Find the hometown of the winner identified in #E1.\n' + '#E2 = Google["hometown of #E1"]', steps: [ [Array] ] } } ----- ---EXECUTE TOOL--- _getCurrentTask { task: 'what is the hometown of the winner of the 2023 australian open?', planString: 'Plan: Identify the winner of the 2023 Australian Open.\n' + '#E1 = Google["winner of the 2023 Australian Open"]\n' + '\n' + 'Plan: Find the hometown of the winner identified in #E1.\n' + '#E2 = Google["hometown of #E1"]', steps: [ [ 'Identify the winner of the 2023 Australian Open.\n', '#E1', 'Google', '"winner of the 2023 Australian Open"', 'Plan: Identify the winner of the 2023 Australian Open.\n' + '#E1 = Google["winner of the 2023 Australian Open"]' ] ], results: {}, result: '' } ---ROUTE TASK--- _getCurrentTask { task: 'what is the hometown of the winner of the 2023 australian open?', planString: 'Plan: Identify the winner of the 2023 Australian Open.\n' + '#E1 = Google["winner of the 2023 Australian Open"]\n' + '\n' + 'Plan: Find the hometown of the winner identified in #E1.\n' + '#E2 = Google["hometown of #E1"]', steps: [ [ 'Identify the winner of the 2023 Australian Open.\n', '#E1', 'Google', '"winner of the 2023 Australian Open"', 'Plan: Identify the winner of the 2023 Australian Open.\n' + '#E1 = Google["winner of the 2023 Australian Open"]' ] ], results: { '#E1': `"A one-set shoot-off to decide the winner of the 2023 Australian Open. There could not have been a better script. SECOND SET (* denotes server) Sabalenka* 6-3 Rybakina - Wide second serve into the deuce court from Sabalenka and forehand return from Rybakina is long. Deep backhand crosscourt return from Rybakina draws a shot ball from Sabalenka ...\\nThe Crossword Solver found 30 answers to \\"Tennis player Sabalenka, winner of the 2023 Australian Open\\", 5 letters crossword clue. The Crossword Solver finds answers to classic crosswords and cryptic crossword puzzles. Enter the length or pattern for better results. Click the answer to find similar crossword clues .\\nAccording to bet365, Djokovic has even odds of winning the title at Melbourne Park -- meaning that the 35-year-old has a 50% chance of being the winner of the 2023 Australian Open men's singles ...\\nWe found 40 solutions for Tennis player Sabalenka, winner of the 2023 Australian Open. The top solutions are determined by popularity, ratings and frequency of searches. The most likely answer for the clue is ARYNA. How many solutions does Tennis player Sabalenka, winner of the 2023 Australian Open have?"` }, result: '' } { tool: { results: { '#E1': `"A one-set shoot-off to decide the winner of the 2023 Australian Open. There could not have been a better script. SECOND SET (* denotes server) Sabalenka* 6-3 Rybakina - Wide second serve into the deuce court from Sabalenka and forehand return from Rybakina is long. Deep backhand crosscourt return from Rybakina draws a shot ball from Sabalenka ...\\nThe Crossword Solver found 30 answers to \\"Tennis player Sabalenka, winner of the 2023 Australian Open\\", 5 letters crossword clue. The Crossword Solver finds answers to classic crosswords and cryptic crossword puzzles. Enter the length or pattern for better results. Click the answer to find similar crossword clues .\\nAccording to bet365, Djokovic has even odds of winning the title at Melbourne Park -- meaning that the 35-year-old has a 50% chance of being the winner of the 2023 Australian Open men's singles ...\\nWe found 40 solutions for Tennis player Sabalenka, winner of the 2023 Australian Open. The top solutions are determined by popularity, ratings and frequency of searches. The most likely answer for the clue is ARYNA. How many solutions does Tennis player Sabalenka, winner of the 2023 Australian Open have?"` } } } ----- ---SOLVE--- { solve: { result: 'Belgrade, Serbia' } } -----
const snapshot = await app.getState(threadConfig);
console.log(snapshot.values.result);
Belgrade, Serbia
Conclusion¶
Congratulations on implementing ReWOO! Before you leave, I'll leave you with a couple limitations of the current implementation from the paper:
- If little context of the environment is available, the planner will be ineffective in its tool use. This can typically be ameliorated through few-shot prompting and/or fine-tuning.
- The tasks are still executed in sequence, meaning the total execution time is impacted by every tool call, not just the longest-running in a given step.