How to pass config to tools¶
You may need to pass values to a tool that are only known at runtime. For example, the tool logic may require using the ID of the user who made the request.
Most of the time, such values should not be controlled by the LLM. In fact, allowing the LLM to control the user ID may lead to a security risk.
Instead, the LLM should only control the parameters of the tool that are meant to be controlled by the LLM, while other parameters (such as user ID) should be fixed by the application logic.
To pass run time information, we will use tools that leverage the LangChain Runnable interface. The standard runnables methods (invoke, batch, stream etc.) accept a 2nd argument which is a RunnableConfig
. RunnableConfig has a few standard fields, but allows users to use other fields for run time information.
Here, we will show how to set up a simple agent that has access to three tools for saving, reading, and deleting a list of the user's favorite pets.
Setup¶
First we need to install the packages required
npm install @langchain/langgraph @langchain/openai @langchain/core
Next, we need to set API keys for OpenAI (the LLM we will use)
export OPENAI_API_KEY=your-api-key
Optionally, we can set API key for LangSmith tracing, which will give us best-in-class observability.
export LANGCHAIN_TRACING_V2="true"
export LANGCHAIN_CALLBACKS_BACKGROUND="true"
export LANGCHAIN_API_KEY=your-api-key
Defining Tools and Model¶
import { tool } from '@langchain/core/tools';
import { z } from 'zod';
let userToPets = {};
const updateFavoritePets = tool(
async (input, config) => {
let userId = config?.configurable.user_id;
userToPets[userId] = input.pets;
return "update_favorite_pets called.";
},
{
name: "update_favorite_pets",
description: "add to the list of favorite pets.",
schema: z.object({
pets: z.array(z.string()),
}),
}
);
const deleteFavoritePets = tool(
async (_, config) => {
let userId = config?.configurable.user_id;
if (userId in userToPets) {
delete userToPets[userId];
}
return "delete_favorite_pets called.";
},
{
name: "delete_favorite_pets",
description: "Delete the list of favorite pets.",
schema: z.object({}),
}
);
const listFavoritePets = tool(
async (_, config) => {
let userId = config?.configurable.user_id;
return JSON.stringify(userToPets[userId] ?? []);
},
{
name: "list_favorite_pets",
description: "List favorite pets if any.",
schema: z.object({}),
}
);
let tools = [updateFavoritePets, deleteFavoritePets, listFavoritePets];
We'll be using a small chat model from Anthropic in our example. To use chat models with tool calling, we need to first ensure that the model is aware of the available tools. We do this by calling .bindTools
method on ChatAnthropic
moodel
import { ChatAnthropic } from "@langchain/anthropic";
let model = new ChatAnthropic({ model: "claude-3-haiku-20240307", temperature: 0});
let modelWithTools = model.bindTools(tools);
ReAct Agent¶
Let's set up a graph implementation of the ReAct agent. This agent takes some query as input, then repeatedly call tools until it has enough information to resolve the query. We'll be using prebuilt ToolNode
and the Anthropic model with tools we just defined
import { END, START, StateGraph } from "@langchain/langgraph";
import { AIMessage } from "@langchain/core/messages";
import { RunnableConfig } from "@langchain/core/runnables";
import { Annotation } from "@langchain/langgraph";
import { BaseMessage } from "@langchain/core/messages";
import { ToolNode } from "@langchain/langgraph/prebuilt";
const AgentState = Annotation.Root({
messages: Annotation<BaseMessage[]>({
reducer: (x, y) => x.concat(y),
}),
});
const routeMessage = (state: typeof AgentState.State) => {
const { messages } = state;
const lastMessage = messages[messages.length - 1] as AIMessage;
// If no tools are called, we can finish (respond to the user)
if (!lastMessage?.tool_calls?.length) {
return END;
}
// Otherwise if there is, we continue and call the tools
return "tools";
};
const callModel = async (
state: typeof AgentState.State,
config?: RunnableConfig,
) => {
// For versions of @langchain/core < 0.2.3, you must call `.stream()`
// and aggregate the message from chunks instead of calling `.invoke()`.
const { messages } = state;
const responseMessage = await modelWithTools.invoke(messages, config);
return { messages: [responseMessage] };
};
const workflow = new StateGraph(AgentState)
.addNode("agent", callModel)
.addNode("tools", new ToolNode(tools))
.addEdge(START, "agent")
.addConditionalEdges("agent", routeMessage)
.addEdge("tools", "agent");
const graph = workflow.compile();
Use it!¶
Let's use our graph now!
console.log(`User information prior to run: ${JSON.stringify(userToPets)}`);
let inputs = { messages: [{ role: "user", content: "my favorite pets are cats and dogs" }] };
let config = {"configurable": {"user_id": "123"}};
for await (
const chunk of await graph.stream(inputs, {
...config,
})
) {
for (const [node, values] of Object.entries(chunk)) {
console.log(`Output from node: ${node}`);
console.log("---");
console.log(values);
console.log("\n====\n");
}
}
console.log(`User information after run: ${JSON.stringify(userToPets)}`);
User information prior to run: {} Output from node: agent --- { messages: [ AIMessage { "id": "msg_013LjmK6FWaHZoVms4UF1mKU", "content": [ { "type": "text", "text": "Okay, let's update your favorite pets list:" }, { "type": "tool_use", "id": "toolu_01WCBpWiNnqDhKatJmWr24vM", "name": "update_favorite_pets", "input": { "pets": "[Array]" } } ], "additional_kwargs": { "id": "msg_013LjmK6FWaHZoVms4UF1mKU", "type": "message", "role": "assistant", "model": "claude-3-haiku-20240307", "stop_reason": "tool_use", "stop_sequence": null, "usage": { "input_tokens": 509, "output_tokens": 71 } }, "response_metadata": { "id": "msg_013LjmK6FWaHZoVms4UF1mKU", "model": "claude-3-haiku-20240307", "stop_reason": "tool_use", "stop_sequence": null, "usage": { "input_tokens": 509, "output_tokens": 71 }, "type": "message", "role": "assistant" }, "tool_calls": [ { "name": "update_favorite_pets", "args": { "pets": "[Array]" }, "id": "toolu_01WCBpWiNnqDhKatJmWr24vM", "type": "tool_call" } ], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 509, "output_tokens": 71, "total_tokens": 580 } } ] } ==== Output from node: tools --- { messages: [ ToolMessage { "content": "update_favorite_pets called.", "name": "update_favorite_pets", "additional_kwargs": {}, "response_metadata": {}, "tool_call_id": "toolu_01WCBpWiNnqDhKatJmWr24vM" } ] } ==== Output from node: agent --- { messages: [ AIMessage { "id": "msg_01TYuAz6GZ96iN1b9P1xM7GB", "content": "I've added \"cats\" and \"dogs\" to your list of favorite pets.", "additional_kwargs": { "id": "msg_01TYuAz6GZ96iN1b9P1xM7GB", "type": "message", "role": "assistant", "model": "claude-3-haiku-20240307", "stop_reason": "end_turn", "stop_sequence": null, "usage": { "input_tokens": 598, "output_tokens": 21 } }, "response_metadata": { "id": "msg_01TYuAz6GZ96iN1b9P1xM7GB", "model": "claude-3-haiku-20240307", "stop_reason": "end_turn", "stop_sequence": null, "usage": { "input_tokens": 598, "output_tokens": 21 }, "type": "message", "role": "assistant" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 598, "output_tokens": 21, "total_tokens": 619 } } ] } ==== User information after run: {"123":["cats","dogs"]}
console.log(`User information prior to run: ${JSON.stringify(userToPets)}`);
inputs = { messages: [{ role: "user", content: "what are my favorite pets?" }] };
for await (
const chunk of await graph.stream(inputs, {
...config,
})
) {
for (const [node, values] of Object.entries(chunk)) {
console.log(`Output from node: ${node}`);
console.log("---");
console.log(values);
console.log("\n====\n");
}
}
console.log(`User information after run: ${JSON.stringify(userToPets)}`);
User information prior to run: {"123":["cats","dogs"]}
Output from node: agent --- { messages: [ AIMessage { "id": "msg_01F7SRWVEzkh3vSTtTiLsJeq", "content": [ { "type": "text", "text": "Let me check your favorite pets:" }, { "type": "tool_use", "id": "toolu_01RuHX4KKsrofuqmqP1mkNAh", "name": "list_favorite_pets", "input": {} } ], "additional_kwargs": { "id": "msg_01F7SRWVEzkh3vSTtTiLsJeq", "type": "message", "role": "assistant", "model": "claude-3-haiku-20240307", "stop_reason": "tool_use", "stop_sequence": null, "usage": { "input_tokens": 508, "output_tokens": 46 } }, "response_metadata": { "id": "msg_01F7SRWVEzkh3vSTtTiLsJeq", "model": "claude-3-haiku-20240307", "stop_reason": "tool_use", "stop_sequence": null, "usage": { "input_tokens": 508, "output_tokens": 46 }, "type": "message", "role": "assistant" }, "tool_calls": [ { "name": "list_favorite_pets", "args": {}, "id": "toolu_01RuHX4KKsrofuqmqP1mkNAh", "type": "tool_call" } ], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 508, "output_tokens": 46, "total_tokens": 554 } } ] } ==== Output from node: tools --- { messages: [ ToolMessage { "content": "[\"cats\",\"dogs\"]", "name": "list_favorite_pets", "additional_kwargs": {}, "response_metadata": {}, "tool_call_id": "toolu_01RuHX4KKsrofuqmqP1mkNAh" } ] } ==== Output from node: agent --- { messages: [ AIMessage { "id": "msg_017j8SSSwoUBs8irSSantNfU", "content": "Based on the results, your favorite pets are cats and dogs.", "additional_kwargs": { "id": "msg_017j8SSSwoUBs8irSSantNfU", "type": "message", "role": "assistant", "model": "claude-3-haiku-20240307", "stop_reason": "end_turn", "stop_sequence": null, "usage": { "input_tokens": 570, "output_tokens": 17 } }, "response_metadata": { "id": "msg_017j8SSSwoUBs8irSSantNfU", "model": "claude-3-haiku-20240307", "stop_reason": "end_turn", "stop_sequence": null, "usage": { "input_tokens": 570, "output_tokens": 17 }, "type": "message", "role": "assistant" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 570, "output_tokens": 17, "total_tokens": 587 } } ] } ==== User information after run: {"123":["cats","dogs"]}
console.log(`User information prior to run: ${JSON.stringify(userToPets)}`);
inputs = { messages: [{ role: "user", content: "please forget what i told you about my favorite animals" }] };
for await (
const chunk of await graph.stream(inputs, {
...config,
})
) {
for (const [node, values] of Object.entries(chunk)) {
console.log(`Output from node: ${node}`);
console.log("---");
console.log(values);
console.log("\n====\n");
}
}
console.log(`User information after run: ${JSON.stringify(userToPets)}`);
User information prior to run: {"123":["cats","dogs"]}
Output from node: agent --- { messages: [ AIMessage { "id": "msg_015Yx8T4vxWgsN8YhYjWaHLb", "content": [ { "type": "tool_use", "id": "toolu_018SCadcTtjo7KQCmW7Vg1ct", "name": "delete_favorite_pets", "input": {} } ], "additional_kwargs": { "id": "msg_015Yx8T4vxWgsN8YhYjWaHLb", "type": "message", "role": "assistant", "model": "claude-3-haiku-20240307", "stop_reason": "tool_use", "stop_sequence": null, "usage": { "input_tokens": 512, "output_tokens": 38 } }, "response_metadata": { "id": "msg_015Yx8T4vxWgsN8YhYjWaHLb", "model": "claude-3-haiku-20240307", "stop_reason": "tool_use", "stop_sequence": null, "usage": { "input_tokens": 512, "output_tokens": 38 }, "type": "message", "role": "assistant" }, "tool_calls": [ { "name": "delete_favorite_pets", "args": {}, "id": "toolu_018SCadcTtjo7KQCmW7Vg1ct", "type": "tool_call" } ], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 512, "output_tokens": 38, "total_tokens": 550 } } ] } ==== Output from node: tools --- { messages: [ ToolMessage { "content": "delete_favorite_pets called.", "name": "delete_favorite_pets", "additional_kwargs": {}, "response_metadata": {}, "tool_call_id": "toolu_018SCadcTtjo7KQCmW7Vg1ct" } ] } ==== Output from node: agent --- { messages: [ AIMessage { "id": "msg_017bg7oCa17fVyptYFExJcpE", "content": "I have deleted the list of your favorite pets as requested. The list of favorite pets has been cleared.", "additional_kwargs": { "id": "msg_017bg7oCa17fVyptYFExJcpE", "type": "message", "role": "assistant", "model": "claude-3-haiku-20240307", "stop_reason": "end_turn", "stop_sequence": null, "usage": { "input_tokens": 568, "output_tokens": 25 } }, "response_metadata": { "id": "msg_017bg7oCa17fVyptYFExJcpE", "model": "claude-3-haiku-20240307", "stop_reason": "end_turn", "stop_sequence": null, "usage": { "input_tokens": 568, "output_tokens": 25 }, "type": "message", "role": "assistant" }, "tool_calls": [], "invalid_tool_calls": [], "usage_metadata": { "input_tokens": 568, "output_tokens": 25, "total_tokens": 593 } } ] } ==== User information after run: {}