{ "cells": [ { "cell_type": "markdown", "id": "7240d5b5-9dac-4070-8a9e-2350fb01e0be", "metadata": {}, "source": [ "# How to share state between threads\n", "\n", "By default, state in a graph is scoped to that thread.\n", "LangGraph also allows you to specify a \"scope\" for a given key/value pair that exists between threads. This can be useful for storing information that is shared between threads. For instance, you may want to store information about a user's preferences expressed in one thread, and then use that information in another thread.\n", "\n", "In this notebook we will go through an example of how to construct and use such a graph.\n", "\n", "## Setup\n", "\n", "First, let's install the required packages and set our API keys\n", "\n", "```bash\n", "npm install @langchain/openai @langchain/langgraph @langchain/core zod uuid\n", "```\n", "\n", "Then set your enviroment variables for OpenAI:\n", "\n", "```typescript\n", "process.env.OPENAI_API_KEY = \"your-openai-api-key\";\n", "```" ] }, { "cell_type": "markdown", "id": "51b6817d", "metadata": {}, "source": [ "
\n", "

Set up LangSmith for LangGraph development

\n", "

\n", " Sign up for LangSmith to quickly spot issues and improve the performance of your LangGraph projects. LangSmith lets you use trace data to debug, test, and monitor your LLM apps built with LangGraph — read more about how to get started here. \n", "

\n", "
" ] }, { "cell_type": "markdown", "id": "c4c550b5-1954-496b-8b9d-800361af17dc", "metadata": {}, "source": [ "## Create graph\n", "\n", "In this example we will create a graph that will let us store information about a user's preferences. We will do so by defining a state key that will be scoped to a `user_id`, and allowing the model to populate this field as it deems fit (by providing the model with a tool to save information about the user).\n", "\n", " \n", "
\n", "

Typing shared state keys

\n", "

\n", " Shared state channels (keys) MUST be objects (see info channel in the AgentState example below)\n", "

\n", "
" ] }, { "cell_type": "code", "execution_count": 1, "id": "337ee88f", "metadata": {}, "outputs": [], "source": [ "import { z } from \"zod\";\n", "import {\n", " START,\n", " END,\n", " Annotation,\n", " StateGraph,\n", " MemoryStore,\n", " SharedValue,\n", " MemorySaver,\n", "} from \"@langchain/langgraph\";\n", "import { ChatOpenAI } from \"@langchain/openai\";\n", "import {\n", " type AIMessage,\n", " type BaseMessage,\n", " ToolMessage,\n", "} from \"@langchain/core/messages\";\n", "import { RunnableConfig } from \"@langchain/core/runnables\";\n", "import { v4 as uuidv4 } from \"uuid\";\n", "\n", "const infoSchema = z.object({\n", " fact: z.string().describe(\"The fact about the user\"),\n", " topic: z.string().describe(\"The topic of the fact\"),\n", "});\n", "\n", "const AgentAnnotation = Annotation.Root({\n", " messages: Annotation({\n", " reducer: (a, b) => a.concat(b),\n", " default: () => [],\n", " }),\n", " // IMPORTANT\n", " // This is how you define a shared state value.\n", " // The string passed to `.on` is the key that will\n", " // be used to store the value in the shared state.\n", " info: SharedValue.on(\"user_id\"),\n", "});\n", "\n", "const prompt = `You are helpful assistant.\n", "\n", "Here is what you know about the user:\n", "\n", "\n", "{info}\n", "\n", "\n", "Help out the user. If the user tells you any information about themselves, save the information using the \\`Info\\` tool.\n", "\n", "This means if the user provides any sort of fact about themselves, be it an opinion they have, a fact about themselves, etc. SAVE IT!\n", "`;\n", "\n", "const model = new ChatOpenAI({\n", " model: \"gpt-4o\",\n", " temperature: 0,\n", "}).bindTools([\n", " {\n", " name: \"Info\",\n", " description: \"Save the information provided by the user\",\n", " schema: infoSchema,\n", " },\n", "]);\n", "\n", "const callModel = async (\n", " state: typeof AgentAnnotation.State\n", "): Promise> => {\n", " const facts = Object.values(state.info).map((d) => d.fact);\n", " const info = facts.join(\"\\n\");\n", " const systemMsg = prompt.replace(\"{info}\", info);\n", " const response = await model.invoke([\n", " { role: \"system\", content: systemMsg },\n", " ...state.messages,\n", " ]);\n", " return { messages: [response] };\n", "};\n", "\n", "const route = (state: typeof AgentAnnotation.State): string => {\n", " const lastMessage = state.messages[state.messages.length - 1];\n", " if (!(\"tool_calls\" in lastMessage)) {\n", " throw new Error(\"Expected an AI message with tool calls.\");\n", " }\n", "\n", " return (lastMessage as AIMessage).tool_calls?.length ? \"update_memory\" : END;\n", "};\n", "\n", "const updateMemory = (\n", " state: typeof AgentAnnotation.State\n", "): Partial => {\n", " const toolResponseMessages: ToolMessage[] = [];\n", " const memories: Record> = {};\n", "\n", " const lastMessage = state.messages[state.messages.length - 1];\n", " if (!(\"tool_calls\" in lastMessage)) {\n", " throw new Error(\"Expected an AI message with tool calls.\");\n", " }\n", " const castLastMessage = lastMessage as AIMessage;\n", "\n", " castLastMessage.tool_calls?.forEach((tc) => {\n", " toolResponseMessages.push(\n", " new ToolMessage({\n", " content: \"Saved!\",\n", " tool_call_id: tc.id as string,\n", " })\n", " );\n", " memories[uuidv4()] = {\n", " fact: tc.args.fact,\n", " topic: tc.args.topic,\n", " };\n", " });\n", "\n", " return { messages: toolResponseMessages, info: memories };\n", "};\n", "\n", "const memory = new MemorySaver();\n", "// IMPORTANT\n", "// In order to use shared values, you must initialize a store like this:\n", "const kv = new MemoryStore();\n", "\n", "const graph = new StateGraph(AgentAnnotation)\n", " .addNode(\"call_model\", callModel)\n", " .addNode(\"update_memory\", updateMemory)\n", " .addEdge(\"update_memory\", END)\n", " .addEdge(START, \"call_model\")\n", " .addConditionalEdges(\"call_model\", route);\n", "\n", "const compiledGraph = graph.compile({\n", " checkpointer: memory,\n", " // Then, pass it to `.compile` like this:\n", " store: kv,\n", "});" ] }, { "cell_type": "markdown", "id": "552d4e33-556d-4fa5-8094-2a076bc21529", "metadata": {}, "source": [ "## Run graph on one thread\n", "\n", "We can now run the graph on one thread and give it some information" ] }, { "cell_type": "code", "execution_count": 2, "id": "18bd8679-3a73-4033-bfb4-5093ac1f5d7f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " call_model: {\n", " messages: [\n", " AIMessage {\n", " \"id\": \"chatcmpl-A8VfAQGfZZ4WBbG4zrbwzc34k1Ji3\",\n", " \"content\": \"Hello! How can I assist you today?\",\n", " \"additional_kwargs\": {},\n", " \"response_metadata\": {\n", " \"tokenUsage\": {\n", " \"completionTokens\": 10,\n", " \"promptTokens\": 137,\n", " \"totalTokens\": 147\n", " },\n", " \"finish_reason\": \"stop\",\n", " \"system_fingerprint\": \"fp_a5d11b2ef2\"\n", " },\n", " \"tool_calls\": [],\n", " \"invalid_tool_calls\": [],\n", " \"usage_metadata\": {\n", " \"input_tokens\": 137,\n", " \"output_tokens\": 10,\n", " \"total_tokens\": 147\n", " }\n", " }\n", " ]\n", " }\n", "}\n", "{\n", " call_model: {\n", " messages: [\n", " AIMessage {\n", " \"id\": \"chatcmpl-A8VfAh46qQIvHejVESnvqYwnAIor6\",\n", " \"content\": \"\",\n", " \"additional_kwargs\": {\n", " \"tool_calls\": [\n", " {\n", " \"id\": \"call_IbF0aL78Xep9Xpz3UHLn0POR\",\n", " \"type\": \"function\",\n", " \"function\": \"[Object]\"\n", " }\n", " ]\n", " },\n", " \"response_metadata\": {\n", " \"tokenUsage\": {\n", " \"completionTokens\": 23,\n", " \"promptTokens\": 159,\n", " \"totalTokens\": 182\n", " },\n", " \"finish_reason\": \"tool_calls\",\n", " \"system_fingerprint\": \"fp_25624ae3a5\"\n", " },\n", " \"tool_calls\": [\n", " {\n", " \"name\": \"Info\",\n", " \"args\": {\n", " \"fact\": \"The user likes pepperoni pizza\",\n", " \"topic\": \"Food Preferences\"\n", " },\n", " \"type\": \"tool_call\",\n", " \"id\": \"call_IbF0aL78Xep9Xpz3UHLn0POR\"\n", " }\n", " ],\n", " \"invalid_tool_calls\": [],\n", " \"usage_metadata\": {\n", " \"input_tokens\": 159,\n", " \"output_tokens\": 23,\n", " \"total_tokens\": 182\n", " }\n", " }\n", " ]\n", " }\n", "}\n", "{\n", " update_memory: {\n", " messages: [\n", " ToolMessage {\n", " \"content\": \"Saved!\",\n", " \"additional_kwargs\": {},\n", " \"response_metadata\": {},\n", " \"tool_call_id\": \"call_IbF0aL78Xep9Xpz3UHLn0POR\"\n", " }\n", " ],\n", " info: { '2bf3fed4-028a-4c86-93fe-717cde64e8e7': [Object] }\n", " }\n", "}\n", "{\n", " call_model: {\n", " messages: [\n", " AIMessage {\n", " \"id\": \"chatcmpl-A8VfBbqFPyFfN65f4Tk3k0kdtXHtJ\",\n", " \"content\": \"\",\n", " \"additional_kwargs\": {\n", " \"tool_calls\": [\n", " {\n", " \"id\": \"call_ZkpyxLljOOfhp4oAMyO2R1Lh\",\n", " \"type\": \"function\",\n", " \"function\": \"[Object]\"\n", " }\n", " ]\n", " },\n", " \"response_metadata\": {\n", " \"tokenUsage\": {\n", " \"completionTokens\": 23,\n", " \"promptTokens\": 208,\n", " \"totalTokens\": 231\n", " },\n", " \"finish_reason\": \"tool_calls\",\n", " \"system_fingerprint\": \"fp_a5d11b2ef2\"\n", " },\n", " \"tool_calls\": [\n", " {\n", " \"name\": \"Info\",\n", " \"args\": {\n", " \"fact\": \"The user just moved to San Francisco\",\n", " \"topic\": \"Location\"\n", " },\n", " \"type\": \"tool_call\",\n", " \"id\": \"call_ZkpyxLljOOfhp4oAMyO2R1Lh\"\n", " }\n", " ],\n", " \"invalid_tool_calls\": [],\n", " \"usage_metadata\": {\n", " \"input_tokens\": 208,\n", " \"output_tokens\": 23,\n", " \"total_tokens\": 231\n", " }\n", " }\n", " ]\n", " }\n", "}\n", "{\n", " update_memory: {\n", " messages: [\n", " ToolMessage {\n", " \"content\": \"Saved!\",\n", " \"additional_kwargs\": {},\n", " \"response_metadata\": {},\n", " \"tool_call_id\": \"call_ZkpyxLljOOfhp4oAMyO2R1Lh\"\n", " }\n", " ],\n", " info: { 'ee0085c3-0396-4be8-b0cd-869194aacd5b': [Object] }\n", " }\n", "}\n" ] } ], "source": [ "const config = {\n", " configurable: {\n", " thread_id: \"1\",\n", " // Notice we're specifying `user_id` here, which matches the key name we passed to `SharedValue.on()`\n", " // Without this, our graph wouldn't be able to access the shared state value.\n", " user_id: \"1\"\n", " },\n", " streamMode: \"updates\" as const\n", "};\n", "\n", "// First let's just say hi to the AI\n", "for await (const update of await compiledGraph.stream({\n", " messages: [{ role: \"user\", content: \"hi\" }],\n", "}, config)) {\n", " console.log(update);\n", "}\n", "\n", "// Let's continue the conversation (by passing the same config) and tell the AI we like pepperoni pizza\n", "for await (const update of await compiledGraph.stream({\n", " messages: [{ role: \"user\", content: \"i like pepperoni pizza\" }],\n", "}, config)) {\n", " console.log(update);\n", "}\n", "\n", "// Let's continue the conversation even further (by passing the same config) and tell the AI we live in SF\n", "for await (const update of await compiledGraph.stream({\n", " messages: [{ role: \"user\", content: \"i also just moved to SF\" }],\n", "}, config)) {\n", " console.log(update);\n", "}" ] }, { "cell_type": "markdown", "id": "b8c416fa-086a-491d-a7d3-57091f6413e3", "metadata": {}, "source": [ "## Run graph on a different thread\n", "\n", "We can now run the graph on a different thread and see that it remembers facts about the user (specifically that the user likes pepperoni pizza and lives in SF):" ] }, { "cell_type": "code", "execution_count": 3, "id": "e240f025-ff8b-4d17-beb7-2420c0575dd9", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " call_model: {\n", " messages: [\n", " AIMessage {\n", " \"id\": \"chatcmpl-A8VgEKTv4D2VzGdGwq6FVv09HczZf\",\n", " \"content\": \"Sure! Since you just moved to San Francisco, here are some popular restaurants you might enjoy:\\n\\n1. **Tony's Pizza Napoletana**\\n - **Cuisine:** Italian, Pizza\\n - **Location:** 1570 Stockton St, San Francisco, CA 94133\\n - **Why you might like it:** They have a great selection of pizzas, including pepperoni!\\n\\n2. **House of Prime Rib**\\n - **Cuisine:** American, Steakhouse\\n - **Location:** 1906 Van Ness Ave, San Francisco, CA 94109\\n - **Why you might like it:** If you're in the mood for a hearty meal, their prime rib is highly recommended.\\n\\n3. **Tartine Bakery**\\n - **Cuisine:** Bakery, Cafe\\n - **Location:** 600 Guerrero St, San Francisco, CA 94110\\n - **Why you might like it:** Perfect for a lighter meal or dessert, their pastries are famous.\\n\\n4. **La Taqueria**\\n - **Cuisine:** Mexican\\n - **Location:** 2889 Mission St, San Francisco, CA 94110\\n - **Why you might like it:** Known for their delicious tacos and burritos.\\n\\n5. **Swan Oyster Depot**\\n - **Cuisine:** Seafood\\n - **Location:** 1517 Polk St, San Francisco, CA 94109\\n - **Why you might like it:** Great spot for fresh seafood.\\n\\nWould you like more information on any of these, or do you have a specific type of cuisine in mind?\",\n", " \"additional_kwargs\": {},\n", " \"response_metadata\": {\n", " \"tokenUsage\": {\n", " \"completionTokens\": 324,\n", " \"promptTokens\": 166,\n", " \"totalTokens\": 490\n", " },\n", " \"finish_reason\": \"stop\",\n", " \"system_fingerprint\": \"fp_25624ae3a5\"\n", " },\n", " \"tool_calls\": [],\n", " \"invalid_tool_calls\": [],\n", " \"usage_metadata\": {\n", " \"input_tokens\": 166,\n", " \"output_tokens\": 324,\n", " \"total_tokens\": 490\n", " }\n", " }\n", " ]\n", " }\n", "}\n" ] } ], "source": [ "const config2 = {\n", " configurable: {\n", " // Notice we have a new thread ID, but the same user ID.\n", " // This allows us to access the shared state value.\n", " thread_id: \"2\",\n", " user_id: \"1\"\n", " },\n", " streamMode: \"updates\" as const\n", "};\n", "\n", "for await (const update of await compiledGraph.stream({\n", " messages: [{ role: \"user\", content: \"where and what should i eat for dinner? Can you list some restaurants?\" }],\n", "}, config2)) {\n", " console.log(update);\n", "}" ] }, { "cell_type": "markdown", "id": "091995d3", "metadata": {}, "source": [ "Perfect! The AI recommended restaurants in SF, and included a pizza restaurant at the top of it's list.\n", "\n", "Notice that the `messages` in this new thread do NOT contain the messages from the previous thread since we didn't store them as shared values across the `user_id`. However, the `info` we saved in the previous thread was saved since we passed in the same `user_id` in this new thread.\n", "\n", "Let's now run the graph for another user to verify that the preferences of the first user are self contained:" ] }, { "cell_type": "code", "execution_count": 4, "id": "f9bf2c15", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\n", " call_model: {\n", " messages: [\n", " AIMessage {\n", " \"id\": \"chatcmpl-A8VgwX6MUbLwBdistYNC0LP1t6y7S\",\n", " \"content\": \"Sure, I can help with that! To give you the best recommendations, could you please tell me your location or the city you're in? Additionally, do you have any preferences or dietary restrictions?\",\n", " \"additional_kwargs\": {},\n", " \"response_metadata\": {\n", " \"tokenUsage\": {\n", " \"completionTokens\": 40,\n", " \"promptTokens\": 151,\n", " \"totalTokens\": 191\n", " },\n", " \"finish_reason\": \"stop\",\n", " \"system_fingerprint\": \"fp_a5d11b2ef2\"\n", " },\n", " \"tool_calls\": [],\n", " \"invalid_tool_calls\": [],\n", " \"usage_metadata\": {\n", " \"input_tokens\": 151,\n", " \"output_tokens\": 40,\n", " \"total_tokens\": 191\n", " }\n", " }\n", " ]\n", " }\n", "}\n" ] } ], "source": [ "// Once again, we're specifying a new `user_id` value here.\n", "// Like the previous examples, this means the graph will not\n", "// be able to access the memory saved from the previous run.\n", "const config3 = {\n", " configurable: {\n", " thread_id: \"3\",\n", " user_id: \"2\"\n", " },\n", " streamMode: \"updates\" as const\n", "}\n", "\n", "for await (const update of await compiledGraph.stream({\n", " messages: [{ role: \"user\", content: \"where and what should i eat for dinner? Can you list some restaurants?\" }],\n", "}, config3)) {\n", " console.log(update);\n", "}" ] }, { "cell_type": "markdown", "id": "b7086cea", "metadata": {}, "source": [ "Perfect! The graph has forgotten all of the previous preferences and has to ask the user for it's location and dietary preferences." ] } ], "metadata": { "kernelspec": { "display_name": "TypeScript", "language": "typescript", "name": "tslab" }, "language_info": { "codemirror_mode": { "mode": "typescript", "name": "javascript", "typescript": true }, "file_extension": ".ts", "mimetype": "text/typescript", "name": "typescript", "version": "3.7.2" } }, "nbformat": 4, "nbformat_minor": 5 }