LangMem

LangMem gives your chat bot long-term memory so it can personalize its interactions to each user and environment.

1. Install

pip install -U --quiet langmem anthropic 

export LANGMEM_API_URL=https://your-instance-url
export LANGMEM_API_KEY=your-api-key

2. Save Conversations

LangMem will extract memories for your bot in the background.

import uuid

from langmem import AsyncClient

client = AsyncClient()
user_id = str(uuid.uuid4())
thread_id = str(uuid.uuid4())

messages = [
    {
        "role": "user",
        "content": "Hi, I love playing basketball!",
        "metadata": {"user_id": user_id},
    },
    {
        "role": "assistant",
        "content": "That's great! Basketball is a fun sport. Do you have a favorite player?",
    },
    {
        "role": "user",
        "content": "Yeah, Steph Curry is amazing!",
        "metadata": {"user_id": user_id},
    },
]

await client.add_messages(thread_id=thread_id, messages=messages)
await client.trigger_all_for_thread(thread_id=thread_id)

3. Remember

Query relevant memories and format them in your prompt

import anthropic
from langsmith import traceable

anthropic_client = anthropic.AsyncAnthropic()


@traceable(name="Claude", run_type="llm")
async def completion(messages: list, model: str = "claude-3-haiku-20240307"):
    system_prompt = messages[0]["content"]
    msgs = []
    for m in messages[1:]:
        msgs.append({k: v for k, v in m.items() if k != "metadata"})
    response = await anthropic_client.messages.create(
        model=model,
        system=system_prompt,
        max_tokens=1024,
        messages=msgs,
    )
    return response


async def completion_with_memory(messages, user_id):
    memories = await client.query_user_memory(
        user_id=user_id,
        text=messages[-1]["content"],
    )
    facts = "\n".join([mem["text"] for mem in memories["memories"]])

    system_prompt = {
        "role": "system",
        "content": "Here are some things you know" f" about the user:\n\n{facts}",
    }

    return await completion([system_prompt] + messages)


new_messages = [
    {
        "role": "user",
        "content": "Do you remember who my favorite basketball player is?",
        "metadata": {"user_id": user_id},
    }
]

response = await completion_with_memory(new_messages, user_id=user_id)
print(response.content[0].text)

Note

LangMem is still in early alpha, so expect improvements and some breaking changes! Your feedback is important to us!

Quick Start

For a coding walkthrough of the other core functionality, check out the LangMem Walkthrough Notebook.

Concepts

At a high level, LangMem lets you track conversational threads between users and your chat bot. Each thread has a list of message's containing the content, role, and optional user information. As the app developer, you can configure different memory types depending on your needs.

For more details, see the Threads page.

Memory Types

LangMem currently supports 3 types of user memories (exposed as memory_function's):

• User State: a structured profile the memory service maintains for each user.
• Semantic Memory: This is an unstructured memory that generates knowledge triplets from conversations. It can be queried based on relevance, importance, and recency.
• Append-only State: This is a hybrid of the above two memory types that lets you customize the memory schema while still retrieving based on relevance and recency.

You can also track thread-level memories via the thread_summary memory type. This is useful if you want to include summaries of the last N conversations in your system prompt.

For more details on memory types and when to use each one, see the Memory Types page.

Reference

• Client: Documentation for the Client and AsyncClient objects in the client.py file.

Thanks!

Thanks for your feedback! Email your questions and requests to will@langchain.dev.