Conversational Memory & State
Raw LLM APIs are completely stateless—they retain zero memory between successive HTTP requests. To deliver natural multi-turn conversations, applications must track dialogue history across user sessions. Learn how to implement modern stateful memory using RunnableWithMessageHistory and persistent storage.
1. Modern Memory with RunnableWithMessageHistory
In legacy LangChain, memory was managed using wrappers like ConversationBufferMemory. Modern LangChain decouples the conversational chain from the storage backend usingRunnableWithMessageHistory:
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.output_parsers import StrOutputParser
from langchain_core.chat_history import InMemoryChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
# 1. Base prompt with placeholder for conversation history
prompt = ChatPromptTemplate.from_messages([
("system", "You are a personalized technical mentor. Keep track of user background."),
MessagesPlaceholder(variable_name="history"),
("human", "{question}")
])
model = ChatOpenAI(model="gpt-4o-mini", temperature=0.3)
base_chain = prompt | model | StrOutputParser()
# 2. In-memory dictionary storing session histories by session_id
session_store = {}
def get_session_history(session_id: str):
if session_id not in session_store:
session_store[session_id] = InMemoryChatMessageHistory()
return session_store[session_id]
# 3. Wrap chain with automated history tracking
stateful_chain = RunnableWithMessageHistory(
base_chain,
get_session_history,
input_messages_key="question",
history_messages_key="history"
)
# 4. First turn for User "alice_123"
r1 = stateful_chain.invoke(
{"question": "My name is Alice and I build low-latency C++ trading systems."},
config={"configurable": {"session_id": "alice_123"}}
)
print("Turn 1:", r1)
# 5. Second turn referencing previous context
r2 = stateful_chain.invoke(
{"question": "What language do I specialize in?"},
config={"configurable": {"session_id": "alice_123"}}
)
print("Turn 2:", r2) # Correctly identifies C++ and trading systems!2. Managing Context Windows with trim_messages
As conversations grow to hundreds of turns, feeding the entire history into every prompt wastes money and eventually exceeds context window limits. LangChain provides trim_messages to bound context:
from langchain_core.messages import trim_messages
# Keep at most the last 1,000 tokens of conversation
trimmer = trim_messages(
max_tokens=1000,
strategy="last",
token_counter=ChatOpenAI(model="gpt-4o-mini"),
include_system=True, # Always preserve the initial system prompt!
allow_partial=False,
start_on="human"
)
# Pipe trimmer directly into the model in your LCEL chain
bounded_chain = (
{"messages": trimmer}
| model
| StrOutputParser()
)3. Enterprise Persistence with Redis
In distributed cloud environments with load balancers and container restarts, storing chat history in server RAM causes immediate memory loss. Use RedisChatMessageHistory for persistent caching:
# Install: pip install langchain-community redis
from langchain_community.chat_message_histories import RedisChatMessageHistory
def get_redis_history(session_id: str):
return RedisChatMessageHistory(
session_id=session_id,
url="redis://localhost:6379/0",
ttl=86400 # Automatically expire chat after 24 hours of inactivity
)Explore Related Tools
Hand-picked utilities and calculators related to this tool.
