Multi-turn conversation

LLMs are stateless by default. Every API call is treated independently so they forget previous questions, answers, or follow-ups.

LitAI introduces a simple way to persist conversation history across requests using named sessions. This allows LLMs to maintain memory and context over multiple turns, without any manual prompt management.

Background

LLM chat format

The LLM chat format typically consists of a list of messages, where each message has a role (like "user", "assistant", or "system") and content. This structured format enables multi-turn conversations by preserving context across messages.

LLM memory

LLM memory refers to the model’s ability to retain and use past interactions to inform future responses. While LLMs are stateless by default, LitAI simulate memory by storing a history of messages in a conversation. This allows the model to carry context across multiple turns, enabling more coherent and personalized interactions.

Multi-turn chat with LitAI

Each conversation is identified by a unique conversation name. LitAI automatically stores and manages the message history for each thread.

Some of the use cases of multi-turn conversation are:

• Building assistants that remember previous questions
• Research agents that follow up on earlier answers
• Multi-turn summarizers or chat UX

To start a conversation, pass a unique name using the conversation parameter. All future messages using that name will share context.

from litai import LLM

llm = LLM(model="openai/gpt-4")

llm.chat("What is LitAI?", conversation="product-help")

llm.chat("Does it support multiple models?", conversation="product-help")

Without conversation="product-help", the second question would be ambiguous.

Inspect and reset memory

You can access the full conversation memory to debug or persist messages:

from litai import LLM

llm = LLM(model="openai/gpt-4")
llm.chat("What is LitAI?", conversation="product-help")
llm.chat("Does it support multiple models?", conversation="product-help")

history = llm.get_history("product-help")

for msg in history:
    print(f'{msg["role"]}: {msg["content"]}')

To reset or clear the conversation history (e.g. when the user starts a new topic):

from litai import LLM

llm = LLM(model="openai/gpt-4")
llm.chat("What is LitAI?", conversation="product-help")
llm.chat("Does it support multiple models?", conversation="product-help")

llm.reset_conversation("product-help")


history = llm.get_history("product-help")  # empty

Manage multiple conversations

LitAI keeps each conversation fully isolated. This is useful when a single user runs different tasks simultaneously.

from litai import LLM

llm = LLM(model="openai/gpt-4")

llm.chat("Summarize this article", conversation="summarizer")
llm.chat("What is a vector database?", conversation="ml-help")

print(llm.list_conversations())
# ('summarizer', 'ml-help')