Quick start

What is LitAI?

LitAI is an LLM router (OpenAI format) and minimal agent framework. Chat with any model (ChatGPT, Anthropic, etc) in one line with retries, fallbacks, unified billing, and logging. Build agents with tool use in clean, testable Python - no magic, no flaky APIs, no heavy frameworks.

Pay for any model with your Lightning AI credits without additional fees or subscriptions.

Quick start

First install LitAI

pip install litai

To use LitAI, add these lines to your Python program

from litai import LLM

llm = LLM(model="openai/gpt-4")
output = llm.chat("who are you?")
print(output)

# "I'm an AI developed by OpenAI."

Quick tour

Choose any model

Use any of your favorite models from over 20+ LLM providers only changing the model name.

from litai import LLM

llm = LLM(model="openai/gpt-4")
print(llm.chat("who are you?"))
# I'm an AI by OpenAI

llm = LLM(model="google/gemini-2.5-flash")
print(llm.chat("who are you?"))
# I'm an AI by Google

# ...

llm = LLM(model="anthropic/claude-3-5-sonnet-20240620")
print(llm.chat("who are you?"))
# I'm an AI by Anthropic

Streaming

LLMs can generate long content which takes time, LitAI streams the output as soon as they are generated to the client.

from litai import LLM

llm = LLM(model="openai/gpt-4")
for chunk in llm.chat("hello", stream=True):
    print(chunk, end="", flush=True)

Fault tolerance

LitAI automatically retries requests and switches to fallback models. This ensures reliable responses even if a model is unavailable.

from litai import LLM

llm = LLM(
    model="openai/gpt-4",
    fallback_models=["google/gemini-2.5-pro", "anthropic/claude-3-5-sonnet-20240620"],
    max_retries=4,
)

print(llm.chat("How do I fine-tune an LLM?"))