Manage models

In most LLM applications, switching between models or providers involves setting up separate APIs, handling varied parameter formats, and managing cost differences, all of which introduce friction and reduce flexibility. LitAI solves this by unifying access to 20+ models through a simple Python interface. You can choose the best model per task, switch dynamically, control output length, and add fallbacks.

Background

What is a Large Language Model?

A large language model (LLM) is an AI system trained on massive text data to understand and generate human-like language. It can answer questions, write content, generate code, and perform reasoning just by receiving text prompts.

LLM providers

LLM providers are companies or organizations that host and serve large language models through APIs. Popular providers include OpenAI (GPT-4), Anthropic (Claude), Google (Gemini), and Mistral (Mixtral). Each offers different capabilities, pricing, and model strengths. LitAI lets you access all of them through a single interface.

LLM proxy

An LLM proxy acts as a unified interface between your application and multiple underlying LLM providers. It handles routing, authentication, model selection, and fallback logic so you can interact with any model through a single API without changing your code.

LLM text generation with LitAI

LitAI makes it easy to use and manage any language model whether it's from OpenAI, Anthropic, Google, Mistral, or an open-source provider like Hugging Face under a single unified interface.

Supported capabilities

• Dynamically choose a model for each request
• Configure output length (max_tokens)
• Add fallbacks in case of failures
• Switch between models seamlessly at runtime

Select default model

To specify a default model for text generation, pass the model argument during the LLM class initialization.

from litai import LLM

llm = LLM(model="openai/gpt-4")
print(llm.chat("What's the capital of France?"))

Switch models dynamically

LitAI lets us dynamically switch models at request time. We set a default model when initializing LLM and override it with the model parameter only when needed. For example, the default model can be a generic text generation model and later it can be switched to a specific model such as code generation.

from litai import LLM

llm = LLM(model="openai/gpt-4")
print(llm.chat("Who created you?"))
# >> I am a large language model, trained by OpenAI.

print(llm.chat("Who created you?", model="google/gemini-2.5-flash"))
# >> I am a large language model, trained by Google.

print(llm.chat("Who created you?"))
# >> I am a large language model, trained by OpenAI.

Control output length

We can use the max_tokens parameter to limit the length of the generated output. This is useful for:

• Truncating overly verbose responses
• Saving on token usage
• Adhering to provider limits

from litai import LLM

llm = LLM(model="openai/gpt-4")

response = llm.chat("Explain quantum computing to a 5-year-old", max_tokens=50)
print(response)