LiteLLM

Sitting between your application and the model providers, LiteLLM is an open source LLM gateway that lets you call more than 100 models using the OpenAI format.

Providers named include OpenAI, Azure, Cohere, Anthropic, Replicate and Google, so switching models does not mean rewriting your calls.

LiteLLM returns consistent outputs and exceptions across all of them, which is the main reason to use a gateway like this.

The proxy side handles the operational work: authentication, load balancing, spend tracking, budgets, rate limiting and model access controls.

You also get logging and error tracking for every model, LLM observability, S3 logging, prompt management, guardrails, a batches API and pass-through endpoints.

Teams running several providers at once and needing one place to watch costs will get the most from LiteLLM.