- Teams routing agent traffic across multiple model providers
- Developers who want OpenAI-compatible access to many LLM APIs
- Builders adding cost tracking, load balancing, and gateway logs
See repository · Tools
LiteLLM
AI gateway and Python SDK for calling many LLM providers through OpenAI-compatible or native formats.
49K stars 8.5K forks See repository license 2026-06-02 verified
$
pip install litellm Overview
What is LiteLLM?
LiteLLM is a Python SDK and proxy server used by AI builders to route requests across many model providers, track cost, add logging, and manage gateway behavior.
Model serving
LiteLLM surfaces model serving as a core capability in its published project metadata and source links.
This gives readers a starting point for evaluating whether the project fits their workflow before visiting the source repository or docs.Inference
LiteLLM surfaces inference as a core capability in its published project metadata and source links.
This gives readers a starting point for evaluating whether the project fits their workflow before visiting the source repository or docs.Connectors
LiteLLM surfaces connectors as a core capability in its published project metadata and source links.
This gives readers a starting point for evaluating whether the project fits their workflow before visiting the source repository or docs. Install
One command to start
$
pip install litellm Use cases
What teams use it for
Ecosystem
Tags & capabilities
Comparison
How it stacks up
When to choose LiteLLM
Compare it with nearby tools by looking at hosting model, integration surface, license, and whether the official docs show the workflow you need.
FAQ
Questions
What should I check before using LiteLLM?
Connect one low-risk service or local server, then inspect auth scope, logs, schema clarity, and failure behavior.
Is LiteLLM open source?
LiteLLM is listed on OpenAgent.bot with See repository based on the current resource metadata. Re-check the official repository, docs, and license before production use.
Decision brief
JSON
Should you use LiteLLM?
- Teams that only call one model provider directly
- Users who do not want to operate a gateway or proxy
- Verified 2026-06-02
- License: See repository
- Repo: BerriAI/litellm
- Open-source status needs review
Deployment
self hosted, cloud
Permission surface
external services
Decision signals
No extra signals recorded
Agent packet
Structured decision data for LiteLLM
This packet is the compact machine-readable view agents should use before following source links or taking action.
Capabilities
model serving, inference, connectors
Constraints
source available
Deployment
self hosted, cloud
Permission surface
external services
Recommended workflows
Connector or protocol layer
Overview
What LiteLLM does
What it is
LiteLLM is listed on OpenAgent.bot as a tools resource for open AI builders.
Why it matters
Agent applications often need provider routing, fallbacks, cost tracking, and observability before they can be trusted in production. LiteLLM gives teams a practical gateway layer for those needs.
How to evaluate it
Start from the official source links, then validate the project against your deployment needs, license requirements, and maintenance expectations.
Facts
Known metadata and operating surface
These fields are separated from editorial interpretation so agents can reason over facts and missing checks.
Resource type tool
Category Tools
Maturity active
Difficulty Unknown
License See repository
Pricing free
Verified 2026-06-02
Source confidence high
Risk level low
Fit matrix
Where LiteLLM fits in an agent stack
strong
Connector or protocol layer
LiteLLM has multiple signals for connector or protocol layer, including matching tags, capabilities, category, or positioning.
- Connect one low-risk service, then inspect schemas, auth scope, errors, and logs.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
partial
Browser automation
LiteLLM has at least one signal for browser automation, but should be checked against a real task before adoption.
- Run one non-sensitive website task and inspect clicks, waits, retries, and changed URLs.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
partial
Coding agent workflow
LiteLLM has at least one signal for coding agent workflow, but should be checked against a real task before adoption.
- Run a small repository change and inspect the diff, tests, and rollback path.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
partial
Local or private AI stack
LiteLLM has at least one signal for local or private ai stack, but should be checked against a real task before adoption.
- Verify hardware requirements, data path, storage, and whether all calls stay in your environment.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
weak
Evaluation and observability
LiteLLM is not primarily positioned for evaluation and observability in the current metadata.
- Add one repeatable test case and confirm results can run again in review or CI.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
weak
Memory or RAG workflow
LiteLLM is not primarily positioned for memory or rag workflow in the current metadata.
- Create, update, retrieve, correct, and delete memory or retrieval objects with real data.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
Inputs and outputs
What an agent should inspect
Likely inputs
- Repositories, files, issues, terminal output, and test results
- Tool schemas, API requests, service resources, and auth scopes
- Prompts, messages, documents, images, or model inputs
- Official setup instructions and a small real workflow
Likely outputs
- Diffs, commits, explanations, test results, or review notes
- A decision on whether this resource fits the target workflow
Evidence
Sources, claims, and missing checks
Claims are marked separately from source links so future crawlers and reviewers can update them without rewriting the page.
GitHub github
Repository source for code, license, issues, releases, and implementation details.
Homepage homepageOfficial or project-controlled source for this resource profile.
LiteLLM is not currently marked as open source in OpenAgent metadata.
License metadata: See repositoryLiteLLM has a recorded GitHub repository: BerriAI/litellm.
Resource facts and GitHub source link.LiteLLM supports these recorded deployment modes: self hosted, cloud.
OpenAgent decision signal metadata.LiteLLM is tagged with model serving, inference, connectors capabilities.
OpenAgent capability taxonomy.- Dedicated docs link is missing.
- Repository freshness has not been recorded.
Next action
How to start evaluating LiteLLM
Inspect repository
Check license, recent activity, issues, examples, and security-sensitive code paths.
Open sourceOpen Homepage
Start from the official source before adopting third-party instructions.
Open sourceInstall or run
Run only after checking the official source and local environment assumptions.
pip install litellm Compare
Alternatives and nearby resources
Use related resources to compare category fit, license, deployment model, and first-workflow behavior.
FAQ
Common questions about LiteLLM
What is LiteLLM used for?
LiteLLM is used as a tool for tools workflows. The most relevant recorded capabilities are model serving, inference, connectors.
Is LiteLLM open source?
LiteLLM is not currently marked as open source in OpenAgent metadata. Check official links for current licensing.
Can agents use LiteLLM directly?
LiteLLM has recorded interfaces such as repo, docs. Agents should prefer the JSON or Markdown profile first, then follow official docs for real execution.
What should I check before production use?
Check source confidence (high), risk level (low), license, maintenance freshness, permission surface, required credentials, and whether the first workflow succeeds in a sandbox.