Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]
32.1k
Stars
4.9k
Forks
1.0k
Issues
1.1k
Contributors
150
Watchers
anthropiclangchainllmllmopsopenaiai-gatewayazure-openaibedrockgatewayopenai-proxyvertex-aillm-gatewaylitellmmcp-gateway
Python
{"name":"Other","spdxId":"NOASSERTION"}
Project Description
A tool to simplify the API calls of large models. This project can unify the interfaces of various AI large models and services into the format of OpenAI, simplifying the work of switching and managing different AI services/large models. In addition, it also supports setting budgets, limiting request frequencies, managing API keys, and configuring OpenAI proxy servers, among other functions.