api-for-open-llm
A unified backend interface for open-source large language models, supporting multiple language models.
- Region
- Domestic
- Pricing
- Free
- Open source
- Yes
- GitHub Stars
- ★ 2.5k
- Source
- GitHub
- Added
- 2026-06-07
- Last verified
- 2026-06-07
Overview
api-for-open-llm is an open-source unified backend interface for large language models, supporting various models such as LLaMA, BLOOM, etc. It simplifies the process of calling different open-source LLMs by providing compatibility with the OpenAI ChatGPT API. Users can simply modify environment variables to use different open-source models as substitutes for ChatGPT, making it ideal for applications requiring rapid integration and deployment of large models.
Key features
- ▪OpenAI-style API compatibility
- ▪Supports streaming responses
- ▪Implements text embedding models
- ▪Supports LangChain tools
- ▪Easy model switching by modifying environment variables
Use cases
Pros
- ＋Easy to integrate
- ＋Supports multiple models
- ＋Efficient streaming responses
Limitations / notes
- －Requires environment variable configuration
- －Depends on specific versions of Python and PyTorch
Who it's for
This overview was compiled by AI from public sources and may contain inaccuracies — please refer to the official site.
FAQ
How do I start the model?
Please refer to the 'Startup Methods' section in the project documentation for detailed steps.
Which models are supported?
Supports LLaMA, BLOOM, Baichuan, and other open-source large language models.
Something wrong? Let us know on the About page and we'll fix it.