lorax
Multi-LoRA inference server supporting efficient serving of thousands of fine-tuned models.
- Region
- Overseas
- Pricing
- Free
- Open source
- Yes
- GitHub Stars
- ★ 3.8k
- Source
- GitHub
- Added
- 2026-06-06
- Last verified
- 2026-06-06
Overview
LoRAX is a multi-LoRA inference server that can serve thousands of fine-tuned models on a single GPU, significantly reducing service costs without sacrificing throughput or latency. It achieves high efficiency through techniques like dynamic adapter loading and heterogeneous continuous batching. Ideal for scenarios requiring large-scale deployment of fine-tuned language models.
Key features
- ▪Supports thousands of fine-tuned models
- ▪Dynamic adapter loading
- ▪Heterogeneous continuous batching
- ▪Optimized inference performance
- ▪Production-ready
Use cases
Pros
- ＋High throughput and low latency
- ＋Supports multiple large language models
- ＋Open-source and free for commercial use
Limitations / notes
- －Requires some technical expertise
- －Depends on specific hardware
Who it's for
This overview was compiled by AI from public sources and may contain inaccuracies — please refer to the official site.
FAQ
Which large language models does LoRAX support?
Supports Llama, Mistral, Qwen, and other large language models.
How do I get started with LoRAX?
After installing dependencies, start the LoRAX server and make requests via REST API or Python client.
Something wrong? Let us know on the About page and we'll fix it.