petals

Petals is a distributed large language model runtime platform that supports efficient local or cloud-based inference and fine-tuning. It accelerates large language model inference and fine-tuning through a BitTorrent-like distributed network, achieving up to 10x speed improvement. Users can connect to the distributed network via simple Python code and leverage various pre-trained models for text generation or fine-tuning tasks.