vllm-mlx
An OpenAI and Anthropic compatible inference server for Apple Silicon, supporting multimodal models.
- Type
- MCP
- Transport
- http
- Open source
- Yes
- GitHub Stars
- ★ 1.4k
- Source
- mcp-github
- Repository
- github.com/waybarrios/vllm-mlx
Overview
vllm-mlx is an inference server designed for Apple Silicon, enabling the execution of LLMs and vision-language models (such as Llama, Qwen-VL, LLaVA). It offers continuous batching, MCP tool calling, and multimodal support. Built natively on the MLX backend, it achieves throughput of over 400 tok/s and is compatible with Claude Code. This capability allows AI models to run efficiently on Apple Silicon across various use cases requiring high-performance inference.
Capabilities
- ▪OpenAI and Anthropic API compatibility
- ▪Continuous batching
- ▪Multimodal support
- ▪High-throughput inference
- ▪MCP tool calling
Use cases
Setup
pip install vllm-mlx
This information was compiled by AI from public sources and may contain inaccuracies — please refer to the source.
FAQ
vllm-mlx supports which hardware?
Only Apple Silicon (M1, M2, M3, M4, M5).
How do I start the vllm-mlx server?
Use the command `vllm-mlx serve <model> --port 8000`.
Related skills
gemini-cli
Gemini CLI is an open-source AI agent that brings Gemini's powerful capabilities directly into the terminal.
private-gpt
Provides a complete API layer for local models, supporting RAG, skills, tools, and more.
planning-with-files
File-based persistent planning skill, ideal for AI coding agents and long-running tasks.
scientific-agent-skills
Transform any AI agent into a scientific assistant with 147 ready-to-use research skills.
susi_alexa_skill
A skill that enables question-and-answer interactions between Alexa and Susi AI.
claude-context
Provides code search capabilities for Claude Code, turning the entire codebase into context.