OpenLLM

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

A high-throughput and memory-efficient inference and serving engine for LLMs

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.