lmdeploy

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

A high-throughput and memory-efficient inference and serving engine for LLMs