← Back to List
⚠
Notice:This resource is provided by a third-party author. Please review the code with AI tools or manually before use to ensure security and compatibility.
C++Tiiny-AI/PowerInfer

PowerInfer

High-speed Large Language Model Serving for Local Deployment

53.6/100
★ 9.8KForks: 607
View on GitHub →
Loading report...

Similar Projects

lemonade

85

Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

C++★ 5.8K

LLM-Hub

78

Local LLM, image&video&music generator, vibecode like cursor with local models on your phone

C++★ 604

deeplake

72

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

C++★ 9.2K

cactus

83

Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.

C++★ 6.1K
← Back to List