← Back to List
⚠
Notice:This resource is provided by a third-party author. Please review the code with AI tools or manually before use to ensure security and compatibility.
C++foldl/chatllm.cpp

chatllm.cpp

Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)

76.1/100
★ 924Forks: 74
View on GitHub →
Loading report...

Similar Projects

PowerInfer

55

High-speed Large Language Model Serving for Local Deployment

C++★ 9.8K

cactus

83

Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.

C++★ 6.0K

lemonade

85

Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

C++★ 5.7K

distributed-llama

69

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

C++★ 3.0K
← Back to List