LLMs as Copilots for Theorem Proving in Lean
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
High-speed Large Language Model Serving for Local Deployment
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.
Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk