tiny-vllm

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.

A highly optimized LLM inference acceleration engine for Llama and its variants.

Local LLM, image&video&music generator, vibecode like cursor with local models on your phone