LMCache

A high-throughput and memory-efficient inference and serving engine for LLMs

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and o

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qw

Build your own high performance LLM inference engine in C++ and CUDA - a smaller