M
Mooncake
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
★6.2kstars
C++
Apache-2.0
Updated: Today
📋 Project at a Glance
Tap to expand
What's this?A disaggregation/inference tool in the Data & Infrastructure category, built with C++, open-source
Who made it?Maintained by kvcache-ai team, 6.2K⭐ on GitHub, #301 out of 3133 in Data & Infrastructure
Why does it exist?The kvcache-ai team recognized that existing disaggregation tools in Data & Infrastructure were hard to use. Mooncake was designed to make inference more accessible.
What can it do?Key use cases: kvcache, llm, rdma
How to install with AI?Use an AI coding assistant to follow the README and automatically handle the install and environment setup.
🔗 github.com/kvcache-ai/Mooncake | 官网 https://kvcache-ai.github.io/Mooncake/
🔗 github.com/kvcache-ai/Mooncake | 官网 https://kvcache-ai.github.io/Mooncake/
Topics
disaggregationinferencekvcachellmrdmareinforcement-learningsglangtokenspeedtrt-llmvllm