P
PowerInfer
Tiiny-AI/PowerInfer
High-speed Large Language Model Serving for Local Deployment
★9.7kstars
C++
MIT
Updated: Today
📋 Project at a Glance
Tap to expand
What's this?Data & Infrastructure project leveraging large-language-models and llama, built with C++, open-source
Who made it?Maintained by Tiiny-AI team, 9.7K⭐ on GitHub, #207 out of 3133 in Data & Infrastructure
Why does it exist?As the Data & Infrastructure landscape evolved, the Tiiny-AI team identified the need for better large-language-models solutions. PowerInfer was created to simplify llama workflows.
What can it do?Key use cases: llm, llm-inference, local-inference
How to install with AI?Use an AI coding assistant to follow the README and automatically handle the install and environment setup.
🔗 github.com/Tiiny-AI/PowerInfer
🔗 github.com/Tiiny-AI/PowerInfer
Topics
large-language-modelsllamallmllm-inferencelocal-inference