R
RWKV-LM
BlinkDL/RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
★14.6kstars
Python
Apache-2.0
Updated: Today
📋 Project at a Glance
Tap to expand
What's this?A attention-mechanism/chatgpt tool in the Models category, built with Python, open-source
Who made it?Maintained by BlinkDL team, 14.6K⭐ on GitHub, #94 out of 3201 in Models
Why does it exist?As the Models landscape evolved, the BlinkDL team identified the need for better attention-mechanism solutions. RWKV-LM was created to simplify chatgpt workflows.
What can it do?Key use cases: deep-learning, gpt, gpt-2
How to install with AI?Use an AI coding assistant (Claude Code, Cursor, Copilot) to automatically set up pip dependencies and virtual env. Follow the README — the AI handles the rest.
🔗 github.com/BlinkDL/RWKV-LM
🔗 github.com/BlinkDL/RWKV-LM
Topics
attention-mechanismchatgptdeep-learninggptgpt-2gpt-3language-modellinear-attentionlstmpytorchrnnrwkvtransformertransformers