gpustack

A high-throughput and memory-efficient inference and serving engine for LLMs

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qw

Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, a

SGLang is a high-performance serving framework for large language models and mul