OpenLLM

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

JetStream is a throughput and memory optimized engine for LLM inference on XLA d

A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-

The easiest way to serve AI apps and models - Build Model Inference APIs, Job qu