large language model (LLM) inference Serving engines

Large Language Models (LLMs)

vLLM ollama TensorRT-LLM Hugging Face TGI (Text Generation Inference) SGLang LMDeploy MLC-LLM

Ray Serve