LM Studio

Ollama is an open source tool that makes it easy to run large language models locally on your own hardware. With a single command, developers can download and run models like Llama, Mistral, Gemma, and others entirely offline. The platform also offers a cloud tier for running larger models on datacenter-grade hardware, parallel inference, and real-time web retrieval. Ollama is designed for privacy, your data is never used for training, and it integrates with coding assistants like Claude Code and agent frameworks like OpenClaw.

vLLM is a high-throughput and memory-efficient inference and serving engine for large language models developed by UC Berkeley's Sky Computing Lab with over 2000 contributors. Its PagedAttention algorithm manages attention key and value memory efficiently. Features include state-of-the-art serving throughput, continuous batching, chunked prefill, prefix caching, FlashAttention and FlashInfer kernels, quantization support (FP8, INT8, INT4, GPTQ, AWQ, GGUF), speculative decoding, structured output generation, and OpenAI-compatible API server.

Text Generation Inference (TGI) is a Rust, Python, and gRPC server for text generation inference developed by Hugging Face. It is used in production at Hugging Face to power Hugging Chat, the Inference API, and Inference Endpoints. Key features include Tensor Parallelism via NCCL for multi-GPU acceleration, continuous batching, token streaming via Server-Sent Events, Flash Attention and Paged Attention for optimized inference, and support for quantization methods including bitsandbytes, GPT-Q, EQTQ, AWQ, Marlin, and fp8.