9 models and APIs

Why you'd want it

Ollama downloads and runs open models such as Gemma with one command (ollama run gemma4) and serves them on a local REST API at localhost:11434. It supports a subset of the OpenAI API, so OpenAI clients work by changing the base URL, and it ships official Python and JavaScript libraries. ollama launch connects local models to coding agents including Claude Code, Codex, Copilot CLI and OpenCode, and the same API can reach Ollama's cloud models with an API key.

Best for

• Running open models on your own machine
• Offline or private agent development
• Pointing coding agents at local models

Use it when

You want a local model server that existing OpenAI-style clients and coding agents can talk to with minimal setup.

Why you'd want it

LiteLLM gives one interface to OpenAI, Anthropic, Gemini, Bedrock, Azure and many other providers using the OpenAI request format, so you can swap providers without rewriting code. The Python SDK includes a Router with retry and fallback logic across deployments. The proxy server adds virtual keys, per-project spend tracking, guardrails, load balancing and an admin dashboard, and it can also act as an MCP gateway and an A2A agent gateway.

Best for

• Platform teams centralizing LLM access
• Multi-provider apps with fallbacks
• Tracking spend per project or key

Use it when

You call several model providers and want one OpenAI-format API, or a central gateway with keys, budgets and logging for a team.

Why you'd want it

The Claude API at api.anthropic.com covers the Messages API, Message Batches (asynchronous, at a 50% cost reduction), token counting, a Files API and a Skills API, with beta Agents, Sessions and Environments APIs for running managed agents in cloud sandboxes. The official SDKs handle auth headers, retries, streaming and typed requests, and the Python SDK adds tool helpers and a tool runner. Claude is also available through Amazon Bedrock, Google Cloud, Claude Platform on AWS and Microsoft Foundry if you need cloud billing and IAM.

Best for

• Building agents on Claude models
• Direct access to the latest Claude features
• Large asynchronous batch jobs

Use it when

You are building on Claude and want direct access to the latest models and features with Anthropic billing and support.

Why you'd want it

llama.cpp is a dependency-free C/C++ inference engine that runs on a wide range of hardware: Metal on Apple silicon, CUDA, HIP, Vulkan, SYCL, WebGPU and CPU builds, with CPU+GPU hybrid inference for models larger than your VRAM. It supports 1.5-bit to 8-bit quantization to cut memory use. llama cli -hf <repo> downloads and runs a model straight from Hugging Face, and llama serve starts an OpenAI-compatible API server with a built-in web UI that agents and editors can call.

Best for

• Local inference on laptops and mixed hardware
• Quantized models on limited memory
• Self-hosted OpenAI-compatible endpoints

Use it when

You want direct control over local inference, quantization and hardware backends, or a lightweight OpenAI-compatible server.

Why you'd want it

vLLM is an inference and serving engine built around PagedAttention, continuous batching, chunked prefill and prefix caching, with many quantization formats and speculative decoding. It runs an OpenAI-compatible API server plus Anthropic Messages API and gRPC support, with tool calling and reasoning parsers and structured outputs via xgrammar or guidance. Tensor, pipeline, data and expert parallelism cover distributed inference, and it supports 200+ Hugging Face model architectures on NVIDIA, AMD and Intel GPUs, CPUs and hardware plugins.

Best for

• Self-hosting open models on GPUs
• Serving many concurrent agent requests
• Distributed multi-GPU inference

Use it when

You need to serve open-weight models to agents or apps at scale on your own GPU infrastructure.

Why you'd want it

DSPy replaces hand-written prompts with Python code: signatures declare typed inputs and outputs, and modules such as Predict, ChainOfThought and ReAct control how a signature runs, including tool use. You compose modules with plain Python control flow into pipelines, RAG systems or agent loops. Optimizers such as GEPA, MIPROv2 and BootstrapFewShot take examples and a scoring function and tune the program's prompts and weights, and the result can be saved and reloaded.

Best for

• Multi-step LLM pipelines
• Teams with labeled examples and a metric
• Replacing brittle prompt templates

Use it when

You want LLM behavior defined in code and improved systematically against a metric rather than by editing prompts by hand.

Why you'd want it

Instructor lets you pass a Pydantic model as response_model and get back a typed, validated object instead of parsing JSON or tool calls yourself. instructor.from_provider uses the same API for OpenAI, Anthropic, Google, Ollama, Groq and other providers. Failed validations are retried with the error message, partial objects can be streamed as they are generated, and nested models work without extra code. Ports exist for TypeScript, Ruby, Go and Elixir.

Best for

• Structured extraction from text
• Pydantic-based Python apps
• Typed outputs across providers

Use it when

You need reliable structured data from an LLM call and want validation and retries handled for you.

Why you'd want it

LLM is Simon Willison's CLI tool and Python library for OpenAI, Claude, Gemini and dozens of other models, via remote APIs or models installed on your own machine. Plugins such as llm-anthropic, llm-gemini and llm-ollama add providers, and llm openai endpoint targets any OpenAI-compatible Chat Completions server without configuration. Prompts and responses are stored in SQLite, and it also generates embeddings, extracts structured content with schemas and lets models execute tools.

Best for

• Scripting LLM calls in the shell
• Comparing models from one CLI
• Keeping a searchable log of prompts

Use it when

You want quick model access from the terminal or a small Python library, with every prompt and response logged locally.

Why you'd want it

OpenRouter exposes hundreds of models behind a single /api/v1/chat/completions endpoint, handles fallbacks automatically and picks the most cost-effective provider for each request. Existing OpenAI SDK code works by pointing the base URL at OpenRouter, and there are also typed client SDKs for TypeScript and Python plus an Agent SDK. It passes through provider pricing without markup, charges a fee when you buy credits, and supports bringing your own provider keys.

Best for

• Switching between models without code changes
• Comparing models from many providers
• Apps that need provider fallbacks

Use it when

You want one API key and one endpoint for models from many providers, with routing and fallbacks handled for you.