---
slug: "llama-cpp"
name: "llama.cpp"
effect: "Runs LLM and VLM inference locally in plain C/C++, with a CLI and an OpenAI-compatible server"
category: "model-api"
url: "https://llama.app"
github: "ggml-org/llama.cpp"
pricing: "open-source"
license: "MIT"
tags: ["local-models", "inference", "c-plus-plus", "gguf", "quantization", "openai-compatible"]
platforms: ["macos", "windows", "linux", "docker"]
install_kind: "cli"
date_added: "2026-10-14"
drop_score: 61
stars: 131189
pushed_at: "2026-10-15T00:31:22Z"
signals_status: "ok"
signals_fetched_at: "2026-10-15T01:19:00.896Z"
page: "https://agenttooldrop.com/tools/llama-cpp"
---

# llama.cpp

Runs LLM and VLM inference locally in plain C/C++, with a CLI and an OpenAI-compatible server

Category: Models and APIs. Pricing: Open source. License: MIT.

Platforms: macos, windows, linux, docker.

## Why you would want it

llama.cpp is a dependency-free C/C++ inference engine that runs on a wide range of hardware: Metal on Apple silicon, CUDA, HIP, Vulkan, SYCL, WebGPU and CPU builds, with CPU+GPU hybrid inference for models larger than your VRAM. It supports 1.5-bit to 8-bit quantization to cut memory use. `llama cli -hf <repo>` downloads and runs a model straight from Hugging Face, and `llama serve` starts an OpenAI-compatible API server with a built-in web UI that agents and editors can call.

## Best for

- Local inference on laptops and mixed hardware
- Quantized models on limited memory
- Self-hosted OpenAI-compatible endpoints

## Use it when

You want direct control over local inference, quantization and hardware backends, or a lightweight OpenAI-compatible server.

## Skip it if

You only use hosted models, or you would rather not manage model files, quantization choices and hardware backends yourself.

## Install

Install with the install script:

```sh
curl -LsSf https://llama.app/install.sh | sh
```

Source: https://github.com/ggml-org/llama.cpp#readme

## Live signals

Fetched 2026-10-15T01:19:00.896Z (status: ok).

- Repository: ggml-org/llama.cpp
- Stars: 131,189
- Forks: 24,411
- Open issues and PRs: 2,562
- License (GitHub): MIT
- Last push: 2026-10-15
- Latest release: v0.6.0 (2026-10-05)

## Links

- Website: https://llama.app
- GitHub: https://github.com/ggml-org/llama.cpp
- Install instructions verified at: https://github.com/ggml-org/llama.cpp#readme
- Source for this entry: https://github.com/ggml-org/llama.cpp#readme
- AgentToolDrop page: https://agenttooldrop.com/tools/llama-cpp
- Full catalog (JSON): https://agenttooldrop.com/tools.json