---
slug: "deepeval"
name: "DeepEval"
effect: "Unit tests LLM apps and agents in a Pytest-style workflow with LLM-as-a-judge and other metrics"
category: "eval-observability"
tier: "rising"
url: "https://deepeval.com"
github: "confident-ai/deepeval"
pricing: "open-source"
license: "Apache-2.0"
tags: ["evals", "python", "pytest", "testing", "rag", "agents"]
platforms: []
install_kind: "pip"
date_added: "2026-10-14"
rank: 37
rising_score: 26
stars: 18875
star_velocity: 16.2
velocity_basis: "since creation"
created_at: "2023-08-10T05:35:04Z"
pushed_at: "2026-10-15T03:22:22Z"
signals_status: "ok"
signals_fetched_at: "2026-10-15T05:02:40.202Z"
page: "https://agenttooldrop.com/tools/deepeval"
---

# DeepEval

Unit tests LLM apps and agents in a Pytest-style workflow with LLM-as-a-judge and other metrics

Category: Evals and observability. Tier: new and rising, rank 37 by new and rising score (26 of 100). Pricing: Open source. License: Apache-2.0.

## Why you would want it

DeepEval works like Pytest for LLM apps: you write test cases and assert them against metrics such as G-Eval, task completion, tool correctness, answer relevancy, faithfulness and hallucination, then run them with `deepeval test run`. Metrics cover agents, RAG, multi-turn chat, MCP use and multimodal output, can be powered by any LLM, and support both end-to-end and component-level evaluation. It also generates synthetic datasets, runs benchmarks like MMLU and GSM8K, and integrates with LangChain, LangGraph, CrewAI, OpenAI Agents, Pydantic AI and others.

## Best for

- Python teams that want evals in their test suite
- RAG and agent metrics
- CI checks on LLM output quality

## Use it when

You want LLM quality checks written as tests that run locally or in any CI environment.

## Skip it if

You are not working in Python (it requires Python 3.9+). Shared reports, iteration comparisons and production monitoring live in the separate Confident AI platform, which needs an account.

## Install

Install with pip:

```sh
pip install -U deepeval
```

Source: https://github.com/confident-ai/deepeval#readme

## Live signals

Fetched 2026-10-15T05:02:40.202Z (status: ok).

- Repository: confident-ai/deepeval
- Repository created: 2023-08-10
- Stars: 18,875 (GitHub's count; it cannot be authenticated)
- Stars per day, since repo creation: 16.2 (an average over the repo's life, not a growth rate)
- Forks: 2,072
- Open issues and PRs: 728
- License (GitHub): Apache-2.0
- Last push: 2026-10-15
- Latest release: python-v4.2.4 (2026-09-22)

## Links

- Website: https://deepeval.com
- GitHub: https://github.com/confident-ai/deepeval
- Install instructions verified at: https://github.com/confident-ai/deepeval#readme
- Source for this entry: https://github.com/confident-ai/deepeval#readme
- AgentToolDrop page: https://agenttooldrop.com/tools/deepeval
- Full catalog (JSON): https://agenttooldrop.com/tools.json