AgentToolDrop

2 of 43

Reef

Turns feedback on an agent's live requests into versioned updates to its model weights or its harness

Install from PyPI with uv

uv venv && source .venv/bin/activate
uv pip install reef-infra
python3 -c "import reef; print(reef.__version__)"

Copied from the project's docs at github.com/Human-Agent-Society/reef

Why you'd want it

Reef sits between an agent and its model as an OpenAI- and Anthropic-compatible inference endpoint, records each interaction and returns a receipt that a later report can score or comment on. Recipes turn eligible feedback into updates, either training model weights with Slime and SGLang on a GPU stack or refining the harness, meaning prompts, rules and skills, through a model API with no local GPUs. Accepted updates go into a version history and are served without restarting Reef. The built-in Reefine recipe refines a coding harness from plain-language asks, such as reproducing a bug with a failing test first.

Best for

  • Agents that should learn from user feedback
  • Harness optimization without GPUs
  • Continual learning research

Use it when

You run an agent with a steady stream of tasks or feedback and want that signal to produce evaluated, versioned improvements instead of manual prompt edits.

Skip it if

You want a stable agent today: harness work needs a model endpoint, representative tasks and an evaluator, weight training needs a supported GPU stack, and it is at v0.1.3 with the repo created on 2026-08-31. The harness install pipes a script into bash and the pi adapter runs commands with no sandbox, so keep the install root outside the project the agent works in.

Editor's note

Benchmark results in the recipe pages are the project's own; we do not repeat them.

Related tools

Same category or shared tags.

  1. Rank 4

    OpenBot

    Framework

    Runs self-hosted AI coworkers with their own browser, files and shell, each action policy-checked and logged

    • score 68
    • 6.5k stars
    • 110 stars/day since creation
    • pushed
    • v0.2.0 on
    • MIT

    Run the published Docker image

    docker run -p 3001:3001 --env-file .env \
  -e EMBEDDED_POSTGRES=on -v openbot-data:/var/lib/postgresql \
  ghcr.io/copilotkit/openbot:latest
    Use it when, skip it if, stats for OpenBot

    Why you'd want it

    OpenBot is a template for an in-house agent platform: Docker Compose brings up the app, PostgreSQL and the Bots, and you supply the model key. Each Bot gets its own container with a Chromium profile, a workspace volume and a shell, and every browser, file, shell, MCP or component action passes through one gateway that evaluates CEL policy rules, writes an audit row and only then acts. Any agent that speaks the AG-UI protocol, built with LangGraph, Mastra, CrewAI, Pydantic AI, Google ADK or by hand, can join as a coworker with its own channel, and you can watch its screen and take over. It ships with General Assistant and Knowledge coworkers plus example roles to copy.

    Best for

    • Self-hosted agent coworkers for a team
    • Auditable browser and shell actions
    • Bringing AG-UI agents into one workspace

    Use it when

    You want to give agents real computers inside your own infrastructure but need allow and deny rules and an audit trail for what they do.

    Skip it if

    You want a finished product: the README calls it an alpha template to clone and change, with no hosted version. It needs Docker, Bun, a model key and a CopilotKit Intelligence project (a free plan exists, or you can self-host it), and Bots drive a browser with saved logins and run shell commands, so set up sign-in before anyone else can reach it.

    Stats

    Stars
    6,491
    Last push
    Release
    v0.2.0,
    License
    MIT
    Pricing
    Open source
    Works with
    Docker, Web
    Stars per day, since repo creation
    110
    New and rising score
    68 of 100

    Full stat block for OpenBot

  2. Rank 13

    Mastra

    Framework

    Builds TypeScript agents and graph-based workflows with memory, RAG, evals and model routing

    • score 48
    • 28.8k stars
    • 36.1 stars/day since creation
    • pushed
    • @mastra/[email protected] on
    • Apache-2.0

    Create a project with npm

    npm create mastra@latest
    Use it when, skip it if, stats for Mastra

    Why you'd want it

    Mastra is a TypeScript framework for agents and workflows that plugs into React, Next.js and Node apps or runs as a standalone server. It connects to 40+ model providers through one interface, has a graph-based workflow engine with .then(), .branch() and .parallel(), and can suspend an agent or workflow for human input and resume from stored state. It also covers conversation history, RAG, observational memory, MCP server authoring, evals and observability.

    Best for

    • TypeScript and Next.js teams
    • Agents with human approval steps
    • Apps that need memory and RAG in one stack

    Use it when

    You want agents, workflows, memory and evals in one TypeScript framework instead of assembling separate libraries.

    Skip it if

    You work in Python. Features in ee/ directories (for example auth) need a Mastra enterprise license for production use.

    Stats

    Stars
    28,827
    Last push
    Release
    @mastra/[email protected],
    License
    Apache-2.0
    Pricing
    Open source
    Stars per day, since repo creation
    36.1
    New and rising score
    48 of 100

    Full stat block for Mastra

  3. Agno

    Framework Standard

    Builds Python agents, teams and workflows and serves them as an API with the AgentOS runtime

    • 42.7k stars
    • 26.3 stars/day since creation
    • pushed
    • v3.1.3 on
    • Apache-2.0

    Install with pip

    pip install -U agno openai
    Use it when, skip it if, stats for Agno

    Why you'd want it

    Agno is a Python SDK with three primitives (agents, teams and workflows) plus a runtime, AgentOS, that serves them through a production API with 50+ endpoints over SSE and websockets. Sessions, memory, knowledge and traces are stored in your own database, and the runtime adds human approval, OpenTelemetry tracing, JWT-based RBAC with multi-tenant isolation, cron scheduling, and interfaces for Slack, Telegram, WhatsApp, Discord, AG-UI and A2A. Starter templates deploy it with Docker to Railway, AWS, GCP, Azure, Fly, Render, Modal or Helm.

    Best for

    • Self-hosted agent platforms
    • Serving agents to many users
    • Agents exposed in Slack or messaging apps

    Use it when

    You want to build agents and also run them as a multi-user service on your own infrastructure, with data kept in your own database.

    Skip it if

    You are not on Python. Agno sends a telemetry event per agent run by default (prompts, messages and outputs are not sent); set AGNO_TELEMETRY=false to turn it off.

    Stats

    Stars
    42,736
    Last push
    Release
    v3.1.3,
    License
    Apache-2.0
    Pricing
    Open source
    Stars per day, since repo creation
    26.3
    Tier
    Standard, not ranked

    Full stat block for Agno

For agents

The same entry, as files an agent can read without scraping this page. 

# read one entry
curl -s https://agenttooldrop.com/tools/reef.md

# list the MCP servers in the catalog
curl -s https://agenttooldrop.com/tools.json \
  | jq -r '.tools[] | select(.category=="mcp-server") | .slug'