Niches · Agent observability & evals

Run the trace store yourself and keep it healthy

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

0companies, full fit
8companies, fit with a limit
22companies carry the tag
10share of the category's work
11first-hand accounts

What counts as done

The stack runs from the documented compose or Helm files, with disk growth and retention controls documented.

Measured, for our own tests: it ingests at the team's volume, and after 30 days the UI still loads a trace list in under 5 seconds. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: self-host it with docker compose, keep traces in our region, disk is full after three months, upgrade broke the UI

Does the job, with a limit 8

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Arize Phoenix BB
Arize AI, Inc.
Docker, Compose and Helm with SQLite or Postgres; PHOENIX_DEFAULT_RETENTION_POLICY_DAYS and per-project retention policies since 9.0, infinite by default. Before that a user hit 44 GB and an unusable UI (issue 8420). source
Helicone AI Gateway + MCP E
Helicone, Inc.
An all-in-one Docker image and Kubernetes guide exist, with S3, Better Auth and the ClickHouse and Postgres stack from the listing. No retention or disk-growth setting appears on the Docker page. source
Laminar API + MCP C
LMNR AI, Inc.
Compose runs frontend, app server, Postgres, ClickHouse and Quickwit; Helm adds RabbitMQ, Redis and a consumer. Signals and alerts need a licence key, no retention setting is documented, and ClickHouse system logs grew without bound on a small instance (issue 2176). source
Langfuse API + MCP BB
Langfuse GmbH (a ClickHouse, Inc. company)
Compose and Helm with Postgres, ClickHouse, Redis and S3; retention is per project with nightly deletes and data is kept indefinitely by default. One user grew a terabyte ClickHouse on a laptop and another lost the UI after an image upgrade (issue 12172). source
LangWatch B
Reasoning Engine B.V. (doing business as LangWatch)
Apache 2.0 self-hosting with Docker or Helm is claimed and retention is configurable per organisation, team or project on paid and self-hosted plans, with 49 days on Free. The component list and disk guidance were not in the pages read. source
MLflow Tracing C
MLflow Project, a Series of LF Projects, LLC
SQLite by default, a Compose bundle with Postgres and RustFS, and a Helm chart; trace archival moves old span payloads to artifact storage under a retention policy. Authentication is off unless basic-auth is enabled. An HN user calls it well established. source
Opik API + MCP BB
Comet ML, Inc.
Local Compose or Kubernetes with ClickHouse, Zookeeper, MySQL, Redis and MinIO; a tiered-storage TTL moves trace data to cold S3 after a set number of weeks. The open-source build has no authentication. source
Overmind C
Overmind Technology Ltd (England and Wales, company 16585996)
Compose with six Celery worker profiles under AGPL, but startup validation requires Modal, OpenRouter, AWS checkpoint and sandbox credentials even to run; no retention or disk guidance. source

Doesn't do this job 14

Listings in the category that do something else, with the reason.

ListingWhy
Agent Tree C
No legal entity found. The LICENSE, plugin.json and marketplace.json name "Mike Pollow, Digitalroots"
Nothing to host. source
agents-tree B
No legal entity found. The LICENSE and pyproject.toml name Karel Blavka
Nothing to host. source
BotGauge E
BotGauge AI Inc. (Delaware, United States)
Hosted only; no self-hosting pages. source
Braintrust API + MCP B
Braintrust Data, Inc.
Only SaaS or BYOC, and BYOC needs the Enterprise plan with Braintrust operating the data plane; there is no documented Compose or Helm stack to run yourself. source
DeepEval B
Confident AI, Inc.
Nothing to host; runs are written to local JSON or SQLite stores. source
Galileo API + MCP D
Galileo Technologies, Inc.
On-prem and VPC deployment are Enterprise only per the pricing notes; no Compose or Helm files are documented. source
Gauge CLI B
Gauge
Local mode runs in Docker with results in .gauge/gauge.sqlite; there is no trace store to host. source
HoneyHive D
HoneyHive Inc.
Self-hosting is BYOC or BYOVPC on AWS, deployed by HoneyHive with Terraform and Helm for Enterprise; there is no stack to run from documented files. source
LangSmith API + MCP BB
LangChain, Inc.
Self-hosted LangSmith is an Enterprise add-on with a licence key; the ClickHouse or SmithDB, Postgres, Redis and blob stack is documented but not available to run otherwise. One builder made a local tool because LangSmith needs a cloud account. source
Navige E
Navige Ltd
Hosted only. source
Prefactor B
Prefactor Pty Ltd
Hosted only. source
Pydantic Logfire B
Pydantic Services Inc.
Self-hosted Logfire is an Enterprise deployment; the Personal, Team and Growth plans are cloud only. source
Respan API + MCP BB
Keywords AI Inc. (d/b/a Respan)
The trace platform is hosted only; the open-source gateway is a proxy with a request log, not a trace store. source
W&B Weave B
Weights and Biases, LLC
Multi-tenant or Dedicated Cloud only on the plans read; the trace server left the public repository in October 2026. source

What stops agents finishing

  • self-hosting is Enterprise only
  • ClickHouse or S3 growth with no retention setting
  • upgrade path breaks the trace tables
  • auth off by default on the self-hosted build
  • cloud account required to see one's own traces

Around it

Before: agent-observability.instrument-app. After: agent-observability.debug-failed-run.

Where this comes from

Weight 10 of 100 in Agent observability & evals: 11 of 86 accounts: Langfuse self-hosters (ClickHouse stack, terabyte disk, upgrade breakage), Phoenix at 44 GB, Laminar ClickHouse logs, MLflow as the established self-hosted option, the Ask HN comparison thread, LangSmith lock-in and cloud-account complaints that led to a local-first alternative

The same as data: /api/v1/use-cases/agent-observability.self-host-retain.json, or the MCP tool get_use_case with id agent-observability.self-host-retain. Other jobs in this category: /api/v1/niches?category=agent-observability.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.