Niches · Agent observability & evals

Log every model call through a gateway with no code changes

Four or more companies' docs show a full fit.

4companies, full fit
4companies, fit with a limit
25companies carry the tag
3share of the category's work
3first-hand accounts

What counts as done

Changing the base URL yields a logged request with cost, with the provider key never reaching the client, and a failure of the gateway surfaced as an error rather than a silent drop.

Measured, for our own tests: the logged request with cost appears within 2 minutes. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: log all my LLM calls by changing the base URL, put a spend cap on the agent, one key for all providers with logging, see what the agent is sending

Does the job 4

The docs show it meets the test above, and nothing we read says otherwise.

ListingWhy
LangSmith API + MCP BB
LangChain, Inc.
The LLM Gateway takes one LangSmith key, keeps provider keys in LangSmith, traces every call with routing and policy decisions, and applies fallbacks, spend and rate limits; a blocked call shows each attempt in the trace. source
MLflow Tracing C
MLflow Project, a Series of LF Projects, LLC
The AI Gateway proxies Claude Code through ANTHROPIC_BASE_URL with each developer's own Anthropic credentials, captures every conversation as a trace with tokens and latency, and adds budget alerts and limits. source
Navige E
Navige Ltd
Change the base URL and send the Navige key in x-navige-key while the provider key stays in its own header; every call is recorded with cost, failures are marked, reads fail open as degraded and writes fail closed, and spend caps and the kill switch are free. source
Respan API + MCP BB
Keywords AI Inc. (d/b/a Respan)
Change the base URL to gateway.respan.ai and the key; each call is logged with usage and cost and grouped into a trace per request, limits refuse with 429 or 402 before any provider is called, and keys can be your own. source

Does the job, with a limit 4

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
Helicone AI Gateway + MCP E
Helicone, Inc.
One OpenAI-compatible base URL and key for 100+ providers with logging, fallbacks and a documented error priority order; two HN users rely on it daily. Credits need Helicone to enable them per organisation, so bring your own keys. source
Langfuse API + MCP BB
Langfuse GmbH (a ClickHouse, Inc. company)
The AI Gateway (beta) routes OpenAI and Anthropic calls through Langfuse and traces usage and cost with no code change, with client pages for Claude Code, Codex and Vercel AI SDK. Only the docs index was read and it is marked beta. source
LangWatch B
Reasoning Engine B.V. (doing business as LangWatch)
The AI gateway issues virtual keys with budgets and its requests emit OTel traces to the owning project with no extra wiring; the gateway pages themselves were not read for failure behaviour. source
Pydantic Logfire B
Pydantic Services Inc.
The FAQ lists an AI Gateway with unified keys, spending limits and provider failover; the gateway page was not read, so base URL logging and failure behaviour are unverified. source

Couldn't check 2

The docs were closed to us, or silent. These carry no mark either way.

ListingWhy
Braintrust API + MCP B
Braintrust Data, Inc.
Braintrust lists a model gateway, but the proxy documentation was not opened in this read, so base URL logging and failure behaviour are unverified.
Opik API + MCP BB
Comet ML, Inc.
The docs index lists a Gateway page under production guardrails, but it was not opened, so base URL logging and failure behaviour are unverified.

Doesn't do this job 12

Listings in the category that do something else, with the reason.

ListingWhy
Agent Tree C
No legal entity found. The LICENSE, plugin.json and marketplace.json name "Mike Pollow, Digitalroots"
No gateway. source
agents-tree B
No legal entity found. The LICENSE and pyproject.toml name Karel Blavka
No gateway. source
Arize Phoenix BB
Arize AI, Inc.
Phoenix has no model gateway; it ingests traces from instrumented code only. source
BotGauge E
BotGauge AI Inc. (Delaware, United States)
No gateway. source
DeepEval B
Confident AI, Inc.
No gateway. source
Galileo API + MCP D
Galileo Technologies, Inc.
No model gateway in the docs index. source
Gauge CLI B
Gauge
No gateway. source
HoneyHive D
HoneyHive Inc.
No model gateway in the docs index. source
Laminar API + MCP C
LMNR AI, Inc.
No model gateway in the docs index; Laminar ingests traces only. source
Overmind C
Overmind Technology Ltd (England and Wales, company 16585996)
Overmind serves fine-tuned models on an OpenAI-compatible API; it is not a logging gateway for other providers. source
Prefactor B
Prefactor Pty Ltd
No gateway. source
W&B Weave B
Weights and Biases, LLC
No model gateway in the Weave docs. source

What stops agents finishing

  • gateway becomes a single point of failure in the agent's path
  • subscription plans not usable through a proxy
  • credits need support approval

Around it

Before: inference.failover-routing. After: agent-observability.cost-per-run.

Where this comes from

Weight 3 of 100 in Agent observability & evals: 3 of 86 accounts: the Helicone Show HN user and the Go-stack founder on agnostic proxies, plus the LLM proxy thread where the proxy users named LiteLLM and Bifrost (not listed) and one argued against proxies at all

The same as data: /api/v1/use-cases/agent-observability.gateway-logging.json, or the MCP tool get_use_case with id agent-observability.gateway-logging. Other jobs in this category: /api/v1/niches?category=agent-observability.

Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.