Niches · Agent observability & evals
Log every model call through a gateway with no code changes
Four or more companies' docs show a full fit.
What counts as done
Changing the base URL yields a logged request with cost, with the provider key never reaching the client, and a failure of the gateway surfaced as an error rather than a silent drop.
Measured, for our own tests: the logged request with cost appears within 2 minutes. No docs we read state these figures, so they don't count against a listing here.
How agents ask for it: log all my LLM calls by changing the base URL
, put a spend cap on the agent
, one key for all providers with logging
, see what the agent is sending
Does the job 4
The docs show it meets the test above, and nothing we read says otherwise.
| Listing | Why |
|---|---|
| LangSmith API + MCP BB LangChain, Inc. | The LLM Gateway takes one LangSmith key, keeps provider keys in LangSmith, traces every call with routing and policy decisions, and applies fallbacks, spend and rate limits; a blocked call shows each attempt in the trace. source |
| MLflow Tracing C MLflow Project, a Series of LF Projects, LLC | The AI Gateway proxies Claude Code through ANTHROPIC_BASE_URL with each developer's own Anthropic credentials, captures every conversation as a trace with tokens and latency, and adds budget alerts and limits. source |
| Navige E Navige Ltd | Change the base URL and send the Navige key in x-navige-key while the provider key stays in its own header; every call is recorded with cost, failures are marked, reads fail open as degraded and writes fail closed, and spend caps and the kill switch are free. source |
| Respan API + MCP BB Keywords AI Inc. (d/b/a Respan) | Change the base URL to gateway.respan.ai and the key; each call is logged with usage and cost and grouped into a trace per request, limits refuse with 429 or 402 before any provider is called, and keys can be your own. source |
Does the job, with a limit 4
It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.
| Listing | Why |
|---|---|
| Helicone AI Gateway + MCP E Helicone, Inc. | One OpenAI-compatible base URL and key for 100+ providers with logging, fallbacks and a documented error priority order; two HN users rely on it daily. Credits need Helicone to enable them per organisation, so bring your own keys. source |
| Langfuse API + MCP BB Langfuse GmbH (a ClickHouse, Inc. company) | The AI Gateway (beta) routes OpenAI and Anthropic calls through Langfuse and traces usage and cost with no code change, with client pages for Claude Code, Codex and Vercel AI SDK. Only the docs index was read and it is marked beta. source |
| LangWatch B Reasoning Engine B.V. (doing business as LangWatch) | The AI gateway issues virtual keys with budgets and its requests emit OTel traces to the owning project with no extra wiring; the gateway pages themselves were not read for failure behaviour. source |
| Pydantic Logfire B Pydantic Services Inc. | The FAQ lists an AI Gateway with unified keys, spending limits and provider failover; the gateway page was not read, so base URL logging and failure behaviour are unverified. source |
Couldn't check 2
The docs were closed to us, or silent. These carry no mark either way.
| Listing | Why |
|---|---|
| Braintrust API + MCP B Braintrust Data, Inc. | Braintrust lists a model gateway, but the proxy documentation was not opened in this read, so base URL logging and failure behaviour are unverified. |
| Opik API + MCP BB Comet ML, Inc. | The docs index lists a Gateway page under production guardrails, but it was not opened, so base URL logging and failure behaviour are unverified. |
Doesn't do this job 12
Listings in the category that do something else, with the reason.
| Listing | Why |
|---|---|
| Agent Tree C No legal entity found. The LICENSE, plugin.json and marketplace.json name "Mike Pollow, Digitalroots" | No gateway. source |
| agents-tree B No legal entity found. The LICENSE and pyproject.toml name Karel Blavka | No gateway. source |
| Arize Phoenix BB Arize AI, Inc. | Phoenix has no model gateway; it ingests traces from instrumented code only. source |
| BotGauge E BotGauge AI Inc. (Delaware, United States) | No gateway. source |
| DeepEval B Confident AI, Inc. | No gateway. source |
| Galileo API + MCP D Galileo Technologies, Inc. | No model gateway in the docs index. source |
| Gauge CLI B Gauge | No gateway. source |
| HoneyHive D HoneyHive Inc. | No model gateway in the docs index. source |
| Laminar API + MCP C LMNR AI, Inc. | No model gateway in the docs index; Laminar ingests traces only. source |
| Overmind C Overmind Technology Ltd (England and Wales, company 16585996) | Overmind serves fine-tuned models on an OpenAI-compatible API; it is not a logging gateway for other providers. source |
| Prefactor B Prefactor Pty Ltd | No gateway. source |
| W&B Weave B Weights and Biases, LLC | No model gateway in the Weave docs. source |
What stops agents finishing
- gateway becomes a single point of failure in the agent's path
- subscription plans not usable through a proxy
- credits need support approval
Around it
Before: inference.failover-routing. After: agent-observability.cost-per-run.
Where this comes from
Weight 3 of 100 in Agent observability & evals: 3 of 86 accounts: the Helicone Show HN user and the Go-stack founder on agnostic proxies, plus the LLM proxy thread where the proxy users named LiteLLM and Bifrost (not listed) and one argued against proxies at all
The same as data: /api/v1/use-cases/agent-observability.gateway-logging.json, or the MCP tool get_use_case with id agent-observability.gateway-logging. Other jobs in this category: /api/v1/niches?category=agent-observability.
For companies
Do agents find, use and choose your tools?
An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.