DeepEval vs Navige

DeepEval

DeepEval runs evaluations and tracing locally under Apache 2.0 with no account, and writes each test run to JSON or SQLite. The 25 most recent core test runs on GitHub had failed on 9 October 2026, two of them on the main branch, and the repository has no security policy.

Navige

Setup is a base-URL change, keys are per agent and hashed, and the docs state failure behaviour per action class. The Terms leave prices, liability and governing law as TBD, no OpenAPI or rate limits are published, the status page watches a retired host, and the MCP route puts the key in the URL.

DeepEval

1. Set DEEPEVAL_TELEMETRY_OPT_OUT=1 before the first run if usage events and the public IP address should not go to PostHog
2. Set a judge model key such as OPENAI_API_KEY, or use the non-LLM metrics. Most metrics call an LLM judge and bill the owner's provider account
3. Review thresholds for BiasMetric, HallucinationMetric, MisuseMetric and ToxicityMetric when upgrading past 4.2.0. Higher scores now mean better
4. Read results from .deepeval/.latest_run_full.json or a results_folder. deepeval inspect opens a terminal interface meant for a person
5. Pass an existing key with deepeval login --api-key in CI. Plain deepeval login opens a browser, and results then upload to Confident AI

Navige

1. Send the Navige key in x-navige-key on the proxy. Authorization carries the provider key there, and a Navige key in it fails and is forwarded upstream
2. Act only on allow or approved from /enforce. Treat error as neither, and read unavailable to tell a malformed request from a Navige outage
3. Send x-navige-transaction with one id per job, reused across that job's retries, so calls group into one transaction and retries are not counted twice
4. On a 429 from the proxy, check for a blocking budget before retrying. Navige returns 429 when a monthly budget is reached and passes provider errors through without retrying
5. Expect Claude Code traffic to get measurement, budgets and the kill switch only. Context trimming, routing, the output ceiling and prompt caching are switched off for it