{
  "fixes": {
    "slug": "khoj",
    "name": "Khoj",
    "listing": "https://www.anchorterminal.com/tools/khoj",
    "markdown": "# Fix list: Khoj\n\nFrom Anchor Terminal's listing at https://www.anchorterminal.com/tools/khoj, the October 2026 research run, assessed 3 October 2026. Grade E, 38.8 out of 100.\n\nThis is everything the published grade says the listing lacks, the biggest possible gain to the total first. It comes from the reason given for each score, the checklist each category was scored against (https://www.anchorterminal.com/benchmark/#checklist), the provenance checks, the deductions, what we couldn't check and what the review panel asked for. A fix counts at the next check, once it's public.\n\nFor a coding agent working on Khoj: work through the items below in the product, its docs and its public pages. Each category gives the reason for its score, with the points each checklist item earned, and the checklist itself, so the gap is the items that earned less than their points. Change the product, not the wording, and keep a note of what you changed and where it's published.\n\n## 1. Security \u0026 auth, 29 out of 100, up to 12.4 more on the total\n\nWhy it scored 29: Graded on the HTTP API with the tool lines. A signed-in user creates named `kk-` keys in Settings, lists them and revokes one with `DELETE /auth/token`, which takes the key as a `token` query parameter (the web app's own call, not a documented way to authenticate). Keys record their last use, have no scopes or expiry and are stored as plain text, which is 20 for plain revocable keys. We took 8 off, a departure we'd make for any listing whose documented default runs with no credential. Both quick starts run `--anonymous-mode`, which serves every request as a default user and doesn't mount the /auth routes, and the Compose file binds 0.0.0.0, publishes port 42110 on every host interface and ships `KHOJ_ADMIN_PASSWORD=password` and `KHOJ_DJANGO_SECRET_KEY=secret` (12 of 30). Admin and user roles, but no read-only key or scope, and API deletes (files, chat history, agents and the account itself through `DELETE /api/self`) run without confirmation. The Compose file runs code in a separate Terrarium container and computer use stays off unless an operator turns it on (4 of 20). Chat and research mode feed web pages, indexed files and MCP results to the model, a 2024 advisory (GHSA-h2q2-vch3-72qm) was XSS triggered by prompt injection, and we found no injection guidance (2 of 15). Keys record their last access, conversations are stored and the server logs at `-vv`, with no per-call audit trail (5 of 15). Private vulnerability reporting is on and six advisories have been published since 2024, but GitHub says the project has not set up a SECURITY.md, khoj.dev's security.txt returns 404 per the listing's check, there's no bounty, and both 2026 advisories list no patched version, one of them wrongly, since the path-traversal fix shipped in 2.0.0-beta.25 (6 of 20).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-security):\n\n- 0 to 30, the credential model. 30 for OAuth 2.1 with scopes, or scoped and revocable keys with rotation. 20 for plain revocable API keys. 10 for one all-powerful key. 10 off when a secret can travel in a URL query string as a documented option.\n- 0 to 20, read-only or least-privilege modes, and confirmation or approval for destructive actions.\n- 0 to 15, prompt-injection posture where the tool returns untrusted content (documented mitigations or guidance). A tool that returns no untrusted content gets 10.\n- 0 to 15, audit logs or per-call visibility for the operator.\n- 0 to 20, a security programme. security.txt or a disclosure policy, a bug bounty, SOC 2 or ISO 27001, advisories handled in public.\n\nModels are read for retention, whether API data trains models (and whether that's off by default), zero-retention options and certifications. Frameworks for telemetry defaults, approval hooks, guardrails and sandboxing.\n\n## 2. Schema \u0026 documentation, 34 out of 100, up to 10.7 more on the total\n\nWhy it scored 34: Graded on the HTTP API under /api, the interface an agent calls. Khoj calls MCP servers in research mode but doesn't run one. In 1.42.10, 2.0.0-beta.28 and master the app turns off only the Swagger UI (`docs_url=None`), so FastAPI's /openapi.json stays at its default on a running server. That's a machine-readable contract an agent could fetch, but it isn't published, linked or mentioned in the docs, and we read it from the source rather than a running server (10 of 25). No llms.txt or Markdown copies in the docs source (0). The docs explain the app to people and have no API reference, and route docstrings are one line or missing (3 of 20). FastAPI types the query parameters and Pydantic types the chat body, with one enum (search type `t`), bounds only on the chat export route, no free-form JSON bodies, and single-letter names (`q`, `n`, `t`, `r`, `d`) with no descriptions (8 of 15). No API examples in the docs (the one curl example is for the Terrarium sandbox) and no documented errors, though the query-filter page shows the filter syntax that goes inside `q` (3 of 15). Dated release notes on GitHub for each beta, and no version prefix on the routes (10 of 15).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-schema):\n\nAPIs and MCP servers.\n\n- 25, a machine-readable contract (a public OpenAPI file or similar; for MCP, typed JSON Schema inputs on every tool).\n- 10, llms.txt or Markdown docs served for agents.\n- 0 to 20, descriptions that say what a tool is for, when to use it and when not to, read from the tool definitions in the source or the API reference.\n- 0 to 15, typed inputs with enums, constraints and required fields, and no free-form JSON blobs.\n- 0 to 15, examples and documented error responses.\n- 15, versioning and a public changelog.\n\nModels are read from the API reference, the OpenAPI file, llms.txt, the structured-output and tool-use docs and the model cards. Frameworks from docs a model can follow, typed interfaces, examples and the API reference.\n\n## 3. Agent ergonomics, 46 out of 100, up to 8.8 more on the total\n\nWhy it scored 46: Graded on the HTTP API. `/api/search` returns `n` results (default 5) with a `max_distance` cut-off and deduplication, and chat takes `n` (default 7) and `d` for references and can stream, but chat answers have no length control and nothing selects fields (14 of 25). Search filters by content type, and the query syntax filters by file, date and word, with no offset for paging search results (12 of 20). Errors are FastAPI `detail` strings with status codes and no documented list. Some name the fix (\"Contact the server administrator to add a chat model\"), others are chatty quota messages, and a database failure during sign-in answers 503 asking the caller to report it on GitHub, Discord or by email (8 of 20). GET search is safe to repeat, chat POSTs append to a conversation, and there are no idempotency keys or retry guidance (5 of 20). Only `q` is required for search and chat, but there's no official SDK, only the web, desktop, Obsidian and Emacs clients (7 of 15).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-ergonomics):\n\n- 0 to 25, context cost. For MCP, the number and size of the tool definitions (25 for ten or fewer compact tools, 15 for 11 to 30, 5 for more than 30, plus up to 10 back for toolsets, dynamic loading or read-only subsets). For APIs, whether responses can be sized (field selection, limits, summaries).\n- 20, pagination, filtering and output-size controls.\n- 20, actionable, documented error responses, codes and messages an agent can recover from.\n- 20, idempotency or safe retries, and for MCP the `readOnlyHint` and `destructiveHint` annotations.\n- 15, sensible defaults, few required parameters, and official SDKs in at least two languages.\n\nModels are read for tool use, structured output, prompt caching, context length, batch and SDKs. Frameworks for how much code and how many defaults a tool-calling agent with MCP needs.\n\n## 4. Maintenance \u0026 community, 19 out of 100, up to 7.1 more on the total\n\nWhy it scored 19: The last tagged release is 2.0.0-beta.28 of 26 March 2026, 191 days before this check. PyPI's 2.0.0b29 .dev builds of 24 June and 2 August are automatic builds of master, which we don't count (0). No tagged release in the last 90 days (0). 12 commits on master since 1 April. A maintainer last authored commits on 24 and 25 June and merged two contributor pull requests on 2 August, one fixing #1374 within three weeks, and two July reports got a `fix` label, but we saw no maintainer reply on the open bug reports (8 of 25). Not an MCP server and no official SDK, and the stable PyPI release and the `latest` image trail master by 14 months (5 of 15). Dependencies were bumped on 24 June 2026 and test CI passes, but no release carries the bump, and what pip installs dates from July 2025 (6 of 10).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-maintenance):\n\n- 0 to 30, time since the last release, or the last published model or API change for a closed service. 30 within 30 days, 20 within 90, 10 within 180, 0 older.\n- 20, at least three releases or dated changelog entries in the last 90 days.\n- 0 to 25, responsiveness. Issues and pull requests answered on GitHub (the open issues and how recent the replies are). For closed services, a public changelog and a support or community channel that answers, 0 to 15.\n- 15, presence in the official MCP registry under a verified namespace (MCP servers), or current official SDKs (APIs and models).\n- 10, package health, current dependencies and CI.\n\nModels are read for deprecation notice periods and model churn rather than release counts.\n\n## 5. Reliability, 65 out of 100, up to 7 more on the total\n\nWhy it scored 65: Read with the local-software lines, as the Goose and Aider calibration dossiers do. `khoj` is on PyPI for Python 3.10 to 3.12 with images on ghcr.io, but both documented installs land on 1.42.10 of 15 July 2025. `pip install 'khoj[local]'` takes the newest stable release and the setup docs never mention `--pre`, and the dockerize workflow moves the `latest` image tag the Compose file pulls only on X.Y.Z version tags, while every build since is a 2.0.0 pre-release (14 of 20). The test workflow runs pytest against Postgres on Python 3.10, 3.11 and 3.12, and the master runs we saw passed, the last on 2 August 2026 (25). 99 open issues on 3 October, the newest from 24 July, among them an Emacs client crash (#1378), Windows PDF indexing (#1368) and a GPT 5.6 temperature error (#1377). Two July reports carry a `fix` label, and we saw no maintainer reply on them (12 of 25). Dated GitHub release notes for each 2.0 beta with no breaking-change sections, while the betas dropped in-process GGUF chat models (gone by 2.0.0-beta.13) and Stability AI images (2.0.0-beta.21) (6 of 15). 1.42.10 is a stable 1.x release and PyPI says Production/Stable, but all work since July 2025 has shipped as 2.0.0 betas (8 of 15).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-reliability):\n\nHosted APIs, MCP servers, models and platforms.\n\n- 20, a public status page with component history (Statuspage, Instatus, BetterStack or the vendor's own).\n- 0 to 30, the incident record for the last 90 days on that page. 30 for a clean record or trivial incidents only, 20 for minor incidents only, 10 for one major outage (an hour or more of a core API down, or errors across the board), 0 for several. 5 when there's no history we could read, and the note says so.\n- 15, rate limits documented with numbers.\n- 15, documented 429 or overload handling (Retry-After, backoff guidance), and idempotency keys or safe-retry guidance where writes are involved.\n- 10, an SLA published for any paid tier.\n- 10, the surface agents use is generally available, not beta or preview.\n\nLocal packages, SDKs, frameworks and stdio MCP servers.\n\n- 20, installs from an official package with supported runtimes stated.\n- 25, a public CI and test suite, passing on the default branch.\n- 0 to 25, open crash or regression issues relative to activity (25 for few and handled, 0 for many, old and unanswered).\n- 15, semver discipline and breaking changes called out in a changelog.\n- 15, version 1.0 or later, or declared stable.\n\nProtocols are read from their reference implementations, the public facilitators or servers, spec stability and test vectors.\n\n## 6. Payments \u0026 pricing, 60 out of 100, up to 5 more on the total\n\nWhy it scored 60: Read with the self-hosted rule. The Marmot dossier scored Marmot Cloud because Marmot sells it with published plans and a live sign-up. Khoj's facts aren't the same kind. Khoj Cloud closed on 15 April 2026 (app.khoj.dev shows a sunset notice), and khoj.dev/teams, which the README calls Khoj Enterprise, is a contact form headed Khoj for Teams that names no product, plan, price or licence and says nothing about payment. With nothing on sale that we could find on 3 October, Khoj scores as free software with nothing to buy. No payment protocol (0). Free under AGPL-3.0, so 20, 20 and 20 on the last three lines, and the documented quick starts run with no sign-up.\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-payments):\n\nThe published rubric, also on the [x402 page](https://www.anchorterminal.com/x402/).\n\n- 40, a machine payment protocol (x402, MPP or L402) on the tool's own endpoints. 10 to 30 when it covers only some endpoints or only goes through a third party, and the note says which.\n- 20, per-call or per-unit pricing published without a login. 10 for public plan-only pricing, 0 for \"contact sales\" or prices behind a login.\n- 20, a free tier or trial that doesn't need a card.\n- 20, autonomous onboarding, meaning an agent can get access without a person signing up in a browser (keyless use, x402, a programmatic key API).\n\nPayment platforms and agent wallets rarely charge for their own API over a machine protocol, so the first line has steps for them, and the highest one that applies counts. 40 when x402, MPP or L402 runs on all their own endpoints, 30 when it runs on part of their own API, 25 when their merchants can accept one, 20 for running a facilitator, 15 for paying as a buyer, and 0 when the only protocol is their own. Merchant acceptance sits above a facilitator because the platform's own customers can charge agents through it, while a facilitator settles for sellers who wire up the protocol themselves. The counter-argument (a facilitator does more for the protocol as a whole) has a point. Each note says which step applied.\n\nOpen-source software you run yourself is scored on its hosted or paid option if it has one. A free, self-hosted package with nothing to buy gets 20, 20 and 20 for the last three lines, and 0 to 40 for the first only if it ships a payment protocol.\n\n## 7. Transparency \u0026 trust, 64 out of 100, up to 3.2 more on the total\n\nMade of editorial 60, provenance 67.\n\nWhy it scored 64: AGPL-3.0-or-later (30). The privacy policy of 5 June 2024 names Khoj Inc. with no address, predates the cloud's closure, names no third parties and says log data may include IP addresses. The docs' privacy page says Khoj doesn't log your IP address, while every tagged release sends the caller's IP as `client_host` in telemetry, and the same page still describes Khoj Cloud storage on AWS (8 of 30). The cloud shutdown had dated notice, a banner in the app from 25 March 2026 for 15 April, and app.khoj.dev now shows a dated sunset notice, but the README still says you can use Khoj right away at app.khoj.dev with no setup, and there's no deprecation policy for the software (10 of 20). Telemetry is on by default and documented with its fields, one variable turns it off (`KHOJ_TELEMETRY_DISABLE=True`), and the Compose file has a commented line for it. The IP field contradicted the privacy page in every release, and the 2 August fix is on master only (12 of 20).\n\nThe checklist (https://www.anchorterminal.com/benchmark/#checklist-transparency):\n\n- 0 to 30, source availability and licence clarity. 30 for open source under an OSI licence, 15 for closed with clear terms, 0 for unclear terms.\n- 0 to 30, data handling and retention statements that agree with each other (privacy policy, DPA, retention periods, subprocessors).\n- 0 to 20, a deprecation policy or notices with dates.\n- 0 to 20, telemetry disclosed with an opt-out (local software), or subprocessors and data locations disclosed (hosted).\n\nThe other half of Transparency and trust is the provenance score, computed from checked facts (below). The category score is the mean of the two.\n\nProvenance checks not met in full (half of this category, computed from checked facts):\n\n- Domain age: khoj.dev, registered 2023-05-20 (3 years) (7 of 15)\n- Status page: not found (0 of 10)\n- security.txt: not found (0 of 10)\n\n## Deductions\n\nEach comes off the total. A fixed and documented problem counts for less at the next check.\n\n- 2026-07-13. Default-on telemetry sent the caller's IP (`client_host`) to khoj.beta.haletic.com and on to PostHog while the docs' privacy page said Khoj doesn't log IP addresses. Reported in #1374 and removed on master on 2 August 2026, but 1.42.10 and 2.0.0-beta.28, the versions the documented installs and the latest tag give, still send it. Request metadata rather than content, so the minimum, -2. https://github.com/khoj-ai/khoj/commit/4d7ac85a3f99b05f2d17f311679cff046d70d614\n- 2026-04-15. Khoj Cloud shut down, and on 3 October 2026 the README still says you can use Khoj right away at app.khoj.dev with no setup, the docs site still links to app.khoj.dev, and the Obsidian plugin, Emacs package and desktop app still default their server URL to https://app.khoj.dev. An endpoint removed while still advertised. The shutdown had three weeks' notice in the app, so the minimum, -3. https://github.com/khoj-ai/khoj/blob/master/README.md; https://github.com/khoj-ai/khoj/blob/master/src/interface/obsidian/src/settings.ts\n- 2026-02-01. CVE-2025-69207 (GHSA-6whj-7qmg-86qj, 5.4), an IDOR in the Notion OAuth callback that lets an attacker replace another user's Notion connection and poison their index. The check was hardened on 28 December 2025 and ships in 2.0.0-beta.23 and later, but the advisory lists no patched version, and 1.42.10, which pip and the latest image install, still trusts the `state` parameter. It needs a Notion OAuth app and more than one user, -1. https://github.com/khoj-ai/khoj/security/advisories/GHSA-6whj-7qmg-86qj\n- 2026-06-24. GHSA-62mm-xwmv-crhg, an unauthenticated path traversal through `/home/{file_path:path}` that reads any file the server process can. The route arrived in 2.0.0-beta.23 (29 December 2025) and was guarded in 2.0.0-beta.25 (22 February 2026), so two pre-releases were exposed and 1.42.10 never had the route. Fixed four months before publication, though the advisory still says no version is patched. Fixed and decayed, -1. https://github.com/khoj-ai/khoj/security/advisories/GHSA-62mm-xwmv-crhg; https://github.com/khoj-ai/khoj/commit/21c51b9a\n\n## What we couldn't check\n\nWhat we couldn't read counted as absent. Publishing it on a page a plain HTTP fetch can read (not only in a browser) lets the next check count it.\n\n- Whether Khoj for Teams is a paid service. khoj.dev/teams names no product, plan, price or licence, so we scored Khoj as having nothing to buy\n- Whether the team still maintains Khoj. There's no statement either way, only the gap since 2 August 2026 and the README's lead on Pipali\n- Unchecked: which image `ghcr.io/khoj-ai/khoj:latest` resolves to today. The workflow rule points at the last X.Y.Z tag, 1.42.10, and the registry page we loaded didn't show the `latest` tag\n- Unchecked: whether /openapi.json and /redoc answer on a running server. The source leaves FastAPI's defaults on in every version, and we didn't run one\n- Unchecked: whether docs.khoj.dev serves an llms.txt from outside the docs source\n- Whether the 1.x line will get the fixes for CVE-2025-69207 and the telemetry IP field\n- Who answers issues and how fast, since comment counts didn't load for our reader\n\n## Weaknesses\n\n- No tagged release since 2.0.0-beta.28 on 26 March 2026 and no commit since 2 August\n- `pip install khoj` and the Compose file's `latest` image give 1.42.10 from July 2025, without the fix for CVE-2025-69207\n- Both documented quick starts run in anonymous mode with no credential, and Compose publishes port 42110 on every host interface with example secrets\n- The README, docs and the Obsidian, Emacs and desktop clients still point at Khoj Cloud, which closed on 15 April 2026\n- No API reference, llms.txt or published OpenAPI file\n\n## What costs an agent a turn today\n\nThe notes we give agents before they call it. Each one is a workaround an agent shouldn't need.\n\n- Install with `pip install --pre khoj` or a 2.0.0-beta image tag. Plain `pip install khoj` and `latest` give 1.42.10 from July 2025\n- Point the Obsidian, Emacs or desktop client at your own server. They default to app.khoj.dev, which shut down on 15 April 2026\n- Send a `kk-` key from Settings as a Bearer token when the server runs without `--anonymous-mode`. In anonymous mode /auth isn't mounted and no key exists\n- Call `GET /api/search?q=...\u0026n=5` for passages and put `file:\"notes.md\"` or `dt\u003e=\"2026-01-01\"` inside `q` to filter. No route is documented\n- Set `KHOJ_TELEMETRY_DISABLE=True` before the first start. Tagged releases send the caller's IP with telemetry\n\n## What the review panel asked for\n\n- a stable 2.0 release\n- a maintenance statement\n- sign-in on by default\n- a stable release carrying the fixes\n\n## When it's done\n\nSend what changed and where it's published as a dispute (https://www.anchorterminal.com/builders/#disputes, or `POST https://www.anchorterminal.com/api/v1/contact` with `\"kind\": \"dispute\"`). Disputes are answered in public, and the listing is checked again by the same checklist. Paying for an audit or a listing claim changes nothing here.\n",
    "grade": "E",
    "score": 38.8,
    "assessed": "2026-10-03",
    "run": "October 2026 research run",
    "categories": [
      {
        "key": "security",
        "name": "Security \u0026 auth",
        "score": 29,
        "maxGain": 12.4,
        "reason": "Graded on the HTTP API with the tool lines. A signed-in user creates named `kk-` keys in Settings, lists them and revokes one with `DELETE /auth/token`, which takes the key as a `token` query parameter (the web app's own call, not a documented way to authenticate). Keys record their last use, have no scopes or expiry and are stored as plain text, which is 20 for plain revocable keys. We took 8 off, a departure we'd make for any listing whose documented default runs with no credential. Both quick starts run `--anonymous-mode`, which serves every request as a default user and doesn't mount the /auth routes, and the Compose file binds 0.0.0.0, publishes port 42110 on every host interface and ships `KHOJ_ADMIN_PASSWORD=password` and `KHOJ_DJANGO_SECRET_KEY=secret` (12 of 30). Admin and user roles, but no read-only key or scope, and API deletes (files, chat history, agents and the account itself through `DELETE /api/self`) run without confirmation. The Compose file runs code in a separate Terrarium container and computer use stays off unless an operator turns it on (4 of 20). Chat and research mode feed web pages, indexed files and MCP results to the model, a 2024 advisory (GHSA-h2q2-vch3-72qm) was XSS triggered by prompt injection, and we found no injection guidance (2 of 15). Keys record their last access, conversations are stored and the server logs at `-vv`, with no per-call audit trail (5 of 15). Private vulnerability reporting is on and six advisories have been published since 2024, but GitHub says the project has not set up a SECURITY.md, khoj.dev's security.txt returns 404 per the listing's check, there's no bounty, and both 2026 advisories list no patched version, one of them wrongly, since the path-traversal fix shipped in 2.0.0-beta.25 (6 of 20).",
        "checklist": [
          "- 0 to 30, the credential model. 30 for OAuth 2.1 with scopes, or scoped and revocable keys with rotation. 20 for plain revocable API keys. 10 for one all-powerful key. 10 off when a secret can travel in a URL query string as a documented option.\n- 0 to 20, read-only or least-privilege modes, and confirmation or approval for destructive actions.\n- 0 to 15, prompt-injection posture where the tool returns untrusted content (documented mitigations or guidance). A tool that returns no untrusted content gets 10.\n- 0 to 15, audit logs or per-call visibility for the operator.\n- 0 to 20, a security programme. security.txt or a disclosure policy, a bug bounty, SOC 2 or ISO 27001, advisories handled in public.",
          "Models are read for retention, whether API data trains models (and whether that's off by default), zero-retention options and certifications. Frameworks for telemetry defaults, approval hooks, guardrails and sandboxing."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-security"
      },
      {
        "key": "schema",
        "name": "Schema \u0026 documentation",
        "score": 34,
        "maxGain": 10.7,
        "reason": "Graded on the HTTP API under /api, the interface an agent calls. Khoj calls MCP servers in research mode but doesn't run one. In 1.42.10, 2.0.0-beta.28 and master the app turns off only the Swagger UI (`docs_url=None`), so FastAPI's /openapi.json stays at its default on a running server. That's a machine-readable contract an agent could fetch, but it isn't published, linked or mentioned in the docs, and we read it from the source rather than a running server (10 of 25). No llms.txt or Markdown copies in the docs source (0). The docs explain the app to people and have no API reference, and route docstrings are one line or missing (3 of 20). FastAPI types the query parameters and Pydantic types the chat body, with one enum (search type `t`), bounds only on the chat export route, no free-form JSON bodies, and single-letter names (`q`, `n`, `t`, `r`, `d`) with no descriptions (8 of 15). No API examples in the docs (the one curl example is for the Terrarium sandbox) and no documented errors, though the query-filter page shows the filter syntax that goes inside `q` (3 of 15). Dated release notes on GitHub for each beta, and no version prefix on the routes (10 of 15).",
        "checklist": [
          "APIs and MCP servers.",
          "- 25, a machine-readable contract (a public OpenAPI file or similar; for MCP, typed JSON Schema inputs on every tool).\n- 10, llms.txt or Markdown docs served for agents.\n- 0 to 20, descriptions that say what a tool is for, when to use it and when not to, read from the tool definitions in the source or the API reference.\n- 0 to 15, typed inputs with enums, constraints and required fields, and no free-form JSON blobs.\n- 0 to 15, examples and documented error responses.\n- 15, versioning and a public changelog.",
          "Models are read from the API reference, the OpenAPI file, llms.txt, the structured-output and tool-use docs and the model cards. Frameworks from docs a model can follow, typed interfaces, examples and the API reference."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-schema"
      },
      {
        "key": "ergonomics",
        "name": "Agent ergonomics",
        "score": 46,
        "maxGain": 8.8,
        "reason": "Graded on the HTTP API. `/api/search` returns `n` results (default 5) with a `max_distance` cut-off and deduplication, and chat takes `n` (default 7) and `d` for references and can stream, but chat answers have no length control and nothing selects fields (14 of 25). Search filters by content type, and the query syntax filters by file, date and word, with no offset for paging search results (12 of 20). Errors are FastAPI `detail` strings with status codes and no documented list. Some name the fix (\"Contact the server administrator to add a chat model\"), others are chatty quota messages, and a database failure during sign-in answers 503 asking the caller to report it on GitHub, Discord or by email (8 of 20). GET search is safe to repeat, chat POSTs append to a conversation, and there are no idempotency keys or retry guidance (5 of 20). Only `q` is required for search and chat, but there's no official SDK, only the web, desktop, Obsidian and Emacs clients (7 of 15).",
        "checklist": [
          "- 0 to 25, context cost. For MCP, the number and size of the tool definitions (25 for ten or fewer compact tools, 15 for 11 to 30, 5 for more than 30, plus up to 10 back for toolsets, dynamic loading or read-only subsets). For APIs, whether responses can be sized (field selection, limits, summaries).\n- 20, pagination, filtering and output-size controls.\n- 20, actionable, documented error responses, codes and messages an agent can recover from.\n- 20, idempotency or safe retries, and for MCP the `readOnlyHint` and `destructiveHint` annotations.\n- 15, sensible defaults, few required parameters, and official SDKs in at least two languages.",
          "Models are read for tool use, structured output, prompt caching, context length, batch and SDKs. Frameworks for how much code and how many defaults a tool-calling agent with MCP needs."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-ergonomics"
      },
      {
        "key": "maintenance",
        "name": "Maintenance \u0026 community",
        "score": 19,
        "maxGain": 7.1,
        "reason": "The last tagged release is 2.0.0-beta.28 of 26 March 2026, 191 days before this check. PyPI's 2.0.0b29 .dev builds of 24 June and 2 August are automatic builds of master, which we don't count (0). No tagged release in the last 90 days (0). 12 commits on master since 1 April. A maintainer last authored commits on 24 and 25 June and merged two contributor pull requests on 2 August, one fixing #1374 within three weeks, and two July reports got a `fix` label, but we saw no maintainer reply on the open bug reports (8 of 25). Not an MCP server and no official SDK, and the stable PyPI release and the `latest` image trail master by 14 months (5 of 15). Dependencies were bumped on 24 June 2026 and test CI passes, but no release carries the bump, and what pip installs dates from July 2025 (6 of 10).",
        "checklist": [
          "- 0 to 30, time since the last release, or the last published model or API change for a closed service. 30 within 30 days, 20 within 90, 10 within 180, 0 older.\n- 20, at least three releases or dated changelog entries in the last 90 days.\n- 0 to 25, responsiveness. Issues and pull requests answered on GitHub (the open issues and how recent the replies are). For closed services, a public changelog and a support or community channel that answers, 0 to 15.\n- 15, presence in the official MCP registry under a verified namespace (MCP servers), or current official SDKs (APIs and models).\n- 10, package health, current dependencies and CI.",
          "Models are read for deprecation notice periods and model churn rather than release counts."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-maintenance"
      },
      {
        "key": "reliability",
        "name": "Reliability",
        "score": 65,
        "maxGain": 7,
        "reason": "Read with the local-software lines, as the Goose and Aider calibration dossiers do. `khoj` is on PyPI for Python 3.10 to 3.12 with images on ghcr.io, but both documented installs land on 1.42.10 of 15 July 2025. `pip install 'khoj[local]'` takes the newest stable release and the setup docs never mention `--pre`, and the dockerize workflow moves the `latest` image tag the Compose file pulls only on X.Y.Z version tags, while every build since is a 2.0.0 pre-release (14 of 20). The test workflow runs pytest against Postgres on Python 3.10, 3.11 and 3.12, and the master runs we saw passed, the last on 2 August 2026 (25). 99 open issues on 3 October, the newest from 24 July, among them an Emacs client crash (#1378), Windows PDF indexing (#1368) and a GPT 5.6 temperature error (#1377). Two July reports carry a `fix` label, and we saw no maintainer reply on them (12 of 25). Dated GitHub release notes for each 2.0 beta with no breaking-change sections, while the betas dropped in-process GGUF chat models (gone by 2.0.0-beta.13) and Stability AI images (2.0.0-beta.21) (6 of 15). 1.42.10 is a stable 1.x release and PyPI says Production/Stable, but all work since July 2025 has shipped as 2.0.0 betas (8 of 15).",
        "checklist": [
          "Hosted APIs, MCP servers, models and platforms.",
          "- 20, a public status page with component history (Statuspage, Instatus, BetterStack or the vendor's own).\n- 0 to 30, the incident record for the last 90 days on that page. 30 for a clean record or trivial incidents only, 20 for minor incidents only, 10 for one major outage (an hour or more of a core API down, or errors across the board), 0 for several. 5 when there's no history we could read, and the note says so.\n- 15, rate limits documented with numbers.\n- 15, documented 429 or overload handling (Retry-After, backoff guidance), and idempotency keys or safe-retry guidance where writes are involved.\n- 10, an SLA published for any paid tier.\n- 10, the surface agents use is generally available, not beta or preview.",
          "Local packages, SDKs, frameworks and stdio MCP servers.",
          "- 20, installs from an official package with supported runtimes stated.\n- 25, a public CI and test suite, passing on the default branch.\n- 0 to 25, open crash or regression issues relative to activity (25 for few and handled, 0 for many, old and unanswered).\n- 15, semver discipline and breaking changes called out in a changelog.\n- 15, version 1.0 or later, or declared stable.",
          "Protocols are read from their reference implementations, the public facilitators or servers, spec stability and test vectors."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-reliability"
      },
      {
        "key": "payments",
        "name": "Payments \u0026 pricing",
        "score": 60,
        "maxGain": 5,
        "reason": "Read with the self-hosted rule. The Marmot dossier scored Marmot Cloud because Marmot sells it with published plans and a live sign-up. Khoj's facts aren't the same kind. Khoj Cloud closed on 15 April 2026 (app.khoj.dev shows a sunset notice), and khoj.dev/teams, which the README calls Khoj Enterprise, is a contact form headed Khoj for Teams that names no product, plan, price or licence and says nothing about payment. With nothing on sale that we could find on 3 October, Khoj scores as free software with nothing to buy. No payment protocol (0). Free under AGPL-3.0, so 20, 20 and 20 on the last three lines, and the documented quick starts run with no sign-up.",
        "checklist": [
          "The published rubric, also on the [x402 page](https://www.anchorterminal.com/x402/).",
          "- 40, a machine payment protocol (x402, MPP or L402) on the tool's own endpoints. 10 to 30 when it covers only some endpoints or only goes through a third party, and the note says which.\n- 20, per-call or per-unit pricing published without a login. 10 for public plan-only pricing, 0 for \"contact sales\" or prices behind a login.\n- 20, a free tier or trial that doesn't need a card.\n- 20, autonomous onboarding, meaning an agent can get access without a person signing up in a browser (keyless use, x402, a programmatic key API).",
          "Payment platforms and agent wallets rarely charge for their own API over a machine protocol, so the first line has steps for them, and the highest one that applies counts. 40 when x402, MPP or L402 runs on all their own endpoints, 30 when it runs on part of their own API, 25 when their merchants can accept one, 20 for running a facilitator, 15 for paying as a buyer, and 0 when the only protocol is their own. Merchant acceptance sits above a facilitator because the platform's own customers can charge agents through it, while a facilitator settles for sellers who wire up the protocol themselves. The counter-argument (a facilitator does more for the protocol as a whole) has a point. Each note says which step applied.",
          "Open-source software you run yourself is scored on its hosted or paid option if it has one. A free, self-hosted package with nothing to buy gets 20, 20 and 20 for the last three lines, and 0 to 40 for the first only if it ships a payment protocol."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-payments"
      },
      {
        "key": "transparency",
        "name": "Transparency \u0026 trust",
        "score": 64,
        "maxGain": 3.2,
        "reason": "AGPL-3.0-or-later (30). The privacy policy of 5 June 2024 names Khoj Inc. with no address, predates the cloud's closure, names no third parties and says log data may include IP addresses. The docs' privacy page says Khoj doesn't log your IP address, while every tagged release sends the caller's IP as `client_host` in telemetry, and the same page still describes Khoj Cloud storage on AWS (8 of 30). The cloud shutdown had dated notice, a banner in the app from 25 March 2026 for 15 April, and app.khoj.dev now shows a dated sunset notice, but the README still says you can use Khoj right away at app.khoj.dev with no setup, and there's no deprecation policy for the software (10 of 20). Telemetry is on by default and documented with its fields, one variable turns it off (`KHOJ_TELEMETRY_DISABLE=True`), and the Compose file has a commented line for it. The IP field contradicted the privacy page in every release, and the 2 August fix is on master only (12 of 20).",
        "blend": "editorial 60, provenance 67",
        "checklist": [
          "- 0 to 30, source availability and licence clarity. 30 for open source under an OSI licence, 15 for closed with clear terms, 0 for unclear terms.\n- 0 to 30, data handling and retention statements that agree with each other (privacy policy, DPA, retention periods, subprocessors).\n- 0 to 20, a deprecation policy or notices with dates.\n- 0 to 20, telemetry disclosed with an opt-out (local software), or subprocessors and data locations disclosed (hosted).",
          "The other half of Transparency and trust is the provenance score, computed from checked facts (below). The category score is the mean of the two."
        ],
        "checklistUrl": "https://www.anchorterminal.com/benchmark/#checklist-transparency"
      }
    ],
    "provenance": [
      {
        "label": "Domain age",
        "value": "khoj.dev, registered 2023-05-20 (3 years)",
        "points": 7,
        "max": 15
      },
      {
        "label": "Status page",
        "value": "not found",
        "points": 0,
        "max": 10
      },
      {
        "label": "security.txt",
        "value": "not found",
        "points": 0,
        "max": 10
      }
    ],
    "deductions": [
      "2026-07-13. Default-on telemetry sent the caller's IP (`client_host`) to khoj.beta.haletic.com and on to PostHog while the docs' privacy page said Khoj doesn't log IP addresses. Reported in #1374 and removed on master on 2 August 2026, but 1.42.10 and 2.0.0-beta.28, the versions the documented installs and the latest tag give, still send it. Request metadata rather than content, so the minimum, -2. https://github.com/khoj-ai/khoj/commit/4d7ac85a3f99b05f2d17f311679cff046d70d614",
      "2026-04-15. Khoj Cloud shut down, and on 3 October 2026 the README still says you can use Khoj right away at app.khoj.dev with no setup, the docs site still links to app.khoj.dev, and the Obsidian plugin, Emacs package and desktop app still default their server URL to https://app.khoj.dev. An endpoint removed while still advertised. The shutdown had three weeks' notice in the app, so the minimum, -3. https://github.com/khoj-ai/khoj/blob/master/README.md; https://github.com/khoj-ai/khoj/blob/master/src/interface/obsidian/src/settings.ts",
      "2026-02-01. CVE-2025-69207 (GHSA-6whj-7qmg-86qj, 5.4), an IDOR in the Notion OAuth callback that lets an attacker replace another user's Notion connection and poison their index. The check was hardened on 28 December 2025 and ships in 2.0.0-beta.23 and later, but the advisory lists no patched version, and 1.42.10, which pip and the latest image install, still trusts the `state` parameter. It needs a Notion OAuth app and more than one user, -1. https://github.com/khoj-ai/khoj/security/advisories/GHSA-6whj-7qmg-86qj",
      "2026-06-24. GHSA-62mm-xwmv-crhg, an unauthenticated path traversal through `/home/{file_path:path}` that reads any file the server process can. The route arrived in 2.0.0-beta.23 (29 December 2025) and was guarded in 2.0.0-beta.25 (22 February 2026), so two pre-releases were exposed and 1.42.10 never had the route. Fixed four months before publication, though the advisory still says no version is patched. Fixed and decayed, -1. https://github.com/khoj-ai/khoj/security/advisories/GHSA-62mm-xwmv-crhg; https://github.com/khoj-ai/khoj/commit/21c51b9a"
    ],
    "unchecked": [
      "Whether Khoj for Teams is a paid service. khoj.dev/teams names no product, plan, price or licence, so we scored Khoj as having nothing to buy",
      "Whether the team still maintains Khoj. There's no statement either way, only the gap since 2 August 2026 and the README's lead on Pipali",
      "Unchecked: which image `ghcr.io/khoj-ai/khoj:latest` resolves to today. The workflow rule points at the last X.Y.Z tag, 1.42.10, and the registry page we loaded didn't show the `latest` tag",
      "Unchecked: whether /openapi.json and /redoc answer on a running server. The source leaves FastAPI's defaults on in every version, and we didn't run one",
      "Unchecked: whether docs.khoj.dev serves an llms.txt from outside the docs source",
      "Whether the 1.x line will get the fixes for CVE-2025-69207 and the telemetry IP field",
      "Who answers issues and how fast, since comment counts didn't load for our reader"
    ],
    "weaknesses": [
      "No tagged release since 2.0.0-beta.28 on 26 March 2026 and no commit since 2 August",
      "`pip install khoj` and the Compose file's `latest` image give 1.42.10 from July 2025, without the fix for CVE-2025-69207",
      "Both documented quick starts run in anonymous mode with no credential, and Compose publishes port 42110 on every host interface with example secrets",
      "The README, docs and the Obsidian, Emacs and desktop clients still point at Khoj Cloud, which closed on 15 April 2026",
      "No API reference, llms.txt or published OpenAPI file"
    ],
    "agentNotes": [
      "Install with `pip install --pre khoj` or a 2.0.0-beta image tag. Plain `pip install khoj` and `latest` give 1.42.10 from July 2025",
      "Point the Obsidian, Emacs or desktop client at your own server. They default to app.khoj.dev, which shut down on 15 April 2026",
      "Send a `kk-` key from Settings as a Bearer token when the server runs without `--anonymous-mode`. In anonymous mode /auth isn't mounted and no key exists",
      "Call `GET /api/search?q=...\u0026n=5` for passages and put `file:\"notes.md\"` or `dt\u003e=\"2026-01-01\"` inside `q` to filter. No route is documented",
      "Set `KHOJ_TELEMETRY_DISABLE=True` before the first start. Tagged releases send the caller's IP with telemetry"
    ],
    "requests": [
      {
        "text": "a stable 2.0 release",
        "reviews": 1
      },
      {
        "text": "a maintenance statement",
        "reviews": 1
      },
      {
        "text": "sign-in on by default",
        "reviews": 1
      },
      {
        "text": "a stable release carrying the fixes",
        "reviews": 1
      }
    ],
    "recheck": "https://www.anchorterminal.com/builders/#disputes"
  },
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-10-04",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.3",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run"
  }
}