Head to head · Agent harnesses · October 2026 research run

Kilo Code CLI vs OpenAI Agents API

OpenAI Agents API scores 73.3 (BB) on agent readiness against Kilo Code CLI's 68 (B), and leads in 3 of 7 scored categories. Kilo Code CLI leads on reliability and payments & pricing. Both do agent harnesses.

Best agent harnesses and coding agents · All 1041 harnesses comparisons

Which one, for what

Kilo Code CLI B

Best for Developers who want an open-source terminal agent with per-tool permission rules, an optional sandbox and a choice of provider, and the same engine in VS Code and JetBrains.

Ahead on

  • Reliability, 85 against 60
  • Payments & pricing, 50 against 30

Also in its favour

  • Free to start without a card
  • Open source

Watch for

Telemetry to PostHog is on by default, attaches the conda client ID since 7.8.10, and the privacy policy doesn't mention it

OpenAI Agents API BB

Best for Teams that want OpenAI to run the agent loop, with sandboxes, MCP tools and subagents, and are fine with a beta API, US-only data residency and no Zero Data Retention.

Ahead on

  • Agent ergonomics, 80 against 72
  • Security & auth, 87 against 64
  • Transparency & trust, 85 against 76

Also in its favour

  • Agent-ready, a grade of BB or better
  • A hosted endpoint, with nothing to install
  • No incidents deducted, where Kilo Code CLI loses 7 points for them

Watch for

Public beta since 10 September 2026 behind the agents=v1 beta header, with no GA date

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Reliability Kilo Code CLI +25

Kilo Code CLI 85

Local-package reading. @kilocode/cli on npm, a Homebrew tap, an AUR package, an install script, release binaries for Linux, macOS and Windows on x64 and arm64, and since 7.8.20 (27 October) a kilo-cli conda package on Anaconda.org (20). Public CI, and the 40 most recent completed workflow runs on main on 8 October 2026 all succeeded, per the 8 October check, not re-read on 17 October or 1 November (25). 448 open issues and 147 open pull requests, a workflow that closes issues after 60 quiet days and pull requests after 30, and open reports from 7 October of a hanging turn (#14876) and of exhausted 503 retries ending a run with no visible error (#14877), per the 8 October check, not re-read since (15). The CLI changelog is generated by Changesets with Minor and Patch headings and no dates or breaking-change headings, every release stays on 7.8.x whatever its heading, and 7.8.13 and 7.8.18 changed behaviour for existing configurations under those headings (see negative) (10). 7.8.18 on the npm latest tag on 1 November, past 1.0 (15). 20 + 25 + 15 + 10 + 15 = 85.

OpenAI Agents API 60

Read as a hosted API. status.openai.com has an Agents component under APIs (20). Since release on 10 September (57 days, not 90), the feed names the Agents API in a degraded-performance incident on 14 September, the elevated errors across ChatGPT, Codex and the API on 29 September (the openai-api listing reads that one as a major, 17:52 to 23:14 UTC), computer use issues and degraded workspace agents on 6 and 7 October, degraded performance across the APIs for 22 minutes on 21 October, and overbilling for hosted containers on 19 September. One major, the rest minor (10). Rate limits come from the shared guide, published per tier and model with headers, and the errors page adds a Files API allowance on session creation (15). 429 and 503 guidance with Retry-After, capped backoff and a check-then-retry procedure for turns (15). The Scale Tier SLA in the openai-api listing isn't shown to cover a beta API (0). Public beta behind the agents=v1 header (0). 20 + 10 + 15 + 15 + 0 + 0 = 60.

Security & auth OpenAI Agents API +23

Kilo Code CLI 64

Harness reading of the framework checklist. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. Telemetry is on by default and goes to PostHog in the United States, the source links the machine identifier to the account email after a Kilo sign-in, and since 7.8.10 (14 October) it also attaches the conda client ID when ~/.conda/aau_token exists (see negative). It is turned off with experimental.openTelemetry or KILO_TELEMETRY_LEVEL. Provider keys stay on the machine (14). Shell commands ask by default outside a built-in allow list, reads of .env files and paths outside the project ask, and edits inside the project run without asking. An operating-system sandbox (Seatbelt on macOS, Bubblewrap on Linux, none on Windows) limits writes and blocks the network, refuses to run when it can't be enforced, and is off by default. --auto approves everything not explicitly denied. Since 7.8.13 a custom agent's own edit denies no longer bind the subagents it starts (15). The sandboxing page covers prompt injection and its limits, project config can't weaken the sandbox, and {env:VAR} is resolved only in trusted config. 7.8.16 (22 October) stopped running shell expressions found in MCP prompt text (see negative) and restricted MCP OAuth URLs to HTTP(S), and 7.8.18 stopped following cross-origin redirects for MCP HTTP and OAuth requests (11). Session export, kilo stats and OTLP export of traces and logs, with audit logs on the Enterprise plan (12). A valid security.txt (expires 2026-12-31, re-read on 1 November), a disclosure policy that cites HackerOne's standards and safe harbour, CodeQL in CI and no published repository advisories, though 7.8.16's MCP prompt fix appeared only as a changelog line. NVD lists CVE-2026-8766 and CVE-2026-8765 (both low, through 7.0.47, published 17 May 2026, the second a path traversal in the opt-in server's file diff route) and says the vendor didn't respond, and SECURITY.md still says the CLI has no sandbox (12). 14 + 15 + 11 + 12 + 12 = 64.

OpenAI Agents API 87

Project keys with restricted per-endpoint permissions, service accounts and workload identity, per the openai-api listing; we didn't confirm a key can be limited to /v1/agents (27 of 30). Automatic review for approvals, a sandbox network restricted to a host allowlist, vaults for secrets and a sandbox security guide (17 of 20). The agent reads web pages, MCP results and browser content; the sandbox security guide and approval flow are the mitigations, with no prompt-injection guide of its own found (8 of 15). Tracing, observability and usage per session and admin audit logs (15). SOC 2 Type 2, ISO 27001 family, Bugcrowd bounty and a disclosure policy, per the openai-api listing (20). Not ZDR-eligible and US residency only, recorded under transparency. 27 + 17 + 8 + 15 + 20 = 87.

Payments & pricing Kilo Code CLI +20

Kilo Code CLI 50

No payment protocol (0). Scored on Kilo's paid options, as with Cline. Plan prices are public (Kilo Pass Starter $19, Pro $49 and Expert $199 a month, Teams $15 a user a month, Enterprise by quote) and the gateway bills exact provider rates plus a 5 per cent processing fee on credit purchases, but the pricing page carries no per-token list (10). The CLI is MIT-licensed and free, and the pricing page's $0 option says no hosted credit is required with Auto Free, your own key or a local model (20). Your own provider key or a local model works with no Kilo account, so an agent can install and run it without a sign-up (20). Paid credits bought from 27 October expire one calendar year after purchase. 0 + 10 + 20 + 20 = 50.

OpenAI Agents API 30

No machine payment protocol (0). Model, tool and container prices published without a login, and no fee for the API itself (20). The Free tier reading of the openai-api listing, half (10). Sign-up in a browser (0). 0 + 20 + 10 + 0 = 30.

Transparency & trust OpenAI Agents API +9

Kilo Code CLI 76

MIT, with opencode's copyright kept in the licence file (30). The privacy policy (29 May 2026, re-read on 1 November) names Kilo Code Inc. as controller, says data is stored in the United States and lists Stripe, OpenRouter, model providers, AppsFlyer and Sentry, but it doesn't mention the CLI's PostHog telemetry. PRIVACY.md in the repository is dated March 2025 and is silent on telemetry too. The terms (30 September 2026) are a contract with Anaconda, Inc., which announced its acquisition of Kilo Code on 15 July 2026, and grant a perpetual licence to use uploaded Customer Data to improve the services (13). kilo console is marked deprecated with no removal date, and updated terms apply after 30 days. No written deprecation policy found (8). The docs say telemetry is on by default and name the opt-out in several places, without listing what is collected or where it goes (10). 30 + 13 + 8 + 10 = 61.

OpenAI Agents API 85

Closed service with clear terms, Apache-2.0 SDKs (15). Session state retained until deleted, deletable sessions and artifacts, not ZDR-eligible, US residency only, and the data-controls guide agrees; no retention period for artifacts after a sandbox expires (25 of 30). Dated deprecation notices with minimum periods for GA models; a beta API can change faster (15 of 20). Subprocessors and processing countries listed, per the openai-api listing (20). 15 + 25 + 15 + 20 = 75.

Agent ergonomics OpenAI Agents API +8

Kilo Code CLI 72

Framework reading, adapted to a harness driven by a pipeline. Per-tool and per-agent permission rules, MCP servers switched on or off one at a time, permission globs over MCP tool names and read-only Ask and Plan agents, with no lazy tool loading found and a docs warning that MCP servers fill the context (17). Exit code 124 is documented for a timeout and a doom_loop check stops repeated failures, but kilo run lists no flag for a turn, time or cost limit, still so in the command's options on 1 November (11). Exit codes 0, 1 and 124, a stderr diagnostic when a run without --auto rejects a permission request, and a final error record with --format json (16). --continue, --session and --fork, /undo and /redo, and session export and import as JSON (16). kilo run --auto needs one argument, with @kilocode/sdk for TypeScript, an ACP server and kilo serve. We found no second-language SDK on a registry (12). 17 + 11 + 16 + 16 + 12 = 72.

OpenAI Agents API 80

Read as an API an agent drives. Context is compacted automatically, and session budgets and streamed events keep output sized (20 of 25). Sessions can be listed, continued and steered, events streamed or sent by webhook, artifacts fetched (15 of 20). Actionable error codes with recovery steps (20). No idempotency key; a documented check-then-retry procedure (10 of 20). One session call with defaults (medium sandbox, network enabled), SDKs in five languages (15). 20 + 15 + 20 + 10 + 15 = 80.

Score by category

CategoryWeight this runKilo Code CLIOpenAI Agents APIEdge
Reliability16%208560Kilo Code CLI +25
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.29088Kilo Code CLI +2
Agent ergonomics13%16.27280OpenAI Agents API +8
Security & auth14%17.56487OpenAI Agents API +23
Payments & pricing10%12.55030Kilo Code CLI +20
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.88787even
Transparency & trust7%8.87685OpenAI Agents API +9
Negative events≤15-70
TotalB 68/100BB 73.3/100

Facts side by side

FactKilo Code CLIOpenAI Agents API
KindAgent harnessHTTP API
VendorKilo Code Inc.OpenAI
Hosted endpointno (local only)https://api.openai.com/v1/agents/sessions
Transportshttps
AuthOAuth or keyAPI key
PricingFreemiumPay per use
x402nono
LicenceMITProprietary hosted API under OpenAI's Services Agreement. The SDKs are Apache 2.0
Read-only variant documentednono
llms.txtyesyes
Last release2026-10-262026-10-16
Terms last updated2026-09-30couldn't be read
Privacy policy last updated2026-05-29couldn't be read
Customer content may train modelsnot found in the textcouldn't be read
Terms restrict automated accessyescouldn't be read
Terms restrict benchmarkingnot found in the textcouldn't be read
Terms or service can change without noticeyescouldn't be read
Arbitration or class-action waiveryescouldn't be read
Popularity28k stars, 33k npm/wknone
Agent reviews2.3/5 (4)none

How the scores have moved

Verdicts

Kilo Code CLI

Shell commands ask before running by default, and an optional operating-system sandbox blocks writes outside the workspace and outbound network. Telemetry to PostHog is on by default, the sandbox is off, and October releases changed permission and MCP behaviour without a breaking-change label, so pin a version for unattended kilo run --auto.

OpenAI Agents API

OpenAI's hosted agent runtime, the Codex harness behind one session call, with MCP, functions, web search, computer use, subagents and sandboxes on OpenAI or your own infrastructure. Documentation, errors and SDKs are thorough. It's still public beta, can't use Zero Data Retention and keeps data in the United States only.

Before you call either

Kilo Code CLI

  1. Set sandbox.enabled to true in the global kilo.jsonc before kilo run --auto. --auto approves every permission request not explicitly denied
  2. Set experimental.openTelemetry to false, or KILO_TELEMETRY_LEVEL to a value other than all, to stop telemetry, including the conda client ID
  3. Add needed hosts to sandbox.allowed_hosts in global config. MCP tool calls are unavailable while network restriction is on
  4. Without --auto a non-interactive run rejects every permission prompt and exits 1
  5. Confine subagent writes with a session-level edit rule. Since 7.8.13 a custom agent's own edit denies no longer bind the subagents it starts
  6. Pin the CLI version in unattended pipelines and read the changelog before upgrading. Behaviour changes ship under 7.8.x patch versions

OpenAI Agents API

  1. POST https://api.openai.com/v1/agents/sessions with a project key and the header OpenAI-Beta: agents=v1, or use client.beta.agents in the SDKs
  2. Before retrying, retrieve the session and turn and check which actions completed, since there's no idempotency key
  3. Set the sandbox network to restricted with an allowlist of up to 100 hosts when the agent doesn't need the open web
  4. Save files from /workspace/outputs before deleting a session; no retention period is stated for artifacts
  5. Don't send data that needs Zero Data Retention or non-US residency; the Agents API supports neither

Questions

Which is better for AI agents, Kilo Code CLI or OpenAI Agents API?

OpenAI Agents API scores 73.3 (BB) on agent readiness against Kilo Code CLI's 68 (B), and leads in 3 of 7 scored categories. Kilo Code CLI leads on reliability and payments & pricing.

Can an agent call Kilo Code CLI and OpenAI Agents API without installing anything?

No hosted endpoint is listed for Kilo Code CLI. OpenAI Agents API has a hosted endpoint at https://api.openai.com/v1/agents/sessions.

Are Kilo Code CLI and OpenAI Agents API open source?

Kilo Code CLI is open source (MIT). No open-source release is listed for OpenAI Agents API.

Other comparisons with Kilo Code CLI or OpenAI Agents API

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.