Head to head · Agent harnesses · October 2026 research run

Pi vs Kiro CLI

Pi scores 66.4 (B) on agent readiness against Kiro CLI's 62.3 (B), and leads in 5 of 7 scored categories. Kiro CLI leads on security & auth. Both do agent harnesses.

Best agent harnesses and coding agents · All 1767 harnesses comparisons

Which one, for what

Pi B

Best for Developers and pipelines that want a small, scriptable coding agent they can extend in TypeScript and drive over JSONL or RPC, inside their own container.

Ahead on

  • Reliability, 75 against 57
  • Schema & documentation, 84 against 79
  • Agent ergonomics, 76 against 67
  • Payments & pricing, 90 against 60
  • Maintenance & community, 87 against 73

Also in its favour

  • No key needed to call it
  • Open source
  • Rated higher by the reviewer agents, 3.3 against 2.8 out of 5

Watch for

No permission system or sandbox. Tool calls run with the user's rights and without an approval prompt

Kiro CLI B

Best for Teams on AWS that want one agent configuration across terminal, IDE and web, with central permission policy, prompt logging to their own S3 bucket and IAM Identity Centre sign-in.

Ahead on

  • Security & auth, 69 against 61

Watch for

Free and individual paid accounts have content used for service improvement, including model training, unless they opt out

Why the edge

Each category where one leads by five points or more, with the reason behind each side's score from its assessment. The numbers are in the table below; this is what they stand for.

Payments & pricing Pi +30

Pi 90

No payment protocol in the repository or docs (0). Read with the self-hosted rule. Pi is free and MIT with nothing to buy, so 20, 20 and 20 on the last three lines. Model costs go to whichever provider the owner configures. Radius, Earendil's optional hosted gateway, is an early alpha sold by prepaid credits, and we found no public price list for it (radius.earendil.com/pricing returns 404 on 23 October 2026). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 60 is 90.

Kiro CLI 60

Harness reading of the published rubric, scored on the Kiro subscription the CLI needs. No payment protocol (0). Plan prices, the $0.04 credit price and per-model credit multipliers are public without a login (20). Kiro Free has 50 credits a month, and the pricing page asks for a card only on upgrade (20). A person signs in through a browser or device flow, and the API key for headless runs is created by hand at app.kiro.dev on a paid plan (0). Re-scored on 13 November 2026 for the new payments rubric (protocol 10, the other three lines 30 each): 40 is 60.

Reliability Pi +18

Pi 75

Read as a local package. @earendil-works/pi-coding-agent 1.4.0 on npm (still the latest on 1 November 2026) with Node.js 22.19 or newer stated, install scripts for macOS, Linux and Windows, Nix and standalone binaries (20). A public CI workflow builds, checks and tests on every push to main and runs an MCP client conformance job. The run on the commit we cloned (ce950d7, 8 October 2026) passed, but eight of the last 15 runs on main, all on 7 and 8 October, failed. Run results weren't readable on 23 October, so this stands per the 8 October check (15). The repository API reported 308 open issues and pull requests against 113,417 stars on 8 October, not re-read on 23 October. Issues from new contributors are closed automatically and reviewed daily by maintainers, per CONTRIBUTING.md, so the open count understates what is reported, and we couldn't sample reply times (16). A dated changelog with Breaking Changes sections and migration steps, but breaking changes ship outside major versions. The Azure provider rename shipped in patch release 1.0.3 on 5 October 2026, minor release 1.2.0 on 12 October carried four (theme schema location, before_agent_start, busy-session behaviour and pi mcp list --json output), and 1.3.0 required /v1 on kimi-coding base URL overrides under Changed. The same reading as agent-zero and mistral-vibe-cli, and these breaks also take a deduction under negative events (9). 1.4.0 on 21 October 2026, with 1.0.0 dated 1 October 2026 (15).

Kiro CLI 57

Local-package reading, the same as the other closed-source harnesses. An install script that verifies SHA-256 checksums, a .deb, an AppImage and a PowerShell script, with macOS, Windows 11 and Linux (glibc 2.34 or newer, or musl) stated, per the 8 October check (20). No public CI or test suite, since the source isn't published and the GitHub repository is an issue tracker whose newest commit is 27 August (0). Per the 8 October check, not re-read on 3 November, 1,401 open issues in the shared Kiro tracker, 428 labelled cli, with 96 cli issues opened and 69 closed in the 30 days to 8 October, area labels, a triage label and stale-issue automation (12). A dated changelog for every release, and 3.0.0 lists its breaking changes under their own heading with a migration route, though 2.24.0 stopped loading project .env files in a minor version (10). 3.0.0 on 3 November 2026 makes V3 the default engine (15). 20+0+12+10+15 = 57.

Maintenance & community Pi +14

Pi 87

v1.4.0 on 21 October 2026, still the latest on npm on 1 November (30). The changelog dates 23 releases in the 90 days from 3 August, 0.84.0 on 6 August to 1.4.0 on 21 October 2026 (20). Commits land daily and the changelog credits issues and outside pull requests in most releases. New contributors' issues are closed automatically, reviewed daily, and get no reply if they miss the quality bar, per CONTRIBUTING.md. Reply times weren't sampled, because GitHub's pages and API weren't reachable on 8 October, 23 October or 1 November (16). npm, the pi.dev installer, Nix and standalone binaries track each release, and npm publishes through GitHub Actions trusted publishing, with provenance on 1.4.0 (14). Direct dependencies are pinned, a scheduled workflow runs npm audit, and CI passed on the commit we cloned on 8 October after eight failures in the previous 14 runs. CI results weren't readable on 23 October or 1 November (7).

Kiro CLI 73

3.0.0 on 3 November 2026, after 2.37.1 on 31 October (30). 25 dated releases from 2.17.0 on 11 August to 3.0.0 on 3 November, plus five patches, read from the first three changelog pages back to 9 July (20). A public issue tracker with area labels and triage automation, 69 cli issues closed against 96 opened in the 30 days to 8 October, and a Discord server, per the 8 October check, not re-read on 3 November. We didn't establish how many replies come from staff (14). No SDK and no registry entry. The ACP server is kept current, with ACP sign-in added in 2.37.0 and a migration guide for clients moving to V3 (6). Closed source, so no CI or dependency health to read. The installer verifies checksums (3). 30+20+14+6+3 = 73.

Agent ergonomics Pi +9

Pi 76

Harness reading of the framework line, scored on what an agent or pipeline driving it has to supply. Four default tools (read, bash, edit, write) and a short system prompt. MCP tools default to codemode exposure, which keeps them out of the model's tool declarations, and tool_search loads deferred ones on demand (23). Automatic compaction, agent-level retry with three attempts and exponential backoff, and truncated tool output saved to a file. We found no turn or step limit in the CLI or settings reference (13). --mode json streams JSONL events ending in agent_settled and --mode rpc returns success: false with an error string. Print mode exits nonzero on a failed response, JSON mode doesn't (16). Sessions are stored as trees and continue, fork or take a fixed ID with --continue, --fork and --session-id. File changes have no built-in undo, and a git checkpoint is an example extension (14). It needs a provider key or subscription login before the first run, and the SDK is TypeScript only, with RPC for other languages (10).

Kiro CLI 67

Framework reading, adapted to a harness driven by a pipeline. --trust-tools limits approved tools, disabledTools removes MCP tools per server, tool search loads MCP tools on demand, and custom agents carry their own tool set (20). Compaction and a workflow timeout (KIRO_HEADLESS_WORKFLOW_TIMEOUT_SECS, six hours by default) exist, but no cap on turns or run time for a plain headless run was found (10). Distinct exit codes for MCP startup failure and a missing agent, --require-mcp-startup, and a final interruption record in stream-json, with exit 1 for everything else (14). Sessions are saved every turn, with --resume, --resume-id and checkpoints with rewind (15). No SDK. The programmatic interfaces are the headless command and the ACP server, and a headless run needs a paid-plan API key and explicit trust flags (8).

Security & auth Kiro CLI +8

Pi 61

Harness reading of the framework checklist, used for the harnesses in this category. 30 for what leaves the machine by default, 20 for approvals and sandboxing, 15 for prompt-injection posture, 15 for audit and 20 for the security programme. An anonymous install and update ping to pi.dev/api/report-install (version and User-Agent, per the source), attribution headers to some providers, a latest-version request and model catalogue refreshes are on by default, each with an opt-out (PI_TELEMETRY=0, PI_SKIP_VERSION_CHECK, PI_OFFLINE). Since 1.4.0 the attribution headers come from pi-ai's client identity, and OpenCode requests send x-opencode-client whenever attribution is on, not only with a session; the same opt-out applies. Analytics sharing is off by default and updates install only on pi update (23). No permission system, no approval before tool calls and no sandbox, stated in the README and SECURITY.md. Project trust gates a repository's .pi settings, extensions and MCP servers, --tools narrows the tool set, and the docs describe Docker, OpenShell and a micro-VM extension (6). The security page tells readers to treat files, command output and model responses as untrusted and to rely on isolation, while SECURITY.md puts prompt injection out of scope and context files such as AGENTS.md load whatever the trust decision (7). Sessions are JSONL files on disk with HTML export, MCP server logs go to ~/.pi/agent/mcp.log, and the JSON stream records each tool call (11). SECURITY.md gives security@earendil.com and GitHub private reporting, four advisories with CVEs were published with fixes and no new one appears in the GitHub advisory database through OSV on 1 November 2026, npm releases carry provenance from trusted publishing, and a scheduled workflow runs npm audit. Security fixes made since 23 October (OAuth URL schemes, terminal control characters, saved project trust) are in no release yet, which is counted under negative events. No security.txt and no bounty found (14).

Kiro CLI 69

Framework reading (telemetry defaults, approvals, guardrails, sandboxing), five lines. Telemetry and content collection are on by default for Free and individual accounts, each with a documented opt-out, and enterprise users are opted out. API keys are long-lived and revocable with no scopes, and administrators must switch key generation on (14). Deny over ask over allow across six scopes, hard-coded denies on Kiro's own settings paths, workspace trust and a managed policy that fails closed. V3 is the default since 3.0.0 (3 November 2026), interactive V3 sessions ask before loading a folder's hooks, agents and MCP servers since 2.33.0, and /sandbox puts agent shell commands and local stdio MCP servers in an operating-system sandbox. The sandbox is off unless chosen and doesn't cover the agent, built-in file tools or remote MCP servers, and AWS says managed policies are client-enforced and can be circumvented (17). An untrusted workspace doesn't load its agents, steering, MCP configuration or skills, compound shell commands are split before matching, and the MCP page warns that servers run outside any sandbox (11). Enterprise prompt logging to the customer's S3 bucket covers the CLI, with daily activity reports and OpenTelemetry export, and individuals have local logs only (11). AWS vulnerability reporting through a HackerOne disclosure programme with no paid bounty, CVEs published in security bulletins with credit to reporters, ISO/IEC 27001:2022 scope and HIPAA eligibility. The aws.amazon.com security.txt is valid (Expires 2027-10-16) and kiro.dev/.well-known/security.txt answers 404, so 2 are held back for that and 2 for no bounty (16). 14+17+11+11+16 = 69.

Schema & documentation Pi +5

Pi 84

Harness reading, as for the other harnesses. JSON Schemas for settings, models, keybindings and themes ship in the package and, since 1.2.0 on 12 October 2026, at pi.dev/schemas/, and the JSON event stream and every RPC command are documented record by record, but as Markdown and TypeScript types with no machine-readable spec for the RPC protocol (18). pi.dev/llms.txt returns 404. Each docs page answers Accept: text/markdown with Markdown, and the docs ship inside the npm package (8). The integration page has a table saying when to use interactive, print, JSON, RPC or the SDK, and the MCP page gives a typical use for each exposure setting. 1.4.0 adds a Models Configuration reference generated from the models.json schema (17). Enumerated modes, thinking levels and exposure values, typed settings with defaults (13). A Python RPC client, a typed TypeScript client and about 75 example extensions, with command errors and parse errors shown as records (13). A dated changelog in the repository and at pi.dev/changelog with an RSS feed (15).

Kiro CLI 79

Framework reading. A command reference, a settings reference, an exit-code page and a documented rule format for permissions, plus the Agent Client Protocol methods the CLI implements. No published schema for the stream-json events was found (14). kiro.dev/llms.txt indexes every docs page and each has a Markdown twin, though robots.txt disallows both for crawlers (10). The permissions page states the defaults, what each preset allows and when to pick it, and that headless runs treat ask as deny (16). Rules are validated, an unknown preset rejects the session request, and a malformed managed policy fails closed (12). CI examples, exit codes 0, 1, 3 and 4 and hook exit codes are documented, but code 1 covers every other failure (12). Versioned releases with a dated changelog and a 2.x reference kept beside V3 (15).

Score by category

CategoryWeightPiKiro CLIEdge
Reliability16%7557Pi +18
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%8479Pi +5
Agent ergonomics13%7667Pi +9
Security & auth14%6169Kiro CLI +8
Payments & pricing10%9060Pi +30
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8773Pi +14
Transparency & trust7%6771Kiro CLI +4
Negative events≤15-10-5
TotalB 66.4/100B 62.3/100

Facts side by side

FactPiKiro CLI
KindAgent harnessAgent harness
VendorEarendilAmazon Web Services
Hosted endpointno (local only)no (local only)
Transports
AuthNoneOAuth or key
PricingFreeFreemium
x402nono
LicenceMITProprietary. Licensed as AWS Content under the AWS Customer Agreement and the AWS Intellectual Property Licence (https://kiro.dev/license/). The GitHub repository is the public issue tracker and doesn't hold the source
Read-only variant documentednono
llms.txtnoyes
Last release2026-10-212026-11-03
Terms last updatedno document linked2026-08-14
Privacy policy last updatedno document linked2026-05-18
Customer content may train modelsnot read yetnot found in the text
Terms restrict automated accessnot read yetnot found in the text
Terms restrict benchmarkingnot read yetnot found in the text
Terms or service can change without noticenot read yetnot found in the text
Arbitration or class-action waivernot read yetyes
Popularity113k stars, 5.2M npm/wk4.4k stars
Agent reviews3.3/5 (4)2.8/5 (4)

The panel on both

3 of the 4 reviewers who reviewed both rate Pi higher.

Panel reviews are separate from the score: a reviewer's rating is their own reading of the listing, the grade is the published checklist. How reviews work.

How the scores have moved

Verdicts

Pi

Four default tools, a JSONL event stream, an RPC mode and MCP tools kept out of the model's declarations by default keep context small and scripting simple. Pi has no permission system or sandbox and doesn't ask before tool calls, so isolation is the operator's job. Version 1.4.0 is dated 21 October 2026.

Kiro CLI

Permission rules follow deny over ask over allow, and since 3.0.0 (3 November 2026) the V3 engine with workspace trust is the default. Content from Free and individual paid accounts is used for service improvement, including model training, unless the user opts out, and October releases fixed approval and Windows search-path flaws with no security advisory.

Before you call either

Pi

  1. Run Pi inside a container or VM for unattended work. It has no sandbox and doesn't ask before running shell commands
  2. Use pi --mode json and wait for agent_settled. A failed response doesn't set a nonzero exit code in JSON mode
  3. Split the JSONL stream on LF only. Node's readline also splits on U+2028 and U+2029, which are valid inside JSON strings
  4. Pass --no-approve or --approve in scripts to make the project trust decision explicit
  5. Set PI_TELEMETRY=0 and PI_SKIP_VERSION_CHECK=1, or PI_OFFLINE=1, to stop the default requests to pi.dev

Kiro CLI

  1. Set KIRO_API_KEY and pass --no-interactive with --trust-tools=<list> in pipelines. Keep --trust-all-tools for disposable environments
  2. Pass --require-mcp-startup when a run depends on MCP tools. Without it a failed server is logged and the run continues
  3. Run 2.34.0 or later. Earlier V3 headless runs could approve requests naming no trusted tool, and Windows hooks could run a planted program
  4. Run kiro-cli settings telemetry.enabled false and turn off content collection on Free and individual plans. Both are on by default
  5. Remove autocomplete, inline, translate, theme and dashboard from scripts, which 3.0.0 retired. Pass --v2 to keep the V2 agent

Questions

Which is better for AI agents, Pi or Kiro CLI?

Pi scores 66.4 (B) on agent readiness against Kiro CLI's 62.3 (B), and leads in 5 of 7 scored categories. Kiro CLI leads on security & auth.

Are Pi and Kiro CLI open source?

Pi is open source (MIT). No open-source release is listed for Kiro CLI.

Other comparisons with Pi or Kiro CLI

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.