Harness² vs Orca

Harness² 36

0.1.0, first committed on 8 October 2026, with a README fix on 13 October (30). No tags, releases or dated changelog entries (0). 42 pull requests (41 from outside contributors, one from Dependabot) and 6 issues open on 18 October, against one pull request and no issues closed. No outside change is on main, which has three commits by the authors, and reply times couldn't be read (3 of 25). Not on PyPI and no SDK (0). No CI. Dependencies are pinned by version and upstream checkouts by commit, a ruff configuration is in pyproject.toml, and Dependabot's litellm bump of 8 October is still open (3 of 10). 36 in all.

Orca 75

Tags to v1.4.251 and commits on 6 November 2026 (30). Many releases in 90 days (20). GitHub issues and pull requests weren't read (GitHub's pages refuse these sessions), so that line is scored absent (0). Current builds (15). Extensive CI (10). 30 + 20 + 0 + 15 + 10 = 75.

Harness² 20

Local-package reading. Not on PyPI (the release feed for harness2 answers 404). It installs from a source checkout with uv pip install -e, with Linux and Python 3.12 stated (the package allows 3.13), plus Node.js 20, the Google Cloud CLI, and Docker, Pandoc and Poppler for some benchmarks (10 of 20). No CI workflow in the repository. tests/test_workflow.py holds five unit tests over the deterministic demo, which needs no credentials (5 of 25). Six issues and 42 pull requests were open on 18 October, the pull requests mostly fixes by their titles, such as the missing .env.example, report failures, seeds lost between refinement and grading, and resume. None is merged and no issue is closed. Issue text and replies couldn't be read (5 of 25). No tags or changelog (0). Version 0.1.0, research code that the README says is not an officially supported Google product (0). 20 in all.

Orca 58

Local software. Desktop and mobile builds and a Homebrew cask (20). Unit, end-to-end and platform test workflows; pass state not read (15). GitHub issues and pull requests weren't read (GitHub's pages refuse these sessions), so that line is scored absent (0). Tags to v1.4.251, no changelog file (8). Version 1.4 (15). 20 + 15 + 0 + 8 + 15 = 58.

Harness² 51

Framework reading. Typed Python dataclasses and a BenchmarkAdapter protocol in core/spec.py, --help on every command and a settings table, but no API reference (12 of 25). Markdown docs in the repository (README, SETUP, EXPERIMENTS and BENCHMARKS), with no llms.txt (5 of 10). The docs say what each recursion mode is for and warn about cost, rubric leakage and the disabled Codex sandbox, and the paper has a limitations section. Nothing says how to use it outside the three benchmarks (13 of 20). --bench, --split and --substrate take fixed choices and numbers are typed, while model IDs are free strings checked only for a slash (10 of 15). Every command has a worked example and ConfigError messages name the fix, but the quick start's cp .env.example .env fails because the file isn't shipped (8 of 15). Version 0.1.0 with no changelog (3 of 15). 51 in all.

Orca 65

Harness reading. A docs site with CLI orchestration and native chat pages (65).

Harness² 59

Framework reading, adapted to a harness that a pipeline drives. Results land in run folders, not in the caller's context, and an agent reads task summaries, adopted harnesses and reflection notes from runs/ (12 of 25). --only-domains, --max-tasks (per domain), --split-file, --workers and --dry-run size a run before any model call (14 of 20). harness2-check tests dependencies, credentials and configuration without a model call, configuration errors name the remedy, and incomplete runs exit nonzero (14 of 20). Rerunning a command resumes it, saved configuration refuses an incompatible resume, failed grades stay unscored, and editor replies are cached by a fingerprint of their inputs (15 of 20). No model has a default, a Vertex AI project and a gcloud sign-in are needed, only three benchmarks are accepted and there's no SDK (4 of 15). 59 in all.

Orca 70

Harness reading. An orchestrator agent that splits work across workers in their own workspaces, chat or raw CLI, a browser and a CLI in one app (70).

Harness² 49

Apache-2.0 under Google LLC, with each bundled runner's licence and provenance file and a hash list. The WorkBuddy runners carry Tencent's custom licence, which has an EU territorial limitation (26 of 30). The docs list every host setup contacts and say credentials, generated configuration and run artefacts are not release assets. No privacy statement covers the package, and task data goes to the configured model provider (12 of 30). No deprecation policy or notices for the 0.1.0 research release (0). No telemetry found in the source, and the docs don't say either way. OpenCode's telemetry is turned off only in the WorkBuddy configs (10 of 20). 48 in all.

Orca 54

MIT (30). Privacy and telemetry docs on the site (not read) (18 of 30). No deprecation policy (5). Disclosed with an opt-out (15 of 20). 30 + 18 + 5 + 15 = 68.