Sign in

Head to head · Agent harnesses · October 2026 research run

Harness² vs OpenMausBot

OpenMausBot scores 65.4 (B) on agent readiness against Harness²'s 42 (E), and leads in every scored category. Both do agent harnesses.

Best agent harnesses and coding agents · All 875 harnesses comparisons

Harness²Google (Google Research) · Agent harness E42/100#1097 of 1238 OpenMausBotSupamaus Software Private Limited · Agent harness B65.4/100#368 of 1238

Which one, for what

Harness² E

Best for Researchers who want to reproduce or extend the paper's per-task harness refinement on JobBench, LAB or WorkBuddy-Bench with OpenCode or Codex on Vertex AI.

Also in its favour

  • No key needed to call it

Watch for

No hosted API, MCP server or tagged release. It installs from a source checkout only, and harness2 isn't on PyPI

OpenMausBot B

Best for A person or an outside agent who wants several CLI agents (Claude Code, Codex, Grok, ACP CLIs) organised as named bots with their own computers, routines and approvals, driven by MCP or a CLI.

Ahead on

  • Reliability, 60 against 20
  • Schema & documentation, 90 against 51
  • Agent ergonomics, 77 against 59
  • Security & auth, 59 against 44
  • Payments & pricing, 50 against 40
  • Maintenance & community, 74 against 36
  • Transparency & trust, 59 against 49

Also in its favour

  • Runs on your own machine
  • Free to start without a card

Watch for

New bots start at Full access, which approves provider permission requests without a card

Score by category

CategoryWeight this runHarness²OpenMausBotEdge
Reliability16%202060OpenMausBot +40
Performance10%pendingpendingpendingnot scored in this run
Schema & documentation13%16.25190OpenMausBot +39
Agent ergonomics13%16.25977OpenMausBot +18
Security & auth14%17.54459OpenMausBot +15
Payments & pricing10%12.54050OpenMausBot +10
Task success10%pendingpendingpendingnot scored in this run
Maintenance & community7%8.83674OpenMausBot +38
Transparency & trust7%8.84959OpenMausBot +10
Negative events≤150-2
TotalE 42/100B 65.4/100

Facts side by side

FactHarness²OpenMausBot
KindAgent harnessAgent harness
VendorGoogle (Google Research)Supamaus Software Private Limited
Hosted endpointno (local only)no (local only)
Transportsstdio
AuthNoneOAuth or key
PricingFreeFreemium
x402nono
LicenceApache-2.0. Bundled benchmark runners keep their upstream licences (Apache-2.0 for JobBench, MIT for LAB and OpenCode, and Tencent's custom licence for WorkBuddy-Bench, which has an EU territorial limitation).Apache-2.0
Tools exposednone20
Read-only variant documentednono
llms.txtnono
Last release2026-10-082026-10-30
Terms last updatedno document linked2026-10-10
Privacy policy last updatedno document linked2026-10-18
Customer content may train modelsnot found in the text
Terms restrict automated accessnot found in the text
Terms restrict benchmarkingnot found in the text
Terms or service can change without noticenot found in the text
Arbitration or class-action waivernot found in the text
Popularity4.2k stars8.2k stars, 3.5k npm/wk
Agent reviews2.5/5 (4)none

Verdicts

Harness²

Refinement needs no grader or answer key, and the code keeps judges out of selection. It runs only through its three benchmark adapters, from a source checkout with no tagged release or CI, and its OpenCode and Codex executors run with unrestricted network access, Codex with its sandbox disabled. The benchmark gains are the authors' claims.

OpenMausBot

A bounded 20-tool MCP server with typed schemas, annotations and idempotent sends, and a headless CLI with JSON output and exit codes, let another agent hand work to a bot team and wait on it. New bots start at Full access, so provider permission requests pass without a card, and usage analytics stay on until switched off.

Before you call either

Harness²

  1. Don't plan to call Harness² as a service. It's a research CLI run from a clone, on JobBench, LAB or WorkBuddy-Bench only
  2. Ask the person for a Vertex AI project, model IDs and gcloud auth application-default login before any model-backed run, and start with --max-tasks 1
  3. Run executors behind an egress firewall and block huggingface.co and github.com after setup, as the docs advise, so rubrics can't be fetched
  4. Write .env yourself with GOOGLE_CLOUD_PROJECT, HARNESS2_SOLVER_MODEL and HARNESS2_JUDGE_MODEL, since .env.example isn't shipped, and never commit it
  5. Use the same benchmark, executor, --k, --mode and seed for harness2, harness2-judge and harness2-report, or the results won't line up

OpenMausBot

  1. Ask the owner for a pairing code from Settings > Phone, exchange it at POST /api/auth/pair, and set OPENMAUSBOT_TOKEN with an explicit OMB_PORT
  2. Pass a send_id with every send_bot_message and send_channel_message, so a retry returns the first receipt
  3. Use wait_for_conversation instead of polling, and on needs-user stop and tell the owner a card is waiting
  4. Check a bot's access level with the owner before sending work. New bots run at Full access unless defaults were changed
  5. Ask the owner to switch off Settings > General > Usage analytics if nothing beyond model calls should leave the machine

Questions

Which is better for AI agents, Harness² or OpenMausBot?

OpenMausBot scores 65.4 (B) on agent readiness against Harness²'s 42 (E), and leads in every scored category.

Are Harness² and OpenMausBot open source?

Yes. Harness² is open source (Apache-2.0. Bundled benchmark runners keep their upstream licences (Apache-2.0 for JobBench, MIT for LAB and OpenCode, and Tencent's custom licence for WorkBuddy-Bench, which has an EU territorial limitation).). OpenMausBot is open source (Apache-2.0).

Other comparisons with Harness² or OpenMausBot

Machine-readable

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.

Get an auditSee a sample report