{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/desktop-automation.screen-agent-task/",
    "json": "https://www.anchorterminal.com/niches/desktop-automation.screen-agent-task/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/desktop-automation.screen-agent-task/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/desktop-automation.screen-agent-task/index.min.md"
  },
  "markdown": "No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 0 | 2 | 103 | 12 | 7 |\n\n## What counts as done\n\nThe goal's end state is visible on screen or in files, the agent stops and asks before irreversible or credentialled steps, and every action is logged with its screenshot.\n\nMeasured, for our own tests: a 20-step office task completes in under 15 minutes.\n\nHow agents ask for it: set up this app for me; do this on my computer; install and configure the VPN client\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **AutoCua (Ashish Yadav and contributors (ComputerUseAgent on GitHub))**: Takes a goal in words on macOS, Windows or Linux, saves the chat with screens, and sends no telemetry. But it clicks the affirmative on macOS permission dialogs, lets the model answer UAC prompts, documents no VM setup, and acc-23 found Telegram control open to any chat until 4 November.\n- **Mage-VL-Cowork (Microsoft)**: 66.0 per cent on 361 OSWorld-Verified tasks in 50 steps (Microsoft's two-run average), and the card mandates a disposable sandbox. The reference harness asks before nothing (the card tells deployers to add confirmation), loading needs trust_remote_code and BF16 on one GPU, and acc-41 found no FP8 build.\n\n## Couldn't check\n\nThe docs were closed to us, or silent. These carry no mark either way.\n\n- **Automation Anywhere (Automation Anywhere, Inc.)**: The pages read don't describe an open-ended screen agent; AI Agent Studio is listed as an API group but wasn't read.\n- **UiPath (UiPath Inc.)**: The MCP and agent pages read don't describe an open-ended screen agent an outside agent can hand a goal; UiPath's Studio agent functions weren't read.\n\n- not this job: Aura Autopilot (TribeVR Inc. (named in Aura's terms of service and privacy policy), trading as Ramen): Plays games in Unreal, Unity, Godot or Roblox editors through engine plugins; it doesn't operate other desktop apps.\n- not this job: AutoHotkey (AutoHotkey Foundation LLC): No model or screen-reading agent; it runs scripts an agent writes.\n- not this job: AutoIt (AutoIt Consulting Ltd): No model or screen-reading agent; it runs scripts an agent writes.\n- not this job: Beam (beam.win) (Noats, Inc.): Beam shares a Mac window with a person at another Mac; llms.txt lists no API, MCP server, CLI or scripting interface an agent could call.\n- not this job: Robocorp (Sema4.ai (full legal name not confirmed)): Libraries for scripted automation; no goal-driven screen agent or vision model.\n- not this job: Robot Framework (Robot Framework ry (Robot Framework Foundation), business ID 2754775-1, Helsinki): No model or goal-driven agent; it runs keyword suites.\n\n3 listings in the category were added after this reading and aren't marked yet.\n\n## What stops agents finishing\n\n- remote-control channel open to anyone who finds it\n- model weights need remote code or a full-precision GPU\n- headless or virtual display detected by the app\n\n## Around it\n\nAfter: human-in-the-loop.screen-handoff, human-in-the-loop.approve-action.\n\n## Where this comes from\n\nWeight 12 of 100 in Desktop \u0026 mobile app control: 7 of 45 accounts (AutoCua, Mage-VL-Cowork, Pig and Cua threads); demand is clear, completed outcomes are rare in the accounts\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/desktop-automation.screen-agent-task.json, or the MCP tool get_use_case with id desktop-automation.screen-agent-task. Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Hand a screen-reading agent an open-ended task on a computer",
        "url": ""
      }
    ],
    "description": "Hand a screen-reading agent an open-ended task on a computer: 0 companies fully do it, 2 with a limit, 103 carry the tag. Every listing in Desktop \u0026 mobile app control with the reason and a source.",
    "facts": null,
    "h1": "Hand a screen-reading agent an open-ended task on a computer",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/desktop-automation.screen-agent-task/",
    "published": "",
    "section": "agents",
    "title": "Hand a screen-reading agent an open-ended task on a computer | Anchor…",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/desktop-automation.screen-agent-task/"
  },
  "tokens": {
    "markdown": 1100,
    "slim": 280
  },
  "version": 1
}
