Niches · Desktop & mobile app control

Hand a screen-reading agent an open-ended task on a computer

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

0companies, full fit
2companies, fit with a limit
103companies carry the tag
12share of the category's work
7first-hand accounts

What counts as done

The goal's end state is visible on screen or in files, the agent stops and asks before irreversible or credentialled steps, and every action is logged with its screenshot.

Measured, for our own tests: a 20-step office task completes in under 15 minutes. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: set up this app for me, do this on my computer, install and configure the VPN client

Does the job, with a limit 2

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
AutoCua D
Ashish Yadav and contributors (ComputerUseAgent on GitHub)
Takes a goal in words on macOS, Windows or Linux, saves the chat with screens, and sends no telemetry. But it clicks the affirmative on macOS permission dialogs, lets the model answer UAC prompts, documents no VM setup, and acc-23 found Telegram control open to any chat until 4 November. source
Mage-VL-Cowork C
Microsoft
66.0 per cent on 361 OSWorld-Verified tasks in 50 steps (Microsoft's two-run average), and the card mandates a disposable sandbox. The reference harness asks before nothing (the card tells deployers to add confirmation), loading needs trust_remote_code and BF16 on one GPU, and acc-41 found no FP8 build. source

Couldn't check 2

The docs were closed to us, or silent. These carry no mark either way.

ListingWhy
Automation Anywhere D
Automation Anywhere, Inc.
The pages read don't describe an open-ended screen agent; AI Agent Studio is listed as an API group but wasn't read.
UiPath B
UiPath Inc.
The MCP and agent pages read don't describe an open-ended screen agent an outside agent can hand a goal; UiPath's Studio agent functions weren't read.

Doesn't do this job 6

Listings in the category that do something else, with the reason.

ListingWhy
Aura Autopilot C
TribeVR Inc. (named in Aura's terms of service and privacy policy), trading as Ramen
Plays games in Unreal, Unity, Godot or Roblox editors through engine plugins; it doesn't operate other desktop apps. source
AutoHotkey C
AutoHotkey Foundation LLC
No model or screen-reading agent; it runs scripts an agent writes. source
AutoIt D
AutoIt Consulting Ltd
No model or screen-reading agent; it runs scripts an agent writes. source
Beam (beam.win) F
Noats, Inc.
Beam shares a Mac window with a person at another Mac; llms.txt lists no API, MCP server, CLI or scripting interface an agent could call. source
Robocorp B
Sema4.ai (full legal name not confirmed)
Libraries for scripted automation; no goal-driven screen agent or vision model. source
Robot Framework BB
Robot Framework ry (Robot Framework Foundation), business ID 2754775-1, Helsinki
No model or goal-driven agent; it runs keyword suites. source

3 listings in the category were added after this reading and aren't marked yet.

What stops agents finishing

  • remote-control channel open to anyone who finds it
  • model weights need remote code or a full-precision GPU
  • headless or virtual display detected by the app

Around it

After: human-in-the-loop.screen-handoff, human-in-the-loop.approve-action.

Where this comes from

Weight 12 of 100 in Desktop & mobile app control: 7 of 45 accounts (AutoCua, Mage-VL-Cowork, Pig and Cua threads); demand is clear, completed outcomes are rare in the accounts

The same as data: /api/v1/use-cases/desktop-automation.screen-agent-task.json, or the MCP tool get_use_case with id desktop-automation.screen-agent-task. Other jobs in this category: /api/v1/niches?category=desktop-automation.

Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.