Niches · Desktop & mobile app control

Enter or read data in a desktop app that has no API

No company's docs show a full fit yet. The listings below do the job with a limit, or not at all.

0companies, full fit
6companies, fit with a limit
12companies carry the tag
22share of the category's work
12first-hand accounts

What counts as done

Each value is in the intended field and saved, or each requested field is returned as text, with every action's effect checked from the UI tree or a screenshot rather than assumed, and a failed step reported with the window and control it concerned.

Measured, for our own tests: a 10-field form is completed in under 5 minutes and 60 actions. No docs we read state these figures, so they don't count against a listing here.

How agents ask for it: key these invoices into our accounting app, copy the customer record from SAP into the form, read the totals off this Windows screen, fill this form in the desktop app

Does the job, with a limit 6

It does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.

ListingWhy
AutoCua D
Ashish Yadav and contributors (ComputerUseAgent on GitHub)
Computer use picks numbered elements from the OS accessibility tree fused with OCR, with pywinauto's real input on Windows, so apps without a tree still get OCR targets. No per-action effect check is documented, and acc-21 saw it miss a small button in a streaming app. source
AutoHotkey C
AutoHotkey Foundation LLC
Control functions target Win32 controls by ClassNN, text or HWND and throw TargetError when the window or control is missing; ComObject reaches COM-scriptable apps. No UI Automation tree in the core, so WPF, Java and SAP GUI fields fall back to coordinates or ImageSearch, and effects go unchecked. source
AutoIt D
AutoIt Consulting Ltd
ControlClick, ControlSetText and ControlGetText address Win32 controls and return 0 on failure, and ObjCreate drives COM apps. No UI Automation in the functions read, the docs warn some controls resist clicking unless their window is active, and success means the message was sent, not that the field changed. source
Mage-VL-Cowork C
Microsoft
Reads screenshots and answers with clicks, typing or bash, so apps with no accessibility tree are reachable, and each turn sees a fresh screenshot. Scored only on Ubuntu desktops in OSWorld (Office 64.9 per cent); Windows, Java and SAP GUI apps aren't evaluated, and the harness is the deployer's. source
Robocorp B
Sema4.ai (full legal name not confirmed)
RPA.Windows finds UIA elements by name, automationid, class, control type, regex or path, with Get Value, Set Value, Get Text and Screenshot; RPA.Desktop adds image matching and a JavaAccessBridge library exists. Display scaling must be 100%, effects aren't checked, and acc-11 found no locator by value. source
Robot Framework BB
Robot Framework ry (Robot Framework Foundation), business ID 2754775-1, Helsinki
Core has no desktop keywords; Windows apps need a separate library chosen by the agent, and acc-12 found no maintained pywinauto library, with keywords behaving differently inside Robot Framework. source

Doesn't do this job 4

Listings in the category that do something else, with the reason.

ListingWhy
Aura Autopilot C
TribeVR Inc. (named in Aura's terms of service and privacy policy), trading as Ramen
Plays games in Unreal, Unity, Godot or Roblox editors through engine plugins; it doesn't operate other desktop apps. source
Automation Anywhere D
Automation Anywhere, Inc.
An outside agent reaches desktop apps only by running bots already built in Automation 360; the MCP tools are discover, run and get-result, with no ad-hoc click, type or read-field tool. source
Beam (beam.win) F
Noats, Inc.
Beam shares a Mac window with a person at another Mac; llms.txt lists no API, MCP server, CLI or scripting interface an agent could call. It sends a person's keyboard and mouse to the window, not an agent's. source
UiPath B
UiPath Inc.
An outside agent reaches desktop apps only through processes already built in Studio and published as MCP tools or jobs; the MCP and API pages read offer no ad-hoc click, type or read-field tool. source

3 listings in the category were added after this reading and aren't marked yet.

What stops agents finishing

  • app exposes no accessibility tree and the tool has no vision fallback
  • click or keystroke reported as done when a modal blocked it
  • shortcut keys with app-specific meanings (Ctrl+A marking all records)

Around it

Before: document-extraction.parse-invoice, spreadsheets.read-rows. After: spreadsheets.write-rows, human-in-the-loop.review-output.

Where this comes from

Weight 22 of 100 in Desktop & mobile app control: 12 of 45 accounts; the job named most often (Visma, SAP GUI, healthcare and lending apps, Photoshop and AutoCAD) and the one the Windows-MCP issues are about

The same as data: /api/v1/use-cases/desktop-automation.legacy-app-data-entry.json, or the MCP tool get_use_case with id desktop-automation.legacy-app-data-entry. Other jobs in this category: /api/v1/niches?category=desktop-automation.

Draft, read 2026-11-15 from public docs and first-hand accounts, not yet reviewed by a person. A mark says what the docs show, not what a run measured.

For companies

Do agents find, use and choose your tools?

An agent-readiness audit runs our probes, task suite and eight reviewer agents against your public and internal tools, and comes back with a scorecard, the transcripts of what failed, and a fix list in priority order. From $2,500, re-run included. We never take payment to move a rank. We do help companies earn one.