{
  "data": null,
  "kind": "anchor.page",
  "links": {
    "api": "https://www.anchorterminal.com/api/v1/index.json",
    "html": "https://www.anchorterminal.com/niches/decision-models.agent-step-choice/",
    "json": "https://www.anchorterminal.com/niches/decision-models.agent-step-choice/index.json",
    "llms": "https://www.anchorterminal.com/llms.txt",
    "markdown": "https://www.anchorterminal.com/niches/decision-models.agent-step-choice/index.md",
    "mcp": "https://www.anchorterminal.com/mcp",
    "slim": "https://www.anchorterminal.com/niches/decision-models.agent-step-choice/index.min.md"
  },
  "markdown": "Four or more companies' docs show a full fit.\n\n| Companies, full fit | With a limit | Carry the tag | Share of the category's work | First-hand accounts |\n| --- | --- | --- | --- | --- |\n| 5 | 13 | 30 | 7 | 6 |\n\n## What counts as done\n\nA move per call, chosen from the offered options.\n\nMeasured, for our own tests: each move in under 400 ms hosted or 200 ms local, the episode completed at least as often as a rule baseline, without always picking the first option.\n\nHow agents ask for it: which direction should the agent move; click, wait, retry or stop; which link on this page leads to the target; is the game state winning\n\n## Does the job\n\nThe docs show it meets the test above, and nothing we read says otherwise.\n\n- **Clef (Cloudflare, Inc.)**: Images on Workers AI (up to 4 per request, 16 megapixels each) and video through the weights or Clef-omni; Clef-flash median 38.8 ms, Clef 209 ms. No option-order control.\n- **Decision 3.0 (vLLM Semantic Router project)**: Images (several, up to 1.6 megapixels) and video at 2 frames a second; d3-mini answers an image request in 112 ms median, d3 in 290 ms; permutation_average counters first-option bias.\n- **Liquid d1 (Liquid AI, Inc.)**: Up to 8 base64 images a request on the hosted API at 0.20 s p50 (Arize); d1-2B adds video and runs on MLX and llama.cpp locally.\n- **OpenAI Decisions API (OpenAI OpCo, LLC)**: Images as base64 or public URLs, conversation and tool outputs in input, 0.16 s p50 at Arize, and a voice guide for GPT-Live action selection. No option-order control.\n- **pplx-decider (Perplexity AI, Inc.)**: Up to 8 base64 images a question at 2,048 tiles each, a browser-agent cookbook asking six questions a screenshot; images over the tile cap time out at 504 rather than failing fast.\n- **pplx-decider-flash (Perplexity AI, Inc.)**: Images up to 8 a question on the hosted API and the Mac app; the 4B is built for latency, and the Mac app's examples label screenshots locally.\n\n## Does the job, with a limit\n\nIt does the job, but something in the docs or a user's account will get in an agent's way. The limit is named in each row.\n\n- **Celeris-1 Decision (Marqo Inc)**: Up to 8 images a request at 25 megapixels, read in order as image_1, image_2. No published latency for the decision model and no control for option order.\n- **Laya (Convai Innovations Pvt. Ltd.)**: Fast enough at 33 ms on a GPU, but the core checkpoints read text only (images route to the optional vega engine) and issue 131 shows a position bias on score options.\n- **Decider (Mark Marosi (Mapika))**: Images in /v1/systemone since 1.10.0 on the CUDA Python path (GGUF is text only); the README's browser results are 22 click-only MiniWoB++ tasks and issue 9 shows a destructive click. 133 ms on MPS.\n- **Drex Max v1 (Nace AI, Inc.)**: Images need the Python loader on an NVIDIA GPU (up to 4,194,304 pixels each); the GGUF, Q4, Kapitsa and MLX paths read text only. Public vision index 73.92 is the vendor's run. No latency figure for image requests.\n- **elephant-1 (Baseten Labs, Inc.)**: Images are accepted, but the report says the vision encoder is frozen and training ran on text alone, so image decisions are untested by the authors. No per-decision latency on the card.\n- **Kev (Jared Palmer)**: Text only, no image input; a chess demo and the permute endpoint cover order bias, and Kev-4B answers in 18 ms on an H100. Screenshot environments need a text description.\n- **JEV-27B-VL (AutoTrust AI Lab)**: Reads screenshots and camera images: about 150 to 300 ms a decision in the Mario, Tetris and robot-arm demos and 239 ms a robot-arm step, on an 80 GB GPU. Over the 200 ms local target and no order control.\n- **joey-1 (Baseten Labs, Inc.)**: Images are accepted, but the report says the vision encoder is frozen and training ran on text alone, so image decisions are untested by the authors. No per-decision latency on the card.\n- **mammoth-1 (Baseten)**: Images are accepted, but the report says the vision encoder is frozen and training ran on text alone, so image decisions are untested by the authors. No per-decision latency on the card.\n- **Drex 1.5 (Nace AI, Inc.)**: Images (up to 8 PNG or JPEG) are accepted on the Drex Max models only, as an images array or the TypeSafe multipart form. No latency figure for image requests.\n- **RightNow System Zero (RightNow AI, Inc.)**: Images, 16 kHz WAV audio, PDFs and preview video as inline input, each with its own file fee (USD 0.011 an image); a measured latency table exists but no guarantee. Uncalibrated probabilities.\n- **SPEV (Sprag AI, Inc.)**: One inline image per request plus audio and video; no latency figure, one image means no before-and-after comparison.\n- **kettei (Sqwish Ltd, trading as Sqwish Labs)**: kettei-chub and kettei-fat read up to 8 pictures with the context; the default kettei-core is text only and no latency is published.\n- **Strands Decider 2B (Amazon Web Services, Inc.)**: Text first; the Qwen3.5 v1 model answers over images zero-shot with the vision extra (Gemma 4 models are text only), 115 ms on a GPU, and the 55 percent first-answer preference measured by AWS Builder.\n- **Jev (TypeSafe AI, Inc.)**: Images via the TypeSafe-Danger-Zone-Images header and replies under 400 ms through OpenRouter, but the jaggedness page records a lean to the first option and users saw Jev keep picking the first move in a six-room map and fail basic mazes.\n\n- not this job: GLiClass (Knowledgator Engineering Ltd.): Text only; no image input in the pipeline or server.\n- not this job: Granite Switch 4.1 (International Business Machines Corporation): Text and audio only; the model pages document no image input.\n- not this job: Microsoft-Decision-1 (Microsoft): Text and JSON input only; OpenRouter lists text as the sole input modality.\n- not this job: ScaleDown (Carbon ScaleDown, Inc.): Images and PDFs are OCR-extracted to text before classification, so the model never reads pixels; no screenshot decisions.\n- not this job: Vela 2.0 (vLLM Semantic Router project (with KR Labs)): Text only; the card documents no image input.\n\n1 listings in the category were added after this reading and aren't marked yet.\n\n## What stops agents finishing\n\n- first-option bias stalls exploration\n- models fail multi-step reasoning such as mazes\n- images need a CUDA loader on several open models\n\n## Around it\n\nBefore: browser-automation.screenshot, computer-use.read-screen. After: browser-automation.click, computer-use.act.\n\n## Where this comes from\n\nWeight 7 of 100 in Decision models: 6 of 41 accounts, all small experiments (mazes, rooms, games, form filling); weighted below the business uses because none was in production\n\nAs data: https://www.anchorterminal.com/api/v1/use-cases/decision-models.agent-step-choice.json, or the MCP tool get_use_case with id decision-models.agent-step-choice. Draft, read 2026-11-12 from public docs and first-hand accounts, not yet reviewed by a person.\n",
  "meta": {
    "attribution": "Anchor Terminal (https://www.anchorterminal.com)",
    "attributionRequired": true,
    "credit": "CC BY 4.0, credit Anchor Terminal and link the page or API address you took it from (https://www.anchorterminal.com/terms/#data)",
    "docs": "https://www.anchorterminal.com/docs/",
    "generatedAt": "2026-11-16",
    "license": "CC-BY-4.0",
    "method": "https://www.anchorterminal.com/benchmark/",
    "methodology": "0.8",
    "openapi": "https://www.anchorterminal.com/openapi.json",
    "preview": false,
    "run": "2026-10-01",
    "runLabel": "October 2026 research run",
    "terms": "https://www.anchorterminal.com/terms/#data"
  },
  "page": {
    "breadcrumbs": [
      {
        "name": "Home",
        "url": "https://www.anchorterminal.com/"
      },
      {
        "name": "For agents",
        "url": "https://www.anchorterminal.com/agents/"
      },
      {
        "name": "Niches",
        "url": "https://www.anchorterminal.com/niches/"
      },
      {
        "name": "Choose the next step for an agent acting in a UI, game or environment",
        "url": ""
      }
    ],
    "description": "Choose the next step for an agent acting in a UI, game or environment: 5 companies fully do it, 13 with a limit, 30 carry the tag. Every listing in Decision models with the reason and a source.",
    "facts": null,
    "h1": "Choose the next step for an agent acting in a UI, game or environment",
    "image": "https://www.anchorterminal.com/assets/og/default.png",
    "path": "/niches/decision-models.agent-step-choice/",
    "published": "",
    "section": "agents",
    "title": "Choose the next step for an agent acting in a UI, game or environment…",
    "toc": null,
    "updated": "2026-11-15",
    "url": "https://www.anchorterminal.com/niches/decision-models.agent-step-choice/"
  },
  "tokens": {
    "markdown": 1900,
    "slim": 380
  },
  "version": 1
}
