{ "meta": { "attribution": "Anchor Terminal (https://www.anchorterminal.com)", "docs": "https://www.anchorterminal.com/docs/", "generatedAt": "2026-10-04", "license": "CC-BY-4.0", "method": "https://www.anchorterminal.com/benchmark/", "methodology": "0.3", "openapi": "https://www.anchorterminal.com/openapi.json", "preview": false, "run": "2026-10-01", "runLabel": "October 2026 research run" }, "tool": { "category": "agent-observability", "endpoint": "https://completionkit.com/mcp", "jsonUrl": "https://www.anchorterminal.com/api/v1/tools/completionkit-evals.json", "kind": "mcp", "listed": "indexed", "liveUrl": "https://www.anchorterminal.com/api/v1/live/completionkit-evals.json", "markdownUrl": "https://www.anchorterminal.com/tools/completionkit-evals.md", "mcpTools": { "check": { "checker": "anchor-check/1.0", "totalTokens": 5722, "counts": { "error": 0, "note": 1, "warn": 124 }, "findings": [ { "rule": "TC06", "severity": "warn", "tool": "datasets_delete", "message": "the description is \"Delete a dataset\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "datasets_get", "message": "the description is \"Get a dataset by ID\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "datasets_list", "message": "the description is \"List all datasets\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "datasets_update", "message": "the description is \"Update a dataset\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "metric_groups_create", "message": "the description is \"Create a metric group\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "metric_groups_delete", "message": "the description is \"Delete a metric group\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "metric_groups_get", "message": "the description is \"Get a metric group by ID\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "metric_groups_list", "message": "the description is \"List all metric groups\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "metric_groups_update", "message": "the description is \"Update a metric group\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "metrics_delete", "message": "the description is \"Delete a metric\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "metrics_get", "message": "the description is \"Get a metric by ID\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "metrics_list", "message": "the description is \"List all metrics\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "prompts_create", "message": "the description is \"Create a prompt\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "prompts_delete", "message": "the description is \"Delete a prompt\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "prompts_get", "message": "the description is \"Get a prompt by ID\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "prompts_list", "message": "the description is \"List all prompts\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "responses_get", "message": "the description is \"Get a specific response\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "runs_delete", "message": "the description is \"Delete a run\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "runs_list", "message": "the description is \"List all runs\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "runs_update", "message": "the description is \"Update a run\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "tags_get", "message": "the description is \"Get a tag by ID\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "tags_list", "message": "the description is \"List all tags\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC06", "severity": "warn", "tool": "tags_update", "message": "the description is \"Rename a tag.\"", "fix": "Add when to use it, what it returns and any side effect, in two or three sentences." }, { "rule": "TC11", "severity": "warn", "tool": "agreements_create", "message": "none of its 7 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "agreements_list", "message": "none of its 4 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "datasets_create", "message": "none of its 3 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "datasets_create_from_url", "message": "2 parameters without a description: name, tag_names", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "datasets_delete", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "datasets_get", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "datasets_update", "message": "none of its 4 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "judges_compare", "message": "none of its 3 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "judges_replay", "message": "4 parameters without a description: dataset_id, judge_model, metric_id, name", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metric_groups_create", "message": "none of its 4 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metric_groups_delete", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metric_groups_get", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metric_groups_update", "message": "none of its 5 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metric_versions_dismiss", "message": "its one parameter, metric_version_id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metric_versions_list", "message": "its one parameter, metric_id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metric_versions_publish", "message": "its one parameter, metric_version_id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metrics_create", "message": "none of its 6 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metrics_delete", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metrics_get", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metrics_suggest_variants", "message": "1 parameter without a description: metric_id", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "metrics_update", "message": "none of its 7 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "prompts_create", "message": "none of its 5 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "prompts_delete", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "prompts_publish", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "prompts_update", "message": "none of its 6 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "provider_credentials_create", "message": "none of its 4 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "provider_credentials_delete", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "provider_credentials_get", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "provider_credentials_update", "message": "none of its 5 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "responses_get", "message": "none of its 2 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "responses_list", "message": "1 parameter without a description: run_id", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "runs_create", "message": "6 parameters without a description: dataset_id, judge_model, metric_ids, name, prompt_id, tag_names", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "runs_delete", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "runs_generate", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "runs_get", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "runs_regrade", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "runs_rerun", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "runs_retry_failures", "message": "none of its 2 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "runs_update", "message": "8 parameters without a description: dataset_id, expected_column, id, judge_model, metric_ids, name and 2 more", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "tags_create", "message": "its one parameter, name, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "tags_delete", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "tags_get", "message": "its one parameter, id, has no description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC11", "severity": "warn", "tool": "tags_update", "message": "none of its 2 parameters has a description", "fix": "Describe each one: format, units, an example, and what happens when it's left out." }, { "rule": "TC12", "severity": "warn", "tool": "metrics_create", "message": "no type for check_config.expected", "fix": "Give every parameter a type (or an enum)." }, { "rule": "TC12", "severity": "warn", "tool": "metrics_update", "message": "no type for check_config.expected", "fix": "Give every parameter a type (or an enum)." }, { "rule": "TC16", "severity": "warn", "tool": "agreements_create", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "agreements_list", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"list\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "datasets_create", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "datasets_create_from_url", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "datasets_delete", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "datasets_get", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"get\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "datasets_list", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"list\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "datasets_update", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "judges_compare", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "judges_replay", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "metric_groups_create", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "metric_groups_delete", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "metric_groups_get", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"get\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "metric_groups_list", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"list\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "metric_groups_update", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "metric_versions_dismiss", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "metric_versions_list", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"list\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "metric_versions_publish", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "metrics_create", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "metrics_delete", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "metrics_get", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"get\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "metrics_list", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"list\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "metrics_suggest_variants", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "metrics_update", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "promptfoo_import", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "prompts_create", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "prompts_delete", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "prompts_get", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"get\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "prompts_list", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"list\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "prompts_publish", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "prompts_suggest_improvement", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "prompts_update", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "provider_credentials_create", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "provider_credentials_delete", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "provider_credentials_get", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"get\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "provider_credentials_list", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"list\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "provider_credentials_update", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "responses_get", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"get\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "responses_list", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"list\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "runs_create", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "runs_delete", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "runs_generate", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "runs_get", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"get\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "runs_list", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"list\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "runs_regrade", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "runs_rerun", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "runs_retry_failures", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "runs_update", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "tags_create", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "tags_delete", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "tags_get", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"get\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "tags_list", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"list\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC16", "severity": "warn", "tool": "tags_update", "message": "no readOnlyHint or destructiveHint", "fix": "Set readOnlyHint: true if it only reads; otherwise set destructiveHint and idempotentHint." }, { "rule": "TC16", "severity": "warn", "tool": "usage_get", "message": "no readOnlyHint or destructiveHint", "fix": "Its name starts with \"get\"; if it only reads, set readOnlyHint: true so harnesses can run it without asking." }, { "rule": "TC21", "severity": "warn", "tool": "metrics_create", "message": "described almost the same as metrics_update (92% of the same words)", "fix": "Say in each description when to use it instead of the other." }, { "rule": "TC23", "severity": "warn", "message": "54 tools", "fix": "Offer a smaller default toolset and let clients opt into the rest." }, { "rule": "TC24", "severity": "note", "message": "54 of 54 tools have no outputSchema", "fix": "Declare outputSchema for tools that return structured data, and return structuredContent that matches it." } ] }, "checkedAt": "2026-10-04T22:22:26Z", "count": 54, "note": "answered without the initialize handshake", "schemaTokens": 5722, "status": "ok", "tools": [ { "name": "prompts_list", "description": "List all prompts", "inputSchema": { "properties": {}, "required": [], "type": "object" } }, { "name": "prompts_get", "description": "Get a prompt by ID", "inputSchema": { "properties": { "id": { "description": "Prompt ID", "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "prompts_create", "description": "Create a prompt", "inputSchema": { "properties": { "description": { "type": "string" }, "llm_model": { "type": "string" }, "name": { "type": "string" }, "tag_names": { "items": { "type": "string" }, "type": "array" }, "template": { "type": "string" } }, "required": [ "name", "template", "llm_model" ], "type": "object" } }, { "name": "prompts_update", "description": "Update a prompt. If the prompt already has runs, this creates a new DRAFT version (current=false) rather than editing in place or publishing — promote it with prompts_publish — so an agent's edits don't go live without a gate. If it has no runs, it is updated in place.", "inputSchema": { "properties": { "description": { "type": "string" }, "id": { "type": "integer" }, "llm_model": { "type": "string" }, "name": { "type": "string" }, "tag_names": { "items": { "type": "string" }, "type": "array" }, "template": { "type": "string" } }, "required": [ "id" ], "type": "object" } }, { "name": "prompts_delete", "description": "Delete a prompt", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "prompts_publish", "description": "Publish a prompt version, making it the current version", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "prompts_suggest_improvement", "description": "Suggest an improved version of a prompt, grounded in a run's test results and judge feedback. Analyzes the run's responses, scores, and reviews, then returns reasoning plus a rewritten template (preserving {{variables}}) and persists it as a Suggestion. Requires a run that has a prompt (not a scoring-only run).", "inputSchema": { "properties": { "run_id": { "description": "The run whose results ground the improvement.", "type": "integer" } }, "required": [ "run_id" ], "type": "object" } }, { "name": "runs_list", "description": "List all runs", "inputSchema": { "properties": {}, "required": [], "type": "object" } }, { "name": "runs_get", "description": "Get a run by ID, including \"metric_averages\": a per-metric breakdown with each metric's average score (or pass rate for checks), how many rows it graded, and how many scored low. Use this to find the metric dragging a prompt down without listing responses.", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "runs_create", "description": "Create a run. Omit prompt_id and provide output_column to score existing outputs by grading a pre-existing dataset column instead of generating new ones.", "inputSchema": { "properties": { "dataset_id": { "type": "integer" }, "expected_column": { "description": "Dataset column holding each row's answer key / ground truth, graded by checks with compare_to \"expected\" and passed to the judge; defaults to \"expected_output\".", "type": "string" }, "judge_model": { "type": "string" }, "judge_temperature": { "description": "Sampling temperature for the judge, 0 to 1. Defaults to 0 so re-judging the same output gives the same score. Raise it only to measure judge variance on purpose; any value above 0 makes the run's scores irreproducible.", "type": "number" }, "max_tokens": { "description": "Cap on generated tokens per row. Leave unset to use the provider client's default, which is what silently truncates long outputs and makes the judge score malformed JSON. Set it to whatever the prompt uses in production so the eval matches.", "type": "integer" }, "metric_group_id": { "description": "Attach the metrics belonging to this metric group (its current metric_ids). Ignored when metric_ids is also given.", "type": "integer" }, "metric_ids": { "items": { "type": "integer" }, "type": "array" }, "name": { "type": "string" }, "output_column": { "description": "Dataset column to grade when prompt_id is omitted; defaults to \"actual_output\".", "type": "string" }, "prompt_id": { "type": "integer" }, "tag_names": { "items": { "type": "string" }, "type": "array" }, "temperature": { "description": "Sampling temperature for generation, 0 to 1. Leave it unset, which is the default, and no temperature is sent at all, so the model applies its own. Most current frontier models refuse the parameter outright; set it only when you are targeting a model that honours it, such as anything served locally through Ollama. A refused value is re-sent without one and the run is flagged temperature_ignored.", "type": "number" } }, "required": [ "name" ], "type": "object" } }, { "name": "runs_update", "description": "Update a run", "inputSchema": { "properties": { "dataset_id": { "type": "integer" }, "expected_column": { "type": "string" }, "id": { "type": "integer" }, "judge_model": { "type": "string" }, "judge_temperature": { "description": "Sampling temperature for the judge, 0 to 1. Defaults to 0 so re-judging the same output gives the same score. Raise it only to measure judge variance on purpose; any value above 0 makes the run's scores irreproducible.", "type": "number" }, "max_tokens": { "description": "Cap on generated tokens per row. Leave unset to use the provider client's default, which is what silently truncates long outputs and makes the judge score malformed JSON. Set it to whatever the prompt uses in production so the eval matches.", "type": "integer" }, "metric_group_id": { "description": "Replace the run's metrics with those belonging to this metric group. Ignored when metric_ids is also given.", "type": "integer" }, "metric_ids": { "items": { "type": "integer" }, "type": "array" }, "name": { "type": "string" }, "output_column": { "type": "string" }, "tag_names": { "items": { "type": "string" }, "type": "array" }, "temperature": { "description": "Sampling temperature for generation, 0 to 1. Leave it unset, which is the default, and no temperature is sent at all, so the model applies its own. Most current frontier models refuse the parameter outright; set it only when you are targeting a model that honours it, such as anything served locally through Ollama. A refused value is re-sent without one and the run is flagged temperature_ignored.", "type": "number" } }, "required": [ "id" ], "type": "object" } }, { "name": "runs_delete", "description": "Delete a run", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "runs_generate", "description": "Start a run. Required for every run, including score-only runs (no prompt): generates responses with the prompt when there is one, otherwise copies the graded dataset column and grades it.", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "runs_regrade", "description": "Re-grade a run's existing responses with its currently attached metrics, without regenerating. Use after attaching or editing metrics on an already-generated run.", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "runs_rerun", "description": "Create and start a fresh copy of a run with the same prompt, dataset, metrics, and settings. Use when the judge changed and you want a clean run instead of mixing versions.", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "runs_retry_failures", "description": "Re-run only the failed responses of a run, optionally limited to specific response ids via \"only\".", "inputSchema": { "properties": { "id": { "type": "integer" }, "only": { "items": { "type": "integer" }, "type": "array" } }, "required": [ "id" ], "type": "object" } }, { "name": "responses_list", "description": "List responses for a run, in row order. Returns {total, limit, offset, returned, responses}. Defaults to 50 rows because full payloads are large: use \"fields\" to drop the bodies, \"min_score\"/\"max_score\" to isolate low scorers, and sort \"score_asc\" to read the worst rows first. For per-metric averages of the whole run use runs_get instead of aggregating here.", "inputSchema": { "properties": { "fields": { "description": "Only return these keys, keeping the payload small. Response keys: id, run_id, input_data, response_text, expected_output, created_at, score, reviewed, reviews, status, attempts, row_index, error. Prefix with \"reviews.\" to trim each review, e.g. [\"score\", \"reviews.metric_name\", \"reviews.ai_score\"]. id is always included.", "items": { "type": "string" }, "type": "array" }, "limit": { "description": "Rows to return; defaults to 50, capped at 500.", "type": "integer" }, "max_score": { "description": "Only rows whose average judge score is at most this. Use with sort \"score_asc\" for failure-mode analysis.", "type": "number" }, "min_score": { "description": "Only rows whose average judge score is at least this.", "type": "number" }, "offset": { "description": "Rows to skip before returning results.", "type": "integer" }, "run_id": { "type": "integer" }, "sort": { "description": "Row order; defaults to \"id\".", "enum": [ "id", "score_asc", "score_desc" ], "type": "string" }, "status": { "description": "Filter by row status: pending, retrying, succeeded or failed.", "type": "string" } }, "required": [ "run_id" ], "type": "object" } }, { "name": "responses_get", "description": "Get a specific response", "inputSchema": { "properties": { "id": { "type": "integer" }, "run_id": { "type": "integer" } }, "required": [ "run_id", "id" ], "type": "object" } }, { "name": "datasets_list", "description": "List all datasets", "inputSchema": { "properties": {}, "required": [], "type": "object" } }, { "name": "datasets_get", "description": "Get a dataset by ID", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "datasets_create", "description": "Create a dataset with CSV data. First row is the header. Two column names are recognized specially: \"expected_output\" is each row's answer key (ground truth) given to the judge and to checks that compare against the row's expected value, and \"actual_output\" is a pre-made output to score in a prompt-less run. Both are overridable per run (expected_column / output_column). Every column is also available to the prompt as a variable.", "inputSchema": { "properties": { "csv_data": { "type": "string" }, "name": { "type": "string" }, "tag_names": { "items": { "type": "string" }, "type": "array" } }, "required": [ "name", "csv_data" ], "type": "object" } }, { "name": "datasets_update", "description": "Update a dataset", "inputSchema": { "properties": { "csv_data": { "type": "string" }, "id": { "type": "integer" }, "name": { "type": "string" }, "tag_names": { "items": { "type": "string" }, "type": "array" } }, "required": [ "id" ], "type": "object" } }, { "name": "datasets_delete", "description": "Delete a dataset", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "datasets_create_from_url", "description": "Create a dataset by downloading CSV from a URL instead of inlining it. Use this for large datasets: pass a public http(s) URL and the server fetches the CSV directly, so the data never has to pass through the tool-call arguments. The URL is SSRF-checked and the download is capped at 10MB. First row is the header; the \"expected_output\" (answer key) and \"actual_output\" (pre-made output) columns are recognized specially, overridable per run.", "inputSchema": { "properties": { "name": { "type": "string" }, "tag_names": { "items": { "type": "string" }, "type": "array" }, "url": { "description": "Public http(s) URL of the CSV file to download.", "type": "string" } }, "required": [ "name", "url" ], "type": "object" } }, { "name": "metrics_list", "description": "List all metrics", "inputSchema": { "properties": {}, "required": [], "type": "object" } }, { "name": "metrics_get", "description": "Get a metric by ID", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "metrics_create", "description": "Create a metric with evaluation criteria. For a deterministic check set metric_type:\"check\" and check_config. Per-kind required keys: value (contains/not_contains/equals), pattern (regex), json_path+expected (json_path_equals), min and/or max (length_bounds); valid_json takes no extra keys. target_path is required when target is json_path. For contains, not_contains, and equals, set compare_to:\"expected\" to grade against each row's own expected_output (ground truth) instead of a constant value (drop value); add expected_path to dig into the expected value when it is JSON.", "inputSchema": { "properties": { "check_config": { "properties": { "case_sensitive": { "type": "boolean" }, "check_kind": { "enum": [ "contains", "not_contains", "equals", "regex", "valid_json", "json_path_equals", "length_bounds" ], "type": "string" }, "compare_to": { "enum": [ "constant", "expected" ], "type": "string" }, "expected": {}, "expected_path": { "type": "string" }, "json_path": { "type": "string" }, "max": { "type": "integer" }, "min": { "type": "integer" }, "multiline": { "type": "boolean" }, "pattern": { "type": "string" }, "target": { "enum": [ "response_text", "input_data", "json_path" ], "type": "string" }, "target_path": { "type": "string" }, "trim": { "type": "boolean" }, "value": { "type": "string" } }, "type": "object" }, "instruction": { "type": "string" }, "metric_type": { "enum": [ "llm_judge", "check" ], "type": "string" }, "name": { "type": "string" }, "rubric_bands": { "items": { "properties": { "description": { "type": "string" }, "stars": { "type": "integer" } }, "type": "object" }, "type": "array" }, "tag_names": { "items": { "type": "string" }, "type": "array" } }, "required": [ "name" ], "type": "object" } }, { "name": "metrics_update", "description": "Update a metric. For a deterministic check set metric_type:\"check\" and check_config. Per-kind required keys: value (contains/not_contains/equals), pattern (regex), json_path+expected (json_path_equals), min and/or max (length_bounds); valid_json takes no extra keys. target_path is required when target is json_path. For contains, not_contains, and equals, set compare_to:\"expected\" to grade against each row's own expected_output (ground truth) instead of a constant value (drop value); add expected_path to dig into the expected value when it is JSON.", "inputSchema": { "properties": { "check_config": { "properties": { "case_sensitive": { "type": "boolean" }, "check_kind": { "enum": [ "contains", "not_contains", "equals", "regex", "valid_json", "json_path_equals", "length_bounds" ], "type": "string" }, "compare_to": { "enum": [ "constant", "expected" ], "type": "string" }, "expected": {}, "expected_path": { "type": "string" }, "json_path": { "type": "string" }, "max": { "type": "integer" }, "min": { "type": "integer" }, "multiline": { "type": "boolean" }, "pattern": { "type": "string" }, "target": { "enum": [ "response_text", "input_data", "json_path" ], "type": "string" }, "target_path": { "type": "string" }, "trim": { "type": "boolean" }, "value": { "type": "string" } }, "type": "object" }, "id": { "type": "integer" }, "instruction": { "type": "string" }, "metric_type": { "enum": [ "llm_judge", "check" ], "type": "string" }, "name": { "type": "string" }, "rubric_bands": { "items": { "properties": { "description": { "type": "string" }, "stars": { "type": "integer" } }, "type": "object" }, "type": "array" }, "tag_names": { "items": { "type": "string" }, "type": "array" } }, "required": [ "id" ], "type": "object" } }, { "name": "metrics_delete", "description": "Delete a metric", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "metrics_suggest_variants", "description": "Ask the model to rewrite the metric's judge instruction in N variants targeted at the recent disagreements. Each variant is saved as a draft MetricVersion with source=\"suggestion\". Returns the persisted drafts. Stripe-metering hooks fire via ActiveSupport::Notifications under completion_kit.judge_suggestion.generated.", "inputSchema": { "properties": { "count": { "description": "How many variants to request (default 1, max 3). One focused rewrite beats five reworded copies.", "type": "integer" }, "metric_id": { "type": "integer" }, "model": { "description": "Override the model used to generate variants. Defaults to the configured judge model or an available judging model.", "type": "string" } }, "required": [ "metric_id" ], "type": "object" } }, { "name": "metric_groups_list", "description": "List all metric groups", "inputSchema": { "properties": {}, "required": [], "type": "object" } }, { "name": "metric_groups_get", "description": "Get a metric group by ID", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "metric_groups_create", "description": "Create a metric group", "inputSchema": { "properties": { "description": { "type": "string" }, "metric_ids": { "items": { "type": "integer" }, "type": "array" }, "name": { "type": "string" }, "tag_names": { "items": { "type": "string" }, "type": "array" } }, "required": [ "name" ], "type": "object" } }, { "name": "metric_groups_update", "description": "Update a metric group", "inputSchema": { "properties": { "description": { "type": "string" }, "id": { "type": "integer" }, "metric_ids": { "items": { "type": "integer" }, "type": "array" }, "name": { "type": "string" }, "tag_names": { "items": { "type": "string" }, "type": "array" } }, "required": [ "id" ], "type": "object" } }, { "name": "metric_groups_delete", "description": "Delete a metric group", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "metric_versions_list", "description": "List every MetricVersion (drafts + published) for a metric, newest first. Each row carries version_number, state, source, current flag, and timestamps.", "inputSchema": { "properties": { "metric_id": { "type": "integer" } }, "required": [ "metric_id" ], "type": "object" } }, { "name": "metric_versions_publish", "description": "Publish a MetricVersion as the live version of its metric. Works for both 'draft → published' and 'revert to an older published version → current'. Transactionally flips current, demotes peers, and writes the version's instruction + rubric_bands back onto the metric so the judge grades against it.", "inputSchema": { "properties": { "metric_version_id": { "type": "integer" } }, "required": [ "metric_version_id" ], "type": "object" } }, { "name": "metric_versions_dismiss", "description": "Destroy a draft MetricVersion (use for either source: 'edit' or source: 'suggestion'). Published versions are refused — to demote a published version, publish a different one as current instead.", "inputSchema": { "properties": { "metric_version_id": { "type": "integer" } }, "required": [ "metric_version_id" ], "type": "object" } }, { "name": "provider_credentials_list", "description": "List all provider credentials (API keys are not exposed)", "inputSchema": { "properties": {}, "required": [], "type": "object" } }, { "name": "provider_credentials_get", "description": "Get a provider credential by ID (API key is not exposed)", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "provider_credentials_create", "description": "Create a provider credential", "inputSchema": { "properties": { "api_endpoint": { "type": "string" }, "api_key": { "type": "string" }, "api_version": { "type": "string" }, "provider": { "enum": [ "openai", "anthropic", "ollama", "openrouter", "azure_foundry" ], "type": "string" } }, "required": [ "provider", "api_key" ], "type": "object" } }, { "name": "provider_credentials_update", "description": "Update a provider credential", "inputSchema": { "properties": { "api_endpoint": { "type": "string" }, "api_key": { "type": "string" }, "api_version": { "type": "string" }, "id": { "type": "integer" }, "provider": { "type": "string" } }, "required": [ "id" ], "type": "object" } }, { "name": "provider_credentials_delete", "description": "Delete a provider credential", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "tags_list", "description": "List all tags", "inputSchema": { "properties": {}, "required": [], "type": "object" } }, { "name": "tags_get", "description": "Get a tag by ID", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "tags_create", "description": "Create a tag. Color is auto-assigned.", "inputSchema": { "properties": { "name": { "type": "string" } }, "required": [ "name" ], "type": "object" } }, { "name": "tags_update", "description": "Rename a tag.", "inputSchema": { "properties": { "id": { "type": "integer" }, "name": { "type": "string" } }, "required": [ "id" ], "type": "object" } }, { "name": "tags_delete", "description": "Delete a tag. Removes the tag from every linked metric, prompt, run, and dataset.", "inputSchema": { "properties": { "id": { "type": "integer" } }, "required": [ "id" ], "type": "object" } }, { "name": "agreements_list", "description": "List agreements. Filter by run_id, response_id, metric_id, or created_by.", "inputSchema": { "properties": { "created_by": { "type": "string" }, "metric_id": { "type": "integer" }, "response_id": { "type": "integer" }, "run_id": { "type": "integer" } }, "required": [], "type": "object" } }, { "name": "agreements_create", "description": "Upsert an agreement for (run, response, metric, created_by). Verdict is one of agree, disagree, borderline. corrected_score (1..5) is required when verdict is 'disagree'.", "inputSchema": { "properties": { "corrected_score": { "type": "number" }, "created_by": { "type": "string" }, "metric_id": { "type": "integer" }, "note": { "type": "string" }, "response_id": { "type": "integer" }, "run_id": { "type": "integer" }, "verdict": { "enum": [ "agree", "disagree", "borderline" ], "type": "string" } }, "required": [ "run_id", "response_id", "metric_id", "verdict" ], "type": "object" } }, { "name": "judges_replay", "description": "Create a scoring run for the current judge over a dataset's existing outputs (wraps runs_create with prompt_id omitted and output_column supplied). This only sets up the run; call runs_generate to actually re-judge the outputs so you can compare against human verdicts.", "inputSchema": { "properties": { "dataset_id": { "type": "integer" }, "judge_model": { "type": "string" }, "metric_id": { "type": "integer" }, "name": { "type": "string" }, "output_column": { "description": "Dataset column with the existing outputs to grade. Defaults to actual_output.", "type": "string" } }, "required": [ "name", "metric_id", "dataset_id", "judge_model" ], "type": "object" } }, { "name": "judges_compare", "description": "Compare two versions of one metric's agreement stats side by side. Requires metric_id, metric_version_a_id, and metric_version_b_id (both versions must belong to that metric). Unavailable for check metrics.", "inputSchema": { "properties": { "metric_id": { "type": "integer" }, "metric_version_a_id": { "type": "integer" }, "metric_version_b_id": { "type": "integer" } }, "required": [ "metric_id", "metric_version_a_id", "metric_version_b_id" ], "type": "object" } }, { "name": "promptfoo_import", "description": "Import a promptfooconfig.yaml. Creates a prompt, a dataset from the test vars, and metrics from the assert blocks (llm-rubric/g-eval become judge metrics; contains/equals/regex/is-json become deterministic check metrics). Returns a summary of what mapped and what was skipped and why; nothing is dropped silently.", "inputSchema": { "properties": { "config": { "description": "The full promptfooconfig.yaml contents.", "type": "string" } }, "required": [ "config" ], "type": "object" } }, { "name": "usage_get", "description": "Get this organization's plan usage and limits for the current billing period: runs and prompt fetches used, their limits, how many remain, and when the period resets. Call this to pre-check quota before starting runs. Runs are hard-blocked once the run limit is reached (with a small grace band), so a run over the limit will fail with run_limit_reached.", "inputSchema": { "properties": {}, "required": [], "type": "object" } } ] }, "name": "CompletionKit", "note": "Indexed from the official MCP registry: facts and our own checks, not reviewed, so no score, grade or rank.", "packages": null, "pageJsonUrl": "https://www.anchorterminal.com/tools/completionkit-evals.json", "popularity": { "githubStars": 3 }, "registryName": "com.completionkit/evals", "remotes": [ { "type": "streamable-http", "url": "https://completionkit.com/mcp" } ], "repository": "https://github.com/homemade-software-inc/completion-kit", "reviewed": false, "slug": "completionkit-evals", "source": "the official MCP registry", "sourceUrl": "https://registry.modelcontextprotocol.io/v0.1/servers?search=com.completionkit/evals", "summary": "Prompt evals over MCP: run a prompt on your dataset, score each output 1-5 with an LLM judge.", "updatedAt": "2026-07-18T03:09:06Z", "url": "https://www.anchorterminal.com/tools/completionkit-evals", "vendor": "completionkit.com", "vendorUrl": "https://completionkit.com", "version": "1.0.0", "websiteUrl": "https://completionkit.com", "where": "hosted", "why": [ "vendor" ] } }